Virginia Tech Plans with SenseCam: Early Ideas Microsoft Memex Workshop, July 2006

1. Virginia Tech Plans with SenseCam: Early IdeasMicrosoft Memex Workshop, July 2006 Beyond Human Memory: SenseCam Use in Veterinary College and as Assistive Technology Manuel A. Pérez-Quiñones, Edward A. Fox

2. Virginia-Maryland Regional College of Veterinary Medicine • Context of student life at the Vet School • Goals: • improve the recall of clinical procedures studied in lab sessions • promote learning through sharing of information among students • which should result from integration of • audio/photos from multiple SenseCams, • video from the professor, and • live notes taken by several students • Sharing: • Students will have the option of providing access to particular pieces of information by particular users. • Information will be uploaded automatically to a web server where it will be available to other students for download into their MyLifeBits data store.

3. Students with Disabilities • Help students with disabilities in their day-to-day activities on campus • Use location tracking and some form of simple tagging of the audio or video data to help students with prospective remembering • Allow students with motor disabilities to share data with their care-giver and/or with the staff of the Assistive Technologies office • Track locations indoors with 802.11 device

4. Original Timeline

5. Human Information Processing with the Personal Memex Ingrid Burbey, Gyuhyun Kwon, Uma Murthy, Nicholas Polys, Prince Vicent Human Information Processing Project Presentation

6. Overview 1. Project Overview Literature Search: HIP Principles Scenarios Expert Interviews Task Analysis Initial Prototype Cognitive Walkthrough with experts Annotated Bibliography Design Guidelines Revised Prototype

7. Overview 1. Project Overview Analysis Literature review Expert Interview Studies of current related systems Brainstorming Iterative analysis and design based on Usability claims and HIP principle Design Scenario generation Function decomposition Task decomposition Specifying interaction Prototype& Evaluation Low fidelity prototype Cognitive walkthrough

8. 1. Cognitive Workthrough Process Cognitive Walkthrough System Requirements HIP/Usability Principle Performing Cognitive Walkthrough Test Scenario Description Record usability problems found Memex Prototyping Fix the usability problems

9. Human Information Processing: Memory • The Multi-Store Model (Atkinson & Shiffrin, 1968) • Encoding, Storage, Retrieval

10. Human Information Processing: Long-term Memory • Types of LTM (evidence from Amnesiacs) • Semantic: Knowledge & Facts • Episodic: Events & Experiences • Procedural: Skill • Aspects of LTM • Explicit aka ‘declarative’ • Implicit aka ‘non-declarative’

11. Human Information Processing: Long-term Memory • Structure of LTM: Schemas • Developed by experience and pre-programming; accommodation & assimilation • Prototypical elements are always given priority • Can be applied non-consciously • Shared across individuals within a culture • Stable over time

12. Human Information Processing: Long-term Memory Recall can be confounded by: • Proactive interference • prior learning interferes with subsequent learning • Retroactive interference • post hoc learning confounds the prior learning

13. Human Information Processing: Long-term Memory Models of LTM retrieval • Associative Memory • Levels of Processing • Cue-dependent forgetting • Encoding Specificity Principle : Context !!! • Priming & Cues : visual, verbal, …

14. Human Information Processing: Working Memory • Working Memory, Baddeley, 2003

15. Symptoms of Macular Degeneration • Difficult to see colors • Difficulty recognizing faces • Prefer dim backgrounds

16. Mild Cognitive Impairment • Due to Hippocampus shrinking • No universally, accepted definition • Symptoms • Loss of short-term memory • Forgetfulness • Deteriorates over time • Difficulty in associative memory • Remembering names of people or things

17. Design Guidelines Design Guidelines • Formulated in the tradition of Nielsen, Tognazzini • Discovered or confirmed through: • Literature search • Expert interviews • Cognitive walkthrough

18. Design Guidelines General Results • Maintain consistent, meaningful graphics • Support opportunistic navigation • user control and freedom • ‘Bookmarking’ paths, previous searches • Leverage user recognition rather than recall • provide context for users while searching and browsing memories • Provide visibility for mode & system status • Provide Internationalized Tool Tips for icons and buttons • Provide Help w/o loosing state • Provide Tutorials explicitly demonstrating functionality

19. Design Guidelines Population-specific Results: Mild Cognitive Impairment • Simplify task steps • Provide views that can represent time and objects graphically • Provide time, space, and object reminders (for schedules, directions, medication)

20. Design Guidelines Population-specific Results: Visual Impairment • Provide enlargeable fonts & interface items (e.g., buttons) • Provide contrast between font and background • Provide options for audio output: voice, speed, pitch

21. Prototype

35. General Comments • Integration with other applications/devices • Date/time on main screen • Mouse-over magnification • Help menu item • Prototypical terms • Scroll forward to future events

36. Tablet Idea • Monday Tablet session • Discussion that people aided by assistive technologies are delayed a long time before the right affordances arise • Idea that SenseCam could be used along with a Tablet • in synergistic capture, analysis, design, deployment methodology

37. Summary and Conclusions • Two planned efforts • Student project report • Tablet idea • Looking forward to new version! • Hoping Microsoft’s many projects (search, image processing, Tablet, etc.) can be integrated with MyLifeBits • Hope Microsoft staff can help run a JCDL workshop on personal DLs in 2007 • Open for collaboration • fox@vt.edu