1 / 57

The Effectiveness of Haptic Cues as an Assistive Technology for Human Memory

The Effectiveness of Haptic Cues as an Assistive Technology for Human Memory. Stacey Kuznetsov , Anind K. Dey , Scott E. Hudson, Eric Paulos { stace , anind , hudson , paulos } @ cs.cmu.edu. Human- Computer Interaction Institute. Carnegie Mellon. Living Environments Lab.

ira
Download Presentation

The Effectiveness of Haptic Cues as an Assistive Technology for Human Memory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Effectiveness of Haptic Cues as an Assistive Technology for Human Memory Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  2. Human Memory Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon

  3. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon

  4. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon

  5. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon

  6. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon

  7. Memory Impairments Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  8. Visual and Audio Cues DeVaul, R.W., Pentland, A. and Corey, V.R. The memory glasses: subliminal vs. overt memory support with imperfect information. (2005) Disruptive Lin, W. and Hauptmann A.G.A Wearable Digital Library of Personal Conversations. (2002) Prominent Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  9. Vibro-Tactile Cues Unobtrusive Discreet Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  10. Vibro-Tactile Interaction Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  11. Hypothesis Vibro-tactile cues can improve human memory recall and recognition Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  12. Effective cross-modal priming for visual and haptic stimuli Priming Ballesteros S., Reales J.M., Mayas J. and Heller M.A. Selective Attention modulates visual and haptic repetition priming: effects in aging and Alzheimer's disease. (2008) Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  13. Haptic feedback aids attention and navigation Attention and Navigation Young, J.J., Tan, H.Z., Gray, R. Validity of haptic cues and its effect on priming visual spatial attention. (2003) Kildal, J. and Brewster, S.A. VibroTactile External Memory Aids in Non-Visual Browsing of Tabular Data. (2006) Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  14. Multimodal Priming Retrieval of audio and video information is assisted with vibro-tactile cues Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  15. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  16. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  17. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  18. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  19. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  20. Wearable Haptic Feedback Device Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  21. Cue 1: two medium pulses Cue 2: strong oscillating pulses Example Cues Cue 3: short soft pulse Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  22. Training Phase Training Phase Procedure Pause Pause Test Phase Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  23. Method Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  24. “Blending” Audio Recognition Visual Recognition Free Recall Four User Studies Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  25. Blending Shape Recognition Test 24 participants, 15 male, 9 female, ages 21-65 Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  26. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  27. Auditory Recognition Mandarin word translation test 20 participants: 12 male, 8 female, ages 21-65 Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  28. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  29. Visual Recognition Face memory test 22 participants: 16 male, 6 female, ages 21-65 Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  30. Visual Recognition Test Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  31. Free Recall Ebbinghaus nonsense syllable test (“hik”, “lup”) Ebbinghaus, H. Memory: A Contribution to Experimental Psychology. Teacher’s College (1913). Dover Publications. 24 participants: 16 male, 8 female, ages 21-65 Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  32. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  33. Results Audio recognition Shape recognition Face recognition Free recall +11% (5 trials, p= 0.075) no effect no effect -20% (7 trials, p= 0.009) Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  34. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  35. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  36. Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  37. Significantly lower performance with cues Haptic Cues and Free Recall Declarative vs. Perceptual memory Our cues did not have semantic meaning Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  38. Cues were ‘distracting’ Haptic Cues and Cognitive Load Perceptual differences between users Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  39. Memory aid for people with poor or impaired memory Haptic Cues as a Memory Aid Memory enhancing treatments often target low performers Aldenkamp, A. P., et al., Effect of Sabeluzole (R 58 735) on Memory Functions in Patients with Epilepsy. (1995) Mattay, V. S. et. al.. Effects of Dextroamphetamine on Cognitive Performance and Cortical Activation. (2000) Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  40. Cue vocabulary Opportunities Maximum number of cues Effectiveness of cues for people with functional memory impairments Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  41. Pulse 1 Pulse 2 Pulse Duration Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  42. Pulse 1 Pulse 2 Pulse Intensity Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  43. Pulse 1 Pulse 2 Delay Between Pulses Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  44. Cue Vocabulary Study 14 Participants, age 18-45 Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  45. Ratings of Pulse Duration Differences Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  46. Ratings of Pulse Intensity Differences Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  47. Ratings of Differences in Delay Between Pulses Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  48. Systematically pick cues from the sampled space Ongoing Work Determine the maximum number of cues that can be remembered Recruit subjects with functional memory impairments Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  49. Adjust cue intensity to individual’s perception Future Work Add semantic meaning to cues Strive for context-awareness Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

  50. The Effectiveness of Haptic Memory Cues Haptic cues as a memory aid for poor performers Assistive technology for memory-impaired users Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos {stace, anind, hudson, paulos} @ cs.cmu.edu Human- Computer Interaction Institute Carnegie Mellon Living Environments Lab

More Related