1 / 33

HelpMeSpeak ! An email access system for dysarthric speakers.

HelpMeSpeak ! An email access system for dysarthric speakers. Ganesh Arumugam CCIS MS 2014 Stephen Flaherty PHI-CCIS PhD candidate Mansoor Pervaiz PHI-CCIS PhD candidate Zhichun Ye CCIS-MS 2014. The problem.

lenka
Download Presentation

HelpMeSpeak ! An email access system for dysarthric speakers.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. HelpMeSpeak!An email access system for dysarthric speakers. Ganesh Arumugam CCIS MS 2014 Stephen Flaherty PHI-CCIS PhD candidate MansoorPervaiz PHI-CCIS PhD candidate Zhichun Ye CCIS-MS 2014

  2. The problem • People with various Neurological Speech Motor disorders have impairments that limit their ability to communicate.

  3. The problem • These people suffer from Dysarthria - a motor speech disorder resulting from neurological injury that makes if difficult to pronounce words. (wikipedia)

  4. The problem • Coupled with their inability to use physical input devices, dysarthric speakers are not able to use standard computer interfaces and can struggle with assistive devices (AACs)(Best &Butler 2012).

  5. Our user population • Adult residents of The Boston Home-a residence and care center for adults with progressive neurological diseases. www.thebostonhome.org

  6. Motivation • Residents at Boston Home find it difficult to communicate with their • caregivers (in one to one communication) • family members (through email and skype)

  7. Motivation • Currently available systems are cumbersome and difficult for people with speech and cognitive impairment to use (Hux, et al. 2000). • Enable dysarthric speakers to create, send and view emails using only voice commands.

  8. Motivation • Provide a customizable system to recognize their individual vocal characteristics and fit their needs. • Greatly reduce the time and effort needed for our target users to communicate with their friends and family.

  9. Surprises • In our ethnographic studies, we discovered that our target user population was very interested in using computers to interact with friends and family or surf the web.

  10. Surprises • Our target users do not want devices that speak for them. • They want to continue to use their own voice as long as they can.

  11. The best today • AAC devices do not fit our users needs well. • Require physical touch or eye tracking • Slow • Expensive Dynavox Maestro, www.dynavoxtech.com

  12. Standard Speech Recognition • Dragon Naturally Speaking (www.nuance.com) • Windows Speech Recognition

  13. Standard Speech Recognition • Windows Speech Recognition (www.windows.microsoft.com) • Difficult to set up-requires physical interaction • Tedious and tiring to use

  14. Standard Speech Recognition • Grid Systems • Slow and awkward to use. www.windows.microsoft.com

  15. Innovation • We propose a very simple email system that: • Uses pattern recognition instead to regular speech recognition. • Sends an attached voice message instead of text in the email.

  16. Innovation • It was taking our target population at least 40 minutes to write a short email. • Our users were keen to use their own voice and they are understandable to their family and friends.

  17. Demo • HelpMeSpeakhttp://www.ccs.neu.edu/home/zhichun/team/T8/Home.html

  18. User Testing Video

  19. A UI challenge • Original design was only a few screens that provided access to many functions, the idea to reduce navigation and put all the commands in front of the user.

  20. A UI challenge • However, during prototyping the abundance of options proved confusion to our users. • Users only wanted the most critical functions.

  21. A UI challenge • We had to scale back on features and increase the number of different pages in the application. • The result was a much sparser layout with larger buttons.

  22. Tricky programming problem • Provide a popup message when the user gave a command that was not recognized by the system. • This proved difficult to implement • Expect the system to be always on in the subject’s room • Could trigger off of any conversations in the room.

  23. Tricky programming problem • Require ability to distinguish specific commands from continuous speech and take the correct action.

  24. Tricky programming problem • Our system is based on pattern recognition of recorded commands for each user, not live speech recognition and interpretation.

  25. Evaluation method • Paper prototype testing with 3 subjects at Boston Home. • Medium fidelity prototype testing at Boston Home • Methods • Standard performance tasks • Observation • Interviews

  26. Redesign and evaluation • Initial application scope scaled down based on user feedback and prototyping. • Limited visual cognitive ability of user a huge factor. • User testing with 3 subjects at Boston Home

  27. Redesign and evaluation • Minimalist design with clear functionality. • Reduce cognitive load on user • Seek high visibility of features. • Audio cues • Feedback from Team 6 helped identify some consistency issues and broken components.

  28. The future • Design limitations • Minimal design look and feel could be improved with more development time. • Functionality and features can be refined with more user testing. • Improve user voice control options for recording and navigation • User access was our biggest challenge

  29. Our future process • Spend more time field testing prototypes with our user group. • Understand that the system is based around what they want and not what we want.

  30. Our future process • Implement a more functional prototype over the summer • If successful prepare a CHI paper for fall submission

  31. Acknowledgements • Images: • www.nuance.com • www.microsoft.com • The Boston Home www.thebostonhome.org • Thanks to Team 6 for design feedback! • Coding resources: • www.w3schools.com

  32. References • Best, K., & Butler, S. (2012). Disability and Communication: A Consideration of Cross-disability Communication and Technology. Disability Studies Quarterly,32(4). • Hux, K., Rankin-Erickson, J., Manasse, N., & Lauritzen, E. (2000). Accuracy of three speech recognition systems: Case study of dysarthricspeech.Augmentative and Alternative Communication, 16(3), 186-196.

  33. HelpMeSpeak! • Web-based email system for dysarthric speakers • Simple design for physically and visually impaired users • Customizable for the user’s voice and other needs

More Related