HelpMeSpeak ! An email access system for dysarthric speakers.

1. HelpMeSpeak!An email access system for dysarthric speakers. Ganesh Arumugam CCIS MS 2014 Stephen Flaherty PHI-CCIS PhD candidate MansoorPervaiz PHI-CCIS PhD candidate Zhichun Ye CCIS-MS 2014

2. The problem • People with various Neurological Speech Motor disorders have impairments that limit their ability to communicate.

3. The problem • These people suffer from Dysarthria - a motor speech disorder resulting from neurological injury that makes if difficult to pronounce words. (wikipedia)

4. The problem • Coupled with their inability to use physical input devices, dysarthric speakers are not able to use standard computer interfaces and can struggle with assistive devices (AACs)(Best &Butler 2012).

5. Our user population • Adult residents of The Boston Home-a residence and care center for adults with progressive neurological diseases. www.thebostonhome.org

6. Motivation • Residents at Boston Home find it difficult to communicate with their • caregivers (in one to one communication) • family members (through email and skype)

7. Motivation • Currently available systems are cumbersome and difficult for people with speech and cognitive impairment to use (Hux, et al. 2000). • Enable dysarthric speakers to create, send and view emails using only voice commands.

8. Motivation • Provide a customizable system to recognize their individual vocal characteristics and fit their needs. • Greatly reduce the time and effort needed for our target users to communicate with their friends and family.

9. Surprises • In our ethnographic studies, we discovered that our target user population was very interested in using computers to interact with friends and family or surf the web.

10. Surprises • Our target users do not want devices that speak for them. • They want to continue to use their own voice as long as they can.

11. The best today • AAC devices do not fit our users needs well. • Require physical touch or eye tracking • Slow • Expensive Dynavox Maestro, www.dynavoxtech.com

12. Standard Speech Recognition • Dragon Naturally Speaking (www.nuance.com) • Windows Speech Recognition

13. Standard Speech Recognition • Windows Speech Recognition (www.windows.microsoft.com) • Difficult to set up-requires physical interaction • Tedious and tiring to use

14. Standard Speech Recognition • Grid Systems • Slow and awkward to use. www.windows.microsoft.com

15. Innovation • We propose a very simple email system that: • Uses pattern recognition instead to regular speech recognition. • Sends an attached voice message instead of text in the email.

16. Innovation • It was taking our target population at least 40 minutes to write a short email. • Our users were keen to use their own voice and they are understandable to their family and friends.

17. Demo • HelpMeSpeakhttp://www.ccs.neu.edu/home/zhichun/team/T8/Home.html

18. User Testing Video

19. A UI challenge • Original design was only a few screens that provided access to many functions, the idea to reduce navigation and put all the commands in front of the user.

20. A UI challenge • However, during prototyping the abundance of options proved confusion to our users. • Users only wanted the most critical functions.

21. A UI challenge • We had to scale back on features and increase the number of different pages in the application. • The result was a much sparser layout with larger buttons.

22. Tricky programming problem • Provide a popup message when the user gave a command that was not recognized by the system. • This proved difficult to implement • Expect the system to be always on in the subject’s room • Could trigger off of any conversations in the room.

23. Tricky programming problem • Require ability to distinguish specific commands from continuous speech and take the correct action.

24. Tricky programming problem • Our system is based on pattern recognition of recorded commands for each user, not live speech recognition and interpretation.

25. Evaluation method • Paper prototype testing with 3 subjects at Boston Home. • Medium fidelity prototype testing at Boston Home • Methods • Standard performance tasks • Observation • Interviews

26. Redesign and evaluation • Initial application scope scaled down based on user feedback and prototyping. • Limited visual cognitive ability of user a huge factor. • User testing with 3 subjects at Boston Home

27. Redesign and evaluation • Minimalist design with clear functionality. • Reduce cognitive load on user • Seek high visibility of features. • Audio cues • Feedback from Team 6 helped identify some consistency issues and broken components.

28. The future • Design limitations • Minimal design look and feel could be improved with more development time. • Functionality and features can be refined with more user testing. • Improve user voice control options for recording and navigation • User access was our biggest challenge

29. Our future process • Spend more time field testing prototypes with our user group. • Understand that the system is based around what they want and not what we want.

30. Our future process • Implement a more functional prototype over the summer • If successful prepare a CHI paper for fall submission

31. Acknowledgements • Images: • www.nuance.com • www.microsoft.com • The Boston Home www.thebostonhome.org • Thanks to Team 6 for design feedback! • Coding resources: • www.w3schools.com

32. References • Best, K., & Butler, S. (2012). Disability and Communication: A Consideration of Cross-disability Communication and Technology. Disability Studies Quarterly,32(4). • Hux, K., Rankin-Erickson, J., Manasse, N., & Lauritzen, E. (2000). Accuracy of three speech recognition systems: Case study of dysarthricspeech.Augmentative and Alternative Communication, 16(3), 186-196.

33. HelpMeSpeak! • Web-based email system for dysarthric speakers • Simple design for physically and visually impaired users • Customizable for the user’s voice and other needs