A Helping Hand in Communication

1. A Helping Hand in Communication The future of communication for the hearing and vision impaired Mr Iain Murray Dr Euan Lindsay

2. A day in the life of John • John is • A student • Shares a house with two friends • Enjoys shopping • Keeps in touch with friends and family • Totally Blind

3. Colours • Inexpensive • Accurate (enough) • Long battery life • Compact

4. Currency Identifier • Australian Polymer notes are of a similar size • Hard to distinguish denomination if totally blind • Utilises the unique pattern in the clear window

5. A Secure Dual Channel Wireless Headset for Multi-user Environments • Dual channel, two-way, high quality, low power infrared headset prototype which has been field tested by the ABWA and certified.

6. Current Form Ideal Form

7. Braille • Not a direct correlation to text • Similar to shorthand • Consists of 6 dots that may be embossed on both sides

8. Curtin University Brailler (CUB) • Perkins Brailler • Mechanical only • Mountbatten Brailler • Electronic, very old technology

10. Curtin University Brailler

11. Braille Scanner • Aimed at Sighted non-Braille Readers • Converts Braille to Expanded Text in real time

13. Cisco Network Academy Program • The Networking Academy program is an e-learning model that delivers Web-based educational content, online testing, student performance tracking, and instructor training and support, as well as hands-on labs. (Cisco,2002)

14. Cisco Network Academy Program (CNAP) • 10,312 Academies • 1229 in Asia-Pacific • 162 Countries • 25 Countries in Asia Pacific • 97,488 students in Asia Pacific (as of 23rd April 2006)

15. Classmates

16. Cisco Network Academy Program

17. Current Access Methods • For low vision users. • Screen enlargement. • Zoomtext, Magic. • Screen review programs. • Jaws, Slimware, Artic. • Two output methods - speech and Braille displays. • Screen review software must rely on text output. • Tactile Graphics • PIAF • Printed Braille and Audio recordings

18. Network Dominoes

19. Tactile Printing

20. Whiteboard • Cheap camera • <$150 • Portable • Multiple use of laptop

21. iNetSim • Simulates a network • Allows for trials of configurations without expensive equipment • Works with voice output

22. What about John’s Text Books? • Daisy Digital Talking Book (DTB) compatible • Full Indexing • Annotations • Text • Audio • Video • High speed capture • 90 minutes in 4 minutes • DRM enabled

23. What about Tony?

24. Sign Language • Common for hearing impairment • Uses Vision to replace sound • Many types • Auslan • ASL • BSL But what if you don’t have vision either?

26. A Demonstration

27. Some Obvious Challenges… • Need an interpreter • Need a fluent interpreter • Slow to start with

28. … and some not obvious ones • One-to-one • In-person • Slang / Dialects

29. An Alternative

34. Removes Barriers • Communicate with non-signers • Through their choice of interface • Direct interaction, without an interpreter • Can talk to people in other rooms / cities / countries • Multiple people at once • Recording

35. The Hand is A Complex System

36. The Current Hand

37. All good

38. The challenges ahead… • Servomotors instead of air muscles • Artificial Intelligence to learn trajectories for the hand signs • Learn by mimicking

39. More challenges • Communication protocol • Correct data vs On-Time data • Social / Cultural Implications • “SMS” speak?

40. The Team(in no particular order) • Staff • Chris Moore • Serge Mokroous • Kieren Eaton • Leigh Harrison • Students • Andrew Pasquale, Simon Dunn, Ryan Williams, Carl Blair, James Hope, Leong Chen Chew, Ben Farrugia, Ivar Siewert, Peter Scarfe

41. Thank You