Raj Reddy Carnegie Mellon University Pittsburgh, PA 15213 January 21, 2010 rr.cs.cmu

1. To Err is HumanComputational Limits to Human Thinking : Implications for the Design of Human Centered Interfaces Raj Reddy Carnegie Mellon University Pittsburgh, PA 15213 January 21, 2010 www.rr.cs.cmu.edu

2. Outline of Talk • Limitations of Current Interfaces from the Perspective of Cognitive Science • Missing Science of Multimedia Interfaces • Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

3. Outline of Talk • Limitations of Current Interfaces from the Perspective of Cognitive Science • Missing Science of Multimedia Interfaces • Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

4. Human Strengths and Human LimitationsLessons from Cognitive Science • Strengths: People • communicate using speech and natural language • tolerate errorful, ambiguous and imprecise input • exploit vast amounts of knowledge • learn from the environment • Limitations: People • make errors • tend to forget • become impatient • tend to be lazy

5. To Err Is Human • Problem: • Catastrophic loss of data: “I didn’t mean to do that ” • Unanticipated side-effects: “how did that happen??” • Causes: • Sensory, Cognitive and Motor overload • Information Overload: “Like being in a traffic jam” • Timing Errors: Simplify the task • Present Solutions: • Partial Undo • Dialog Boxes • Future Opportunities: • Unlimited Undo • Anytime Anywhere Abort • DWIM (Do What I Mean)

6. To Forget Is Human • Problem: • A non-expert occasional user can’t be expected to remember the details • Causes: • Forgetting is the loss of indexing structure • Redundancy in the indexing structure is the key • Present Solutions: • Recognition vs. recall: GUIs and Menus • On-line manuals • Future Opportunities: • Use of color, fonts, voice responses for focusing attention • Intelligent Help • “How do I” and “What if” MultiMedia documentation • “Reference librarian” agent

7. To be Impatient Is Human • Problem: • Time to get the answer in interactive problem solving • Long Start-up and Shutdown Times • Network operations: Unpredictable retrieval and browsing times • Present Solutions: • Hour glasses and wheels • No idea how long it will take • Progress bars • Future Opportunities: • Updates in the background • Learn from experience: self-aware systems • Look ahead retrieval and computation • Hurry-up algorithms • Keystroke model

8. To be Lazy Is Human • Problem: • Most people use a minimal subset of functionality in Word, PowerPoint, etc. • Most people avoid tasks requiring too much cognitive effort • PGP - too much work • FTP - too complex • Causes • Principle of Least Effort • Future opportunities: • Advice giving agents that look over your shoulder • Just-in-time learning • Gentle slope systems: Easy Things should be Easy to Do • Agents (wizards!) that know about PGP, FTP, or whatever

9. Outline of Talk • Limitations of Current Interfaces from the Perspective of Cognitive Science • Missing Science of Multimedia Interfaces • Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

11. Missing Science of Multimedia InterfacesLessons from Flash Forward Video • SILKy Interfaces • Speech, Image, Language and Knowledge • Forgiving Interfaces: Tolerate error and ambiguity • “You Bet” --- “Did That Mean Yes?” • Self Improving Interfaces: Incremental non-intrusive knowledge acquisition • “Darn” --- “Would you Care to Define It?” • Intelligent Help, Advice and Tutoring • “ Enthusiasm Detected --- Care to View Large Screen?” • Intelligent Agents • Mail Serve: Junk Mail • Communications: Telephone Tag • Intelligent Tutoring Tools • Agent Creation Technology

12. Outline of Talk • Limitations of Current Interfaces from the Perspective of Cognitive Science • Missing Science of Multimedia Interfaces • Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

13. User Interfaces for Rural Environments • Low cost PC: target of less than $100 • A Multifunction Information Appliance: PCtvt • PCtvt: TV, PVR, Video Phone, IP Phone and PC • PCs for consumption, not creation • For Most People in a Village, Entertainment and Communication are of Greater Importance than PC Functionality

14. PCtvt UI Design for Use by Illiterate Persons • An Illiterate person needs a more powerful PC than a Microsoft employee • If not e-mail, use voice-mail • Replace Text Help by Video Help • Radically simple design • One minute learning time • Two click model • Three modes of communication: Video, Audio and Text • Both Synchronous and Asynchronous • All-Iconic interfaces • Multiple input modalities • TV-remote, Speech I/O, Keyboard, Mouse or Cell phone

15. Mapping TV Remote to Iconic Screen Layout 2 3 1 3 1 2 5 6 4 6 4 8 9 7 7 9 0 9 BACK MENU OK OK

16. Video Phone Menu TV Menu 3 2 5 6 4 9 7 0 OK OK

17. The Unfinished Research Agenda • Add Features to Assist in Overcoming Human Limitations such as “Unlimited Undo”, Instant On and Off, and Multimedia Help. • WIMPy to SILKy: Move Towards Intelligent Multimedia Interfaces • Eliminate Language Divide and Literacy Divide in Future Systems: Interfaces that Enable Access to People at the Bottom of the Pyramid