Interaction Challenges for Intelligent Assistants

1. Jim Blythe USC Information Sciences Institute Interaction Challenges for Intelligent Assistants

2. How to build “truly useful assistants”? Personalized, Learn, Engender trust, Become partners Organizer: Neil Yorke-Smith Committee: Pauline Berry, Timothy Bickmore, Mihai Boicu, Justine Cassell, Ed Chi, Mike Cox, John Gersh, Jihie Kim, Jay Modi, Donald Patterson, Debra Schreckenghost, Richard Simpson, Stephen Smith, Sashank Varma 28 accepted papers

3. Topics • Trust • When to assist? • Learning • Modeling • Desktop assistants • Panel with symp. on multidisciplinary collaboration for socially assistive robots • Panel with intentions in intelligent systems

4. How To Make Users Happy • And avoid annoying users - Brad Myers’ invited talk

5. User Happiness? Hu = f (Performance)

6. User Happiness? Hu = f (Performance, Trust)

7. User Happiness! Hu = f (EAssistant ENegative EPositive EValue EUser ECorrected EBy-hand ECost EAvoided EApparentness ECorrect-difficulty ESensible WQuality WCommitment TBy-hand TBy-Hand-start-up TBy-Hand-per-unit TAssistant TTraining-start-up TAssistant-per-unit TInteraction-per-unit TMonitoring TCorrecting TResponsiveness TSystem-Training TUser-training TAverage-for-each-correction AError-rate Nunits PPleasantness UPerceive UWhy UProvenance UPredictability IAssistant-interfere IScreen-space ICognitive IAppropriate-Time CAutonomy CCorrecting SSensible-Actions SUser-models SLearningRSocial-Presence DHand VImportance)

8. The Effort required of the pilot to control the associate must be less than the effort saved by the associate The Pilot is ALWAYS in charge. A Tale of Two Associates • Pilot’s Associate (1985-1991) • Single Pilot • Direct pilot interaction with associate meant added workload • Design philosophy minimized direct pilot interaction with associate • Moderate user acceptance • Rotorcraft Pilot’s Associate (1994-1999) • Two Pilots • 1/3 of human activity is crew coordination • Design philosophy included some direct pilot interaction with associate • Improved User Acceptance

9. Why and how to model multi-modal interaction for a mobile robot companion Shuyin Li & Britta Wrede Best paper • Tested policies with users interacting with a robot • Communicate pre-interaction attention • Need to make social remarks with non-verbal methods (because people tend to reply in kind) Biron and Barthoc

10. Interaction Challenges for Agents with Common Sense Invited talk from Henry Lieberman • We now have several sources of common sense knowledge, e.g. Cyc, Open Mind, ThoughtTreasure • Some strategies and examples of exploiting common sense to build better interfaces

11. Strategies for using common sense in interfaces • Find underconstrained situations • Find situations where every little helps • Know a little about everything, but not too much about anything • Make better mistakes! Not just ‘right’ and ‘wrong’, being reasonable is better • Plausible mistakes can increase trust • Set user expectations

12. Examples of interfaces using common sense • ARIA photo agent: more powerful matching of tags using common sense • Predictive typing: “I’m having landlord problems because my roommate was late with my r..” • BEAM (Gil & Chklovski)

13. Trust • Openness and understanding more important as systems become more complex. • Methods to improve understanding: explanations [McGuinness et al.] • HTN metamodel [Wallace] • Patterson: would I trust a fork? a bridge? a space shuttle? • predictability, understandability, similarity, liability, social/emotional

14. Learning (and Trust) • Adaptive (Learning) vs Adaptable (Instructed by user) • important for believability and trust

15. The Interaction Skills The Motion Skills Supporting interaction in Robocare intelligent assistant agent Cesta et al. Best application paper Use of multiagent technology Endowed with human like I/O channels by engineering state of the art components • Face: Lucia (Piero Cosi, ISTC, Pd) • Voice: Sonic (Univ.Colorado) • Simple Interaction Manager Robust continuous behavior at home with person

16. Multiple Intelligent Systems

17. Supporting interaction in Robocare intelligent assistant agent Integrates multiple systems to produce a socially acceptable robotic care assistant • Interesting DCOP solution to allow multiple systems and guarantee coherent behaviour • System follows a STN to notice deviations from expected behaviour

18. Experiments in face/no-face in RoboCare • People prefer no-face • “less artificial”, “more integrated in the domestic environment”

19. Desktop assistants • Many papers on desktop assistants • 6 from the Calo project PeXA architecture

20. Towel todo manager • Towel [Conley et al]: taking an IM approach to give access to tasks Inspired by Diamond Help [Rich et al. 06]

21. Did Ken sacrifice himself to User Testing? • Registered to give talk at AAAI Spring symposium

22. Should Ken have worked on meeting scheduling? • Registered to give talk at AAAI Spring symposium • Booked another trip in same week

23. Should Ken have worked on meeting scheduling? • Registered to give talk at AAAI Spring symposium • Booked trip to Hawaii in same week • Ultimate in user testing? You decide..