Adaptive Dialog Robots: Expertise Impact and Human Perceptions

1. The Effect of Expertise on Perceptions of a Directive Robot:The Case for Adaptive Dialog Cristen Torrey Dialog on Dialogs August 25, 2005

2. Opportunities for Adaptation • Multiple users with varying levels of expertise or a single user over some period of time • Possibilities for adaptation • Volume • Intonation • Word Choice • Sentence Structure

3. Research Question • What are the outcomes of adaptation (or a lack of adaptation) by robots? • (Do humans respond to robot adaptation in ways similar to human adaptation?) • Constrained problem • Robot is providing information • Adaptation is manipulating amount of detail (Semantic Elaboration)

4. Related Work (Suggestions?) • Adaptive dialog systems • Johanna Moore • Diane Litman • Sharon Oviatt • Robotics • Cynthia Breazeal

5. Predictions • If one needs more information and does not receive it • less successful • task is perceived to be more difficult • If one does not need more information but does receive it • performance is unaffected • speaker is perceived to be less intelligent, less responsive, less effective, etc.

6. Experiment Design • Pre-test for expertise • 2 X 2 between-subjects design • (expertise X dialog style) • NamesOnly • Description • Task • Selection task requires subjects to choose correct object from set of 6 possible

8. Measures • Performance measures • Time on task • Accuracy (out of 10 possible) • Number of turns • Questionnaire measures • Patronizing • Content appropriateness • Communication effectiveness • Responsiveness • Authority • Conversational control • Task ease • Task Difficulty

9. Results: Number of Questions

10. Results: Items Correct

11. Experiment Two • Recalibrated the dialog • Addition of a visible timer • Monetary incentives for accuracy/speed

12. Results: Number of Questions

13. Results: Patronization Example: “My partner’s explanations can be condescending.” Cronbach’s alpha = .90 (4 items)

14. Results: Conversational Effectiveness Example: “I found the conversation to be very useful and helpful.” Cronbach’s alpha = .90

15. Results: Authority Example: Expert/Inexpert, Reliable/Unreliable Cronbach’s alpha = .72 (4 items)

16. Marginal Effects • Task enjoyability (3 items) (p=.12) • “I enjoyed participating in this task” • Responsiveness (5 items) (p=.08) • “My partner can adapt to changing situations” • Content appropriateness (3 items) (p=.10) • “My partner’s explanations were just right for someone like me”

17. Summary: Impact of Time • Expected results only exist under time pressure • Under pressure, people may revert to expectations they have for people

18. Future Work • Repeated exposure (novelty effect) • Different devices, agents, etc. • New population • Older adults and negative outcomes of being “talked down to” • Lots of variability in the population

19. Older Adults & Adaptation

20. Tools Correct: Study Two