Automated Caricature of Robot Expressions in Socially Assistive Human-Robot Interaction

1. Automated Caricature of Robot Expressions in Socially Assistive Human-Robot Interaction Ross Mead and Maja J Matarić Presented by David Feil-Seifer Interaction Lab University of Southern California

2. Outline • Motivation and Background • Approach and Methods • Robot Platform • Experimental Design • Q & A

3. Motivation and Background • Children with autism spectrum disorder (ASD) suffer from a deficit that prevents them from observing, interpreting, and learning social cues. • Clinical studies in social skills training have proposed methods, such as exaggeration, to enhance autism intervention strategies (Rao et al., 2008). • Socially assistive robotics has potential to improve social activity (Feil-Seifer & Matarić, 2005). • The Transporters is an animated series that has shown an improvement in face-based emotion recognition in children with ASD (Golan et al., 2009)… • However, expressive behaviors of the body still remain to be addressed.

4. Approach and Methods • Take inspiration from several principles of animation (Thomas & Johnston, 1981)… • Staging • Exaggeration • Anticipation • Secondary Action • Goal: Automated caricature of social interaction behaviors of robots.

5. Approach and Methods — Staging • Aims to provide some grounding for robot gestures in subsequent operations. • Process of presenting a communicative act in as clear a way as possible… • Minimize or eliminate conflicting signals. • Isolate features that uniquely identify the content of the expression. • Caricaturing highlights such features, providing building blocks for expressions. • Hypothesis: Well-staged expressions provide more clarity to a child with ASD than poorly-staged expressions. Source: http://familypants.files.wordpress.com/2009/02/preston11.jpg

6. Approach and Methods — Exaggeration • Amplify distinct features that identify the expression to make content of the behavior more convincing and/or explicit. • Use feature parameterizations isolated during the staging process to produce exaggerated expressions. • Hypothesis: A child with autism will be more capable of interpreting an exaggerated expressive behavior than a non-exaggerated expressive behavior… • Supported by the peak shift principle (Ramachandran & Hirstein, 1999). Source: http://www.animationbrain.com/2D/38principle.JPG

7. Approach and Methods — Anticipation • Anticipatory action often indicates or emphasizes the intent of the character. • Staging and exaggeration provide insights pertaining to the dynamics of a communicative act… • We aim to utilize this to automatically generate motion paths for both micro and macro anticipatory expressions that precede it. • Hypothesis: Anticipatory actions will provide a child with ASD a better understanding of the intent of his or her social partner than non-anticipatory action. Source: http://www.evl.uic.edu/ralph/508S99/gif/batter.gif

8. Approach and Methods — Secondary Action • The use of redundant signals in an expression to better communicate an idea. • Redundancy plays a key role in social interaction (Birdwhistell, 1970). • Signals must be isolated into distinct parts for proper staging, exaggeration, and anticipation to occur. • We suspect that secondary action has potential with high-functioning children with autism (particularly, those who have participated in the Transporters studies)… • Might be overwhelming for children that are far in the spectrum. Source: http://www.awn.com/tooninstitute/lessonplan/images/walk17.jpg

9. Robot Platform • Currently being implemented on the Sparky Minimatronic™ robot figure… • Courtesy of Walt Disney Imagineering Research & Development. • Uses two servo controllers and 18 servos… • Lightweight and highly dexterous. • Movements are fluid and natural. • Articulated spine allows us to manipulate posture-based expressions. • Will utilize an off-board sensor network that includes color cameras, lasers, Nintendo Wiimotes™, and desktop computer interface. • Being implemented as a tabletop agent, interacting verbally and nonverbally with user.

10. Experimental Design • Preliminary work discusses gesture repetition/persistence (Mead & Matarić, 2009). • Test techniques and hypotheses within the context of ASD social skills intervention. • Must validate with typically developed children… • Consider expressive behaviors that utilize the isolated features determined during the staging process. • Consider their exaggerated, anticipatory, and secondary counterparts. • Conduct a study with children with ASD to determine the impact of each technique.

11. Acknowledgments • This work is supported in part by the National Science Foundation under: • Graduate Research Fellowship Program (GRFP) • Grant CNS-0709296 (“CRI: IAD - Computing Research Infrastructure for Human-Robot Interaction and Socially Assistive Robotics”) • Grant IIS-0803565 (“Personalized Socially-Assistive Human-Robot Interaction: Applications to Autism Spectrum Disorder”) • Grant IIS-0713697 (“HRI :Personalized Assistive Human-Robot Interaction: Validation in Socially Assistive Robotics for Post-Stroke Rehabilitation”) • We would like to thank Akhil Madhani and Walt Disney Imagineering Research & Development for the use of the Sparky Minimatronic™ robot figure.

12. Selected References • S. Baron-Cohen, O. Golan, E. Chapman and Y. Granader, “Transported into a World of Emotion,” in The Psychologist, vol. 20, no. 2, pp. 76-77, 2007. • R. L. Birdwhistell, Kinesics and Context: Essays on Body Motion Communication. Philadelphia, PA: University of Pennsylvania Press, 1970. • S. E. Brennan, “Caricature Generator: The Dynamic Exaggeration of Faces by Computer,” in Leonardo, vol. 18, no. 3, pp. 170-178, 1985. • D. J. Feil-Seifer and M. J. Matarić, “Defining Socially Assistive Robotics,” in International Conference on Rehabilitation Robotics, pp. 465-468, 2005. • O. Golan, E. Ashwin, Y. Granader, S. McClintock, K. Day, V. Leggett, and S. Baron-Cohen, “Enhancing Emotion Recognition in Children with Autism Spectrum Conditions: An Intervention Using Animated Vehicles with Real Emotional Faces,” in the Journal of Autism and Developmental Disorders, DOI: 10.1007/s10803-009-0862-9, in press. • R. Mead and M. J. Matarić, “The Power of Suggestion: Teaching Sequences Through Assistive Robot Motions,” in the Proceedings of The 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI-09), pp. 317-318, 2009. • Z. Mo, J. P. Lewis, and U. Neumann, “Improved Automatic Caricature by Feature Normalization and Exaggeration,” SIGGRAPH 2004 Sketches and Applications, 2004. • V. S. Ramachandran and W. Hirstein, “The Science of Art: A Neurological Theory of Aesthetic Experience,” in the Journal of Consciousness Studies: Special Feature on Art and the Brain, vol. 6, no. 6-7, pp.15-51, 1999. • P. A. Rao, D. C. Beidel, and M. J. Murray, “Social Skills Interventions for Children with Asperger’s Syndrome or High-Functioning Autism: A Review and Recommendations,” in the Journal of Autism and Developmental Disorders, vol. 38, no. 2, pp. 353-361, 2008. • A. Tapus, M. J. Matarić, and B. Scassellati, “The Grand Challenges in Socially Assistive Robotics,” in IEEE Robotics and Automation Magazine, vol. 14, no. 1, pp. 35-42, 2007. • F. Thomas and O. Johnston, The Illusion of Life: Disney Animation. Hyperion, 1981.

13. Q & A Please address praise and easy questions torossmead@usc.edu. Please address criticism and difficult questions to Dave… 