1 / 77

Course SHS Program in Cognitive Psychology Spring 2007 Human-Robot Interaction Social learning and skill acquisition vi

Course SHS Program in Cognitive Psychology Spring 2007 Human-Robot Interaction Social learning and skill acquisition via teaching and imitation Aude G. Billard Learning Algorithms and Systems Laboratory - LASA EPFL, Swiss Federal Institute of Technology Lausanne, Switzerland

xenia
Download Presentation

Course SHS Program in Cognitive Psychology Spring 2007 Human-Robot Interaction Social learning and skill acquisition vi

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Course SHS Program in Cognitive Psychology Spring 2007 Human-Robot Interaction Social learning and skill acquisition via teaching and imitation Aude G. Billard Learning Algorithms and Systems Laboratory - LASA EPFL, Swiss Federal Institute of Technology Lausanne, Switzerland aude.billard@epfl.ch A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  2. A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007 Calinon, S. and Billard, A. (2007) Incremental Learning of Gestures by Imitation in a Humanoid Robot. in Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI).

  3. Motor Learning How is information transferred across multiple modalities? Visuo-motor, Auditor-motor Gesture Recognition How are actions perceived? How is information parsed? Imitation Level of granularity: What is copied? Should it copy the intention, goal or dynamics of movement? A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  4. Biological Inspiration Robotic Implementation Gesture Recognition Learning by Imitation Motor Learning A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  5. BIOLOGICAL INSPIRATION Prior to building any capability in robots, we might want to understand how the equivalent capability works in humans and other animals A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  6. Imitation Learning Gesture Recognition Imitation Capabilities in Animals Which species may exhibit imitation is still a main area of discussion and debate One differentiate “true” imitation from copying (flocking, schooling, following), stimulus enhancement, contagion or emulation Biological Inspiration Learning by Imitation A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  7. Imitation Learning Gesture Recognition Imitation Capabilities in Animals • Copying and Mimicry: Rats, Monkeys • Observe companion actor rats performing different spatial tasks differing according to the experimental requirements. After the observational training, surgical ablation to block any further learning Biological Inspiration Learning by Imitation • Legio et al, Brain Res. Protocols, 2003 • Heyes, Trends in Cog. Sciences, 2001 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  8. Imitation Learning Gesture Recognition Imitation Capabilities in Animals • The observer rats displayed exploration abilities that closely matched the previously observed behaviors. Biological Inspiration Learning by Imitation • Legio et al, Brain Res. Protocols, 2003 • Heyes, Trends in Cog. Sciences, 2001 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  9. Imitation Learning Gesture Recognition Imitation Capabilities in Monkeys Subjects who saw the Lever demonstrations tended to use a levering movement to pop open the lid whereas subjects who viewed Poke, as well as the controls, did not display this behavior at all. Biological Inspiration Learning by Imitation • Whiten et al, Journal of Comparative Psychology, 1996 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  10. Imitation Learning Gesture Recognition Imitation Capabilities in Animals • “True” imitation: Ability to learn new actions not part of the usual repertoire • The appanage of humans only, and possibly great apes Biological Inspiration Learning by Imitation • Whiten & Ham, Advances in the Study of Behaviour, 1992 • Savage & Rumbaugh, Child Devel, 1993 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  11. Imitation Learning Gesture Recognition Imitation Capabilities in Animals • Complex Imitation capabilities in Dolphins & Parrots. Large repertoire of imitation capabilities, demonstrating flexibility and generalization in different contexts. Biological Inspiration Learning by Imitation • Moore, Behaviour, 1999. • Herman, Imitation in Animals & Artifacts, MIT Press, 2002 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  12. Imitation Learning Gesture Recognition Developmental Stages of Imitation • Innate Facial Imitation (newborns  3 months) • Tongue and lips protrusion, mouth-opening, head • movements, cheek and brow motion, eye blinking • Delayed imitation up to 24 hours •  Imitation is mediated by a stored representation Biological Inspiration Learning by Imitation Meltzoff & Moore, Early Development and Parenting, 1997 Meltzoff & Moore, Developmental Psychology, 1989 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  13. Imitation Learning Gesture Recognition Developmental Stages of Imitation • Deferred and delayed imitation - 18 month (Piaget), 9-12 months (Meltzoff) • Deferred imitation of novel behavior • 67% of the infants who saw the display reproduced the act after the week's delay, as compared to 0% of the control infants who had not seen the novel display. Biological Inspiration Learning by Imitation • Piaget, Play, Dreams and Imitation in Infancy, 1962 ; • Meltzoff, Body and the self, 1995 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  14. Imitation Learning Gesture Recognition Goals and Intentions • Infants aged 14 months. • Children imitate new action to achieve the same goal only if they consider it to be the most rational alternative. Biological Inspiration Learning by Imitation • Gergely, Bekkering, Giraly, Nature415, 755, 2002 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  15. Imitation Learning Gesture Recognition Goals and Intentions • 18-months infants • Differentiate between human and machine demonstration •  Attribute intentions only to the human • Learn from unsuccessful examples Biological Inspiration Learning by Imitation • Meltzoff, Dev. Psychol. 31, 1995. A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  16. Imitation Learning Gesture Recognition Goals and Intentions • Imitation is hierarchical and goal-directed • Single-hand motions: accurate ipsilateral imitation, 48% subsitution for crosslateral imitation • Two-hand motions: only 10% substitution for crosslateral imitation. Biological Inspiration Learning by Imitation • Two-phase motion eliminates mistakes • Adding constraints of hand gestures increases mistakes • Bekkering, Wolschlager & Gattis, Quart. J. of Exp. Psych, 2000 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  17. Imitation Learning Gesture Recognition Imitation in adults • Reaches highest level of complexity • Is present in all activities: • Social influence in establishing group norms; collective frame of reference, transmission of phoebias Biological Inspiration Learning by Imitation A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  18. Imitation Learning Gesture Recognition Imitation Capabilities in Adults Movement observation influences movement execution Priming process occurs involuntarily and is not under the actor’s control. Biological Inspiration Learning by Imitation • Brass, Bekkering, Prinz, Acta Psychologia, 2001 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  19. Imitation Learning Gesture Recognition Neural Correlates • Mirror Neuron System – F5 Area of Monkey M1 Biological Inspiration Learning by Imitation • Gallese et al, Brain, 1996. ; Rizzolatti et al, Cog. Brain Res., 1996 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  20. Imitation Learning Gesture Recognition Neural Correlates • Mirror Neuron System – locus of visuo-motor transformation (STS, PM, Broca) Biological Inspiration Learning by Imitation • Iacoboni et al, Science 1999 • Arbib, Billard, Iacoboni, Oztop, Neural Networks, 2000. A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  21. Imitation Learning in Animals Take-Home Message • Range of imitative behaviors in animals •  Increasing in complexity across species • Stages of development in children imitation •  innate facial imitation •  inferring goals •  hierarchy of goals driving imitation • (hand motion takes precedence over arm gesture and location in space) • Imitation in adulthood is influenced by mvmt observation, handedness, orientation of the demonstrator • The underlying neural mechanisms are not yet completely deciphered •  A better understanding of those would help shed light on the different levels of imitation in animal behavior A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  22. Imitation Learning in Animals Take-Home Message • Advantages: When is Imitation useful? • It is a powerful means of transferring skills • It speeds up the learning process by showing possible solutions or conversely by showing bad solutions A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  23. Imitation Learning in Animals Take-Home Message • Disadvantages: • When is Imitation not useful? • Not appropriate: When a good solution for the teacher is not a possible solution for the learner • Disadvantageous: When it induces you in error - bad teacher (e.g. phoebia of spiders) A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  24. Robotic Implementation Gesture Recognition Biological Inspiration Learning by Imitation Motor Learning A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  25. The Transfer Problem Imitator Demonstrator ? A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  26. Same Object, same target location Same direction of motion Same speed, same force Same posture What to imitate? A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  27.  Find the closest solution according to a metric How to Imitate? The correspondence problem Demonstration Imitation ? No solutions (smaller range of motion) A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  28. Imitation Learning Gesture Recognition Following – an imitation mechanism • While following the teacher, the learner robot learns to associate a word with a meaning in terms of sensory inputs Learning by Imitation Robotic Implementation • Billard et al, ESANN’1997, • Billard & Dautenhahn, Robotics & Autonomous Systems 1998, • Billard & Hayes, 99,00 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  29. Imitation Learning Gesture Recognition Following – an imitation mechanism • Teaching path in a Maze • Demiris & Hayes, 1994, 1996; • Teaching how to climb a hill • Dautenhahn, Robotics & Autonomous Systems, 1995 • Teaching a path in the environment • Billard & Hayes, Adaptive Behavior, 1999 • Moga, Gaussier, Applied Artificial Intelligence, 2000 • Kaiser et al, Robotics & Autonomous Systems, 2002 • Nicolescu & Mataric, AGENTS’ 2003 • Teaching a vocabulary • Billard 1997, 1998, 1999 • Vogt & Steels, ECAL, 1999 Learning by Imitation Robotic Implementation A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  30. Imitation Learning Gesture Recognition One-Shot Learning Methods • Segmentation of demonstration into primitives • Classification of gestures into predefined states (e.g. grasp, collision) • Built-in controller for producing sequences of states Learning by Imitation Robotic Implementation • Kuniyoshi et al. IEEE Trans. on Robotics and Automation,1994. • Dillmann et al,Robotics & Autonomous Systems, 2001. • Ritter et al, Rev Neuroscience, 2003 • Aleotti et al, Robotics & Autonomous Systems, 2004. A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  31. Robot Programming by Demonstration One-Shot Learning Methods Sensors: Data Gloves, Fixed cameras, Speech processing Actuators: Mobile robot, 7 DOF arm, 2 fingers Gripper R. Dillmann, Robotics & Autonomous Systems 47:2-3, 109-116, 2004 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  32. Imitation Learning Gesture Recognition One-Shot Learning Methods • Explicit teaching/learning: • Reasoning about tasks • Verbal instructions • Gesture Recognition: • For each sensor a context-dependent • model based on background knowledge • is provided: ‘opening the refrigerator door’, ‘extracting the bottle’ and ‘closing the door’ • Task Reproduction: • Store action sequences in a tree-like • structure of macro-operators Learning by Imitation Robotic Implementation R. Dillmann, Robotics & Autonomous Systems 47:2-3, 109-116 2004 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  33. Imitation Learning Gesture Recognition Robot Programming by Demonstration: Grasping • Because of the large range of possible shapes, generalizing pre-programmed grasps to new and general objects is a rather hard task: • Orientation of the hand • Positioning of the fingers (correspondence problem!) • Tactile forces, stable object contact Learning by Imitation Robotic Implementation Steil et al, Robotics & Autonomous Systems 47:2-3, 129-141, 2004 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  34. Imitation Learning Gesture Recognition Robot Programming by Demonstration: Grasping • (i) a ‘naïve’ imitation strategy, in which the observed joint angle trajectories (after their transformation into the three-finger geometry) were directly applied to control the fingers of the TUM hand during the grasp, until complete closure around the object • (ii) a strategy in which the visually observed hand posture is matched to the initial conditions of a power grip, a precision grip, a three-finger and two-finger grip, respectively, in order to identify the grip type. Learning by Imitation Robotic Implementation Steil et al, Robotics & Autonomous Systems 47:2-3, 129-141, 2004 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  35. Robot Programming by Demonstration • Other related works are, e.g.: • Kuniyoshi et al, ICRA, 1994 • Aleotti et al, Robotics & Autonomous Systems, 47:2-3, 153-167, 2004 • Zhang & Roessler, Robotics & Autonomous Systems 47:2-3, 117-127, 2004 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  36. Imitation Learning Gesture Recognition Learning of Dynamical Systems • Learning the optimal controller • Model of physical system (pendulum) • Reinforcement and locally weighted learning Learning by Imitation Robotic Implementation • Atkeson & Schaal, ICML, 1997. A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  37. Imitation Learning Gesture Recognition Learning of Dynamical Systems • Locally weighted learning • Learning primitives of the system Learning by Imitation Robotic Implementation • Ijspeert, Nakanishi, Schaal, ICRA’01, NIPS’02 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  38. Imitation Learning Gesture Recognition Learning of Dynamical Systems • Locally weighted learning • Learning primitives of the system Learning by Imitation Robotic Implementation • Ijspeert, Nakanishi, Schaal, ICRA’01, NIPS’02 A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  39. Imitation Learning Gesture Recognition Learning of Dynamical Systems The learned trajectory is not sufficient to control the actual robot’s walking pattern. Phase resetting using foot contact information is necessary.  on-line adjustment of the phase of the CPG by sensory feedback from the environment is essential to achieve successful locomotion Learning by Imitation Robotic Implementation Nakanishi et al, Robotics & Autonomous Systems, 47:2-3, 79-91, 2004. A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  40. Imitation Learning in Robots Granularity Cognition How to imitate? Level 3: Learning primitives of motion Level 2: Exact reproduction of trajectories Level 1: One-shot learning Level 0: Following – an implicit imitation mechanism Level of cognition Level of granularity: A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  41. Self Motor Learning - Reinforcement Learning • To adaptto novel situations • To adapt the demonstrated motions to the robot’s body Correspondence problem • (Nehaniv & Dautenhahn 1999) IMITATION LEARNING VERSUS MOTOR LEARNING • Imitation learning – Programming by Demonstration: • A wayto speed up learning, to reduce the search space • A wayto sharewith the robot’s the samevocabulary of motor skills A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  42. Learning What to imitate The robot should learn that the important feature in this task is that the queen should be moved 2 steps forward vertically A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  43. Learning How to Imitate Once the robot has learned the rule of motion for the queen, it can apply this rule for moving the queen from locations not seen during the demonstrations A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  44. Self Motor Learning - Reinforcement Learning • To adaptto novel situations • To adapt the demonstrated motions to the robot’s body Correspondence problem • (Nehaniv & Dautenhahn 1999) IMITATION LEARNING VERSUS MOTOR LEARNING • Imitation learning – Programming by Demonstration: • A wayto speed up learning, to reduce the search space • A wayto sharewith the robot’s the samevocabulary of motor skills A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  45. Transmitting human skills and knowledge to robots Learning a Packaging Task A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  46. From Recognizing to Reproducing Gestures A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  47. From Recognizing to Reproducing Gestures Recovers generalized signal by regression GMM/HMM Encoding: Mixture of k Gaussians A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  48. Learning What to imitate? GMM over 26 demonstrations 3D velocities of the end effector Tracking of object – stereovision Tracking of joint trajectories A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007 Calion, S., Guenter, F. and Billard, A. (2007) On Learning, Representing and Generalizing a Task in a Humanoid Robot. IEEE Transactions on Systems, Man and Cybernetics, 37:2. Part B. Special issue on robot learning by observation, demonstration and imitation.

  49. Learning What to imitate? Calion, S., Guenter, F. and Billard, A. (2007) On Learning, Representing and Generalizing a Task in a Humanoid Robot. IEEE Transactions on Systems, Man and Cybernetics, 37:2. Part B. Special issue on robot learning by observation, demonstration and imitation. A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

  50. Learning What to imitate? Correlations in the latent space of the two hands Hands-Bucket Correlations A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007

More Related