From motor babbling to hierarchical learning by imitation: a robot development pathway

1. From motor babbling to hierarchical learning by imitation: a robot development pathway YiannisDemiris and Anthony Dearden By James Gilbert

2. Overview • Social vs. A-Social Learning • Forward & Inverse Models • HAMMER Architecture

3. Fundamental Questions • ‘How does an individual use the knowledge acquired through self exploration as a manipulable model through which to understand others and benefit from their knowledge?’ • ‘How can developmental and social learning be combined for their mutual benefit? ‘

4. A-Social Learning • The robot attempts to learn to perform a task by interacting with its environment • Doesn’t consider the solutions that other agents in its environment possess with respect to the task.

5. Social Learning • Learning by: • Observation, • Demonstration and, • Imitation. • Equipping robots with the ability to imitate enables them to learn to perform tasks by observing a human demonstrator (Schaal, 1999).

6. Forward Model • ‘We aim at building a system that enables a robot to autonomously learn a forward model with no a priori knowledge of its motor system or the external environment.’ • The robot sends out random commands to its motor system (babbles), • Then together with the information from the vision system learns the structure and parameters of a Bayesian belief network.

7. Inverse Model • Inversely mapping the learnt forward models gives a corresponding inverse model. • This allows the model to seek meaning from a subject being imitated by being able to ask itself what it would do in that situation.

8. Coupled Model • By achieving a coupled inverse and forward model, a robot is capable of • Executing an action, • Rehearsing an action, or • Perceiving an action

9. HAMMER Architecture • (HAMMER – Hierarchical Attentive Multiple Models for Execution and Recognition). • Attempts to combine social and a-social models. • ‘Adopts the simulation theory of mind approach to understand the actions of others, using a distributed network of coupled inverse and forward models.’ • New models can be added either through self-exploration, or by observing others, through imitation.

10. HAMMER Architecture ctd. • The fundamental structure of HAMMER is an inverse model paired with a forward model. • The inverse model receives information about the current state, • Outputs the motor commands it believes are necessary to achieve target goals. • The forward model provides an estimate of the next state. • This is fed back to the inverse model, allowing it to adjust any parameter of the behaviour.

11. HAMMER Architecture ctd. • Matching a visually perceived demonstrated action with the imitator’s equivalent one. • Feeding the demonstrator's current state as perceived by the imitator to the inverse model. • Generate the motor commands that it would output if it was in that state and wanted to execute an action. • The forward model outputs the estimated next state, which is a prediction of what the demonstrator’s next state will be. • The prediction is compared with the demonstrator’s actual state at the next step. • The comparison error can then be used for learning.

12. Open Challenge • Forward models are learnt between the motor commands and the image plane based visual perception of the effects of these actions. A generalisation of these to more abstract representations (for example a 3-D body schema ) will allow more complex relationships to be addressed.

13. Conclusion & Personal Thoughts • By using Bayesian Belief Networks to create both forward and inverse models, it was shown through a simple practical experiment that a robot can mimic the actions of a human. • However the robot doesn’t necessarily understand the meaning of the action. • The experimental robot was very simple and shares very little with the human form it was to mimic.

14. Questions…