Movement Imitation: Linking Perception and Action

1. Movement Imitation: Linking Perception and Action Advanced Topics in Computer Vision, 2004 Lior Noy Department of Computer Science and Applied Mathematics Weizmann Institute of Science

2. Movement Imitation - Example

3. Imitation: Linking Perception and Action semantic world(objects, actions) Perception Imitation Action realm of raw-data (pixels, muscles activation)

4. Outline 1. MovementImitation 2. Programming By Demonstration • 3. Robotic Movement Imitation • Primitives Based Approach (Mataric’) • Real Time Tracking (“mirror-game”) (Ude et al.) 4. Direct Perception and Imitation

5. A Variety of Probes into Imitation Developmental psychology Ethology Imitation Human Brain Imaging Cognitive psychology Neurophysiology Robotics

6. Evaluating ImitationRobot Following in a Hilly Environment

7. Evaluating Imitation

8. Evaluating Imitation

9. Programming By Demonstration (PbD) Methods to program a robot • Human Programming • Reinforcement Learning • Programming by Demonstration

10. Programming By Demonstration (PbD)Applications • Navigation • Locomotion • Playing air-hockey • Manipulating blocks • Balancing a pole • Hitting a tennis-serve • Grasping unfamiliar objects • Imitating dancing movement

11. PbD – Application Example The “Golden Maze”

12. PbD – Application Example Playing Air-Hockey

13. PbD – Application Example Box Manipulations

14. Three Approaches for PbD • Symbolic • Control-Based • Statistical

15. Symbolic Approach for PbD • Analyze observed actions in terms of sub-goals • Match actions needed to fulfill these sub-goals • Createa symbolic description of the environment ( ”object A is above object B” ) • Learn a series of symbolic if-thenrules ( ”if object A is aboveobject B then grasp-object[ object B ]” )

16. Example: Symbolic Approach for PbD (Kunyushi et al., 1994) … but how do you symbolically describes “hitting a tennis serve”?

17. Control-Based Approach for PbD • No symbolic parsing of perceived actions • Assume a pre-defined control policy • Acquire needed parameters from observation

18. Forward Models motor commands jointangles end-effectorposition Inverse Models Control-Based Approach Inverse Models Sometime assume known inverse models (converting desired effect to needed commands)

19. Example: Control-Based Approach for PbD (Schaal, 2003) Tennis movie

20. Statistical Approach for PbD • No prior assumption on used control policy • Statistically match perception and action • Can this be done? More on this later…

21. PCA Example: Statistical Approach for PbD (Asada, 1995)

22. Example: Statistical Approach for PbD • Learning: • Perform random action A(i) • Record resulted optical flow f(i) • Compute principal-component p1(i), p2(i) • Learn the connection A(i) – {p1(i), p2(i)}

23. Outline 1. MovementImitation 2. Programming By Demonstration • 3. Robotic Movement Imitation • Primitives Based Approach (Mataric’) • Real Time Tracking (“mirror-game”) (Ude et al.) 4. Direct Perception and Imitation

24. PbD for Movement ImitationPre-Cursor 1: Cartoons Retargeting (Bregler et al., 2002)

25. y x Cartoons RetargetingTwo Types of Deformations • Affine deformation • Key shape deformation

26. Cartoons Retargeting - Results More on: http://www.cs.weizmann.ac.il/~hassner/cv03/“Animating human motion”, Speakers : Simon Adar, Yoram Atir

27. PbD for Movement ImitationPre-Cursor 2: Guided Movement Synthesis (Zelnik-Manor, Hassner & Irani, 2004)

28. Event-Based Analysis of Video (Zelnik-Manor & Irani, 2001)

29. Guided Movement Synthesis(a.k.a. “Movement Imitation”?)

30. PbD for Movement ImitationPre-Cursor 2: Movement Synthesis

31. X1X2...Xn J1J2...Jm movementprimitive1 movementprimitive2 = = ... movementprimitive K PbD for Movement ImitationCase Study: Primitive-Based Approach The Problem: How to convert visual input to motor output? A Possible Solution: Use a common, sparse representation: sensory-motor primitives. … but what primitives to use?

32. Movement Imitation Using Sensory-Motor Primitives Motor primitives: Sequences of action that accomplish a complete goal-directed behavior. Examples:1. Move hand in “straight line”, “parabola” (Felix…).2. Perform “grasping”, “a tennis serve”.

33. Imitation Learning Using Sensory-Motor Primitives (Schaal, Ijspeert & Billard, 2003)

34. Inspiration for Using Sensory-Motor Primitives • Evidence for: • Coding of goal-directed actions. • Shared representations of perception and action. • Example – Mirror Neurons. (Rizzolatti et al., 2002; Gallese et al. 1996)

35. Movement Imitation Using Sensory-Motor Primitives • General Principles: • Selective attention focusing on end-points movements. • Sensory-motor primitives as integrative representation. • Learning new skills as compositions of primitives. • Experimental test-beds. (Mataric’,1998)

36. What Sensory-Motor Primitives to Use? InnatePre-defined control policies (e.g., central pattern generators) LearnedUn-supervised clustering (using PCA, Isomap ) Primitives End-Points Space(“visual space”) Joints Space(“motor space”)

37. “Experiment in Imitation Using Perceptuo-Motor Primitives”, (Weber, Jenkins & Mataric’,2001) • Extract hand (end-point) movements. • Perform Vector-Quantization to get invariant representation.

38. “Experiment in Imitation Using Perceptuo-Motor Primitives” • Classify movement to primitives (line, arc, circle). • Group adjacent similar primitives.

39. “Experiment in Imitation Using Perceptuo-Motor Primitives” • Determine primitives parameters. • Project to ego-centric space.

40. “Experiment in Imitation Using Perceptuo-Motor Primitives” (Weber, Jenkins & Mataric’,2001)

41. PbD for Movement ImitationCase Study: Real-Time Tracker • The Goal:Mimic movements in real-time • The Problem: • Large amount of data to process (6 MB/Sec) • Need “continuous success” • The Solution: • Probabilistic approach to prevent excessive data interactions (Ude et al.,2001)

42. “Real-Time Visual System for Interaction with Humanoid Robot”(Ude, Shibata & Atkeson, 2001) Estimate positions of tracked “blobs” in the image Compute 3D coordinates of tracked objects using stereo Transform into via-points for robot hand trajectory Compute motor commands from desired trajectory

43. Real-Time Tracker Tracking “Blobs” In a Bayesian Setting probability for the pixel at location u to have Intensity Iu Given the process k a-priori probability for process k

44. Real-Time TrackingMinimize Log-Likelihood overall probabilityto observe image I Goal: determine the parameters that are most likely to produce this image – Maximal Likelihood Problem. computationally easier to minimize the negative log likelihood

45. Real-Time TrackingMinimize Log-Likelihood Find minimum (using Lagrange Multipliers) and get: probability that pixel ustems from process l

46. Real-Time TrackingFind Probabilities Parameters The above equations are solved iteratively by the Expectation-Minimization (EM) algorithm • Expectation stage: • compute Pu,l using the current estimate for Ө and ω. • Minimization stage: • compute new Ө and ω assuming Pu,l are constant. from probabilities of pixels to belong to a certain process (e.g. – the human hand) …

47. “Real-Time Visual System for Interaction with Humanoid Robot” … to object locations

48. Real-Time TrackingGeneral Stages Estimate positions of tracked “blobs” in the image Compute 3D coordinates of tracked objects using stereo Transform into via-points for robot hand trajectory Compute motor commands from desired trajectory

49. Real-Time TrackingEstimate Trajectories with B-splines

50. Real-Time Tracking - Results Robot Compliance Movie