Introduction to Computer Vision and Robotics: Motion Generation

1. Introduction toComputer Vision and Robotics:Motion Generation Tomas Kulvicius Poramate Manoonpong

2. Motion Control:Trajectory Generation

3. Different robots –> different motions -> different trajectories

4. How do we generate/plan trajectories? Depends on -what kind of trajectories we need -apllication

5. Movement overview Movements Periodic Point-to-Point Splines DMPs DMPs RNNs NOs GMMs

6. Overview Movements Periodic Point-to-Point Splines DMPs DMPs RNNs NOs GMMs

7. Polynomial interpolation Example trajectory sampled by blue points

8. Polynomial interpolation Sampled trajectory 4th order polynomial Insufficient fit!

9. Polynomial interpolation Sampled trajectory 6th order polynomial Insufficient fit!

10. Polynomial interpolation 9th order polynomial Sampled trajectory Runge’s phenomenon

11. Runge’s phenomenon Runge function 5-th order polyn. 9-th order polyn.

12. Spline interpolation Idea: many low order polynomials joined together Sampled trajectory Cubic spline No oscillations as compared to polynomial interpolation One can add desired velocity (cubic) or acceleration (5th order) at the end points

13. Overview Movements Periodic Point-to-Point Splines DMPs DMPs RNNs NOs GMMs

14. Dynamic Movement Primitives (DMPs)? “DMPs are units of actions that are formalized as stable nonlinear attractor systems” (Ijspeert et al., 2002, Schaal et al., 2003, 2007)

15. Formalism of discrete DMPs Position change (velocity): Velocity change (acceleration): (v) g – goal t – temp. scal. Exponential decay: Nonlinear function: A set of differential Eqs, which defines a vector field that takes you from any start-point to the goal Kernels: Ijspeert et al., 2002; Schaal et al., 2003, 2007

16. Formalism of discrete DMPs Position change (velocity): Velocity change (acceleration): (v) g – goal t – temp. scal. Exponential decay: Nonlinear function: Kernels: Ijspeert et al., 2002; Schaal et al., 2003, 2007 Time

17. DMP properties: 1. Generalization DMPs can be scaled -in time and -space without losing the qualitative trajectory appearance

18. DMP properties: Position scaling

19. DMP properties: Generalization DMPs can be scaled in time and space without losing the qualitative trajectory appearance

20. DMP properties: 2. Robustness to perturbations Real-time trajectory generator – can react to perturbations during movement

21. DMP properties: 3. Coupling • DMPs allow to add coupling terms easily: • Temporal coupling • Spatial coupling

22. DMP properties: Temporal coupling Velocity change (acceleration): (v) Exponential decay (phase variable): +Ct Adding additional term Ct allows us to modify the phase of the movement, i.e., stop the movement in case of perturbations.

23. DMP properties: Phase stopping DMPs are not directly time dependent (phase based) which allows to control phase of the movement (e.g., phase stopping) With phase stopping Without phase stopping

24. Temporal coupling: Movement stopping

25. Proactive behavior in humans

26. What about robots?

27. DMP properties: Spatial coupling Velocity change (acceleration): +Cs Adding additional term Cs allows us to modify trajectory online by taking sensory information into account, i.e. online obstacle avoidance.

28. Spatial coupling: Obstacle avoidance

29. Spatial coupling: Human-Robot interaction

30. Spatial coupling: Robot-Robot interaction

31. Comparison of discrete movement generators

32. Overview Movements Periodic Point-to-Point Splines DMPs DMPs RNNs NOs GMMs

33. Formalism of discrete DMPs: Reminder Position change (velocity): Velocity change (acceleration): (v) g – goal t – temp. scal. Exponential decay: Nonlinear function: Kernels: Ijspeert et al., 2002; Schaal et al., 2003, 2007 Time

34. Formalism of rhythmic DMPs Position change (velocity): Velocity change (acceleration): (f,A) Limit cycle oscillator with constant phase speed: g – baseline A – amplitude t – frequency Nonlinear function: Kernels: Time Ijspeert et al., 2002; Schaal et al., 2003, 2007

35. Overview Movements Periodic Point-to-Point Splines DMPs DMPs RNNs NOs GMMs

36. Neural oscillators Central Pattern Generator (CPG) Pattern generation without sensory feedback (Open-loop system)

37. CPG methods • Dynamical system approach: • Van der Pol Oscillator • Dynamic Movement Primitives • Neural control approach: • Matsuoka Oscillator • 2-neuron Oscillator

38. 2-neuron oscillator • Neural structure: 2-neuron network [Pasemann et al., 2003] • Central pattern generator (CPG): Self excitatory + excitatory & inhibitory synapses The activation function The transfer function

39. 2-neuron oscillator W12 = - W21 W11, W22 Pasemann, F., Hild, M., Zahedi, K. SO(2)-Networks as Neural Oscillators, Mira, J., and Alvarez, J. R., (Eds.), Computational Methods in Neural Modeling, Proceedings IWANN 2003, LNCS 2686, Springer, Berlin, pp. 144-151, 2003.

40. CPG with modulatory input Modulatory input

41. Different walking gates (AMOS II)

42. Overview Movements Periodic Point-to-Point Splines DMPs DMPs RNNs NOs GMMs

43. Reflexive neural networks Reflexes - local motor response to a local sensation Locomotion as a chain of reflexes: purely sensory-driven system (Closed-loop system).

44. Reflexive neural network: application to bipedal robot RunBot

45. Sensor-triggered generation of movement 1) Left leg touches the ground: GL= active  Left hip flexes (backward) & Left knee extends (straight) = STANCE  Right hip extends (forward) & Right knee flexes (bend) = SWING 2,3) Right Hip anglereaches AEA (Anterior Extreme Angle) AEA= active  Right knee extends (straight) Right leg still in swing, Left leg still in stance 4,5) Right legtouches the ground: GR= active  Right hip flexes (backward) & Right knee extends (straight) = STANCE  Left hip extends (forward) & Left knee flexes (bend) = SWING GL

46. Reflexive neural network of RunBot

47. Passive dynamic walking

48. Learning to walk up a ramp Neural learning

49. RunBot learning to walk up a ramp

50. Reflex based methods Pros: • Very close link between the controller and what the robot actual does Cons: • because of the lack of a centrally generated rhythm, locomotion might be completely stopped because of damage in the sensors and/or external constraints that force the robot in a particular posture.