MURI

1. High-Level Control MURI How is Compliance used in Locomotion? Low-Level Control Berkeley & Stanford: Measurements of Cockroach Locomotion What Compliance Strategies in Human-level Tasks? Fabrication Harvard & Johns Hopkins: Compliance Learning and Strategies for Unstructured Environments

2. High-Level Control Guiding questions What strategies are used in insect locomotion and what are their implications? MURI Low-Level Control Insect locomotion studies (Berkeley Bio) New measurement capabilities (Stanford) What motor control adaptation strategies do people use and how can they be applied to robots? Fabrication Compliance Learning and Strategies for Unstructured Environments (Harvard & Johns Hopkins) Implications for biomimetic robots (Harvard, Stanford)

3. MURI High-Level Control Impedance Adaptation in Unstructured Environments Yoky Matsuoka and Rob Howe Harvard University

4. MURI High-Level Control Manipulation with Impedance • An example of manipulation with impedance • Why is biology superior to current robots in an unstructured environment?

5. MURI High-Level Control Identify Impedance Learning Strategy in Human • Two big questions: • What is the initial strategy used to cope with unknown/unstructured environments? • After learning, what does the biology pick as the good solution for impedance for a given environment? How can these solutions help robot’s control strategy?

6. MURI High-Level Control Comparison Between Analytical and Biological Solutions • We can mathematically derive optimal impedance for a linear world. • Biological system converges to the analytical solution. --- great! • Biological system converges to a different solution. --- what and why: put the biological solution back in the equations and reverse engineer. • What about a nonlinear varying world where it is difficult to derive the optimal impedance? • What does the biological system do? Can it be modeled as a solution for robots?

7. MURI High-Level Control Example: Linear World --- Catching a Ball • Goal: Find the “best” impedance. • For this case, find best Khand. • Uncertainty in the world • mball, kball, ball(0), and khand ball mball kball mball xball kball mhand mhand hand xhand khand khand

8. MURI High-Level Control Example: Linear World --- Catching a Ball • Cases: 1. Hand stiffnes (khand) is too high • hand< 0 bounces up 2. Hand stiffness (khand) is too low • xhand > Threshold bottoms out 3. Hand stiffness (khand) is just right • xball xhand until switch is pressed mball xball kball mhand xhand khand 2 3 1 khand 0 infinite

9. MURI High-Level Control Example: Linear World --- Catching a Ball • Solve for xhand(t) and xball(t) • initial condition • ball(0) > 0 • xball(0) = 0 • hand(0)= 0 • xhand(0) = 0 mball xball kball mhand xhand khand

10. MURI High-Level Control Analytical Linear World to Biological Motor Control • The example relates task performance to limb impedance and optimal solution. • Other examples: leg impedance, etc. • Now measure human strategy…. • “System identification” • Need a new technique

11. MURI High-Level Control Existing System Identification Techniques • Time invariant systems --- easy • assume constant m, b, and k over time. • apply external impulse perturbation force. • repeat the same condition and average.

12. MURI High-Level Control Existing System Identification Techniques • Time varying systems • Cannot apply impulses close to each other. • Need multiple impulses to solve for multiple unknowns. • PRBS (Lacquaniti, et al. 1993)

13. MURI High-Level Control New System Identification Technique to Observe Learning Setup Monitor Robot Processor Force Sensor Handle Data Acquisition System Human Subject Accelerometer

14. MURI High-Level Control New System Identification Technique to Observe Learning • Very short duration • Very clean data from force and acc. sensors F k*x b*v m*a m=F/a b= (F-ma)/v k= (F-ma-bv)/x

15. MURI High-Level Control Testing the New Technique

16. MURI High-Level Control Video of the task here

17. MURI High-Level Control Testing the New Technique • Phantom robot is used as the perturbation/measurement tool. • Task: balance the moving ball on paddle. • ball moves at constant speed • dies when the ball falls off the paddle • perturbation applied every second

18. MURI High-Level Control Impedance Change with Learning m change over time k change over time b change over time

19. MURI High-Level Control Contact Interaction Task -- Impedance Dependent Task • Observe the impedance change within one catch • Observe the impedance change between catches k b ** under development --- pilot studies underway

20. MURI High-Level Control Current Understanding of the Structure of the Biological Controller From Shadmehr

21. MURI High-Level Control Impedance AdaptationConclusions and Future Work • Developed a new impedance identification technique • Based on virtual environment --- extremely versatile • Confirmed ability to measure instantaneous impedance, characterized changes with learning.

22. MURI High-Level Control Impedance AdaptationConclusions and Future Work • Current experiments • Determine human interaction strategies • initial impedance • learning characteristics • final impedance • Next experiments • Determine human interaction strategies for nonlinear varying tasks • e.g. plastic deformation (running in sand)

23. MURI High- Level Control Control of Locomotion Local controller (single limb): Control of the limb based on local information: - position and velocity of the limb Is the limb far enough to the back so that I can start the return stroke? - forces acting on the limb Is the supporting load small enough for me to lift the limb? Task controller (all limbs): Coordination with other limbs: - position of the other legs Across species, control of limb based on local information appears similar (Cruse, TINS, 90). Coordination with other limbs appears highly species dependent.

24. MURI High- Level Control Control of a Limb Based on Local Information Step cycle: generation of power (stance) and return (swing) strokes. Return phase: Move the limb from posterior to anterior position along a desired velocity profile. - Maintain proper impedance to remain stable in case of perturbations After hitting an obstacle, the limb should converge back to the desired path - Adapt impedance to allow for generation of desired behavior in the face of a persistent environment limb is moving in highly viscous fluid, it must adapt its impedance to the characteristics of the environment. Impedance control and adaptation in a position controltask Power phase: Maintain contact, maintain height of load, move limb from front to rear. Impedance control and adaptation in a contact/force control task

25. MURI High- Level Control Current Limb Local Control Models for Locomotion in Insects Cruse et al (Neural Networks 1998): Stick insect model - limb has little or no inertia - no muscle like actuators - controller output is velocity, feedback sensing via position and linear feedback - no ability to adapt Essentially a kinematic model of a limb only, with little or no dynamics This kind of model tells us little about how to design good controllers

26. MURI High- Level Control Designing a Single Limb Impedance Controller General Goals: 1. To understand what impedance strategies a biomechanical controller uses when it moves the limb in a position control task. Apply the results to control of the return phase. 2. To understand impedance strategies of the biomechanical controller in a force control task. Apply the results to the control of the stance phase. Approach: Study the human arm’s impedance adaptive control strategies in both position and force control tasks. Test validity of the strategies on a robotic system.

27. MURI High- Level Control Designing a Single Limb Impedance Controller Task Division: 1. Impedance control at very short time intervals (<10 msec, preflexes) Yoky Matsuoka and Rob Howe 2. Impedance control at intermediate and long time intervals (<300 msec) Tie Wang and Reza Shadmehr 3. Test and implementation on a robotic system Jay Dev Desai and Rob Howe

28. MURI High- Level Control Quantifying Impedance Control Strategies of a Biomechanical Controller • Challenges: • The biomechanics of the human arm are dominated by multiple feedback loops, with various time delays. Impedance measurements are done through imposition of perturbations and measurement of responses. • How do time delays affect measures of arm impedance? • Humans learn internal models when they learn control. How does a change in the internal model affect measures of arm impedance? • Impedance measures require an estimation of where the system would have been if it had not been perturbed. How well can we do this with a non-stationary system like the human arm?

29. MURI High-Level Control Current Understanding of the Structure of the Biological Controller Through modulation of input u to the muscles, impedance of the system is changed. The impedance depends on 3 feedback pathways: 1. Near zero-delay mechanical stiffness/viscosity of the muscles (Yoky). 2. Short delay sensory feedback through spinal structures. 3. Long delay sensory feedback through cortical structures (forward model).

30. MURI High-Level Control Are muscle “preflexes” enough? High-level sensory feedback loop disrupted Intact control system

31. MURI High-Level Control Impedance of a biological arm: A definition

32. MURI High- Level Control Estimating Impedance: Theory

33. MURI High- Level Control Estimating Impedance: Requirements

34. MURI High-Level Control 1.0 Estimating Inertial Dynamics of the Arm (Theory)

35. MURI High- Level Control 1.1 Estimating Inertial Dynamics of the Arm (Methods) Give a force pulse, use data for up to 14 ms after the pulse to estimate inertial parameters.

36. MURI High- Level Control 1.2 Estimating Inertial Dynamics of the Arm (Results) 6 Experiment Model 4 Error 2 Shoulder Torque (Nm) 0 -2 -4 0 10 20 30 40 50 60 70 Index of Array 4 3 2 1 Elbow Torque (Nm) 0 -1 -2 -3 -4 0 10 20 30 40 50 60 70

37. MURI High- Level Control 2.0 Predicting the Un-perturbed Trajectory (Theory) Legend

38. MURI High- Level Control 2.1 Trajectory Prediction (Methods)

39. MURI High- Level Control 2.2 Trajectory Prediction (velocity) Prediction error (%)

40. MURI High- Level Control 2.3 Trajectory Prediction (force) 1 1 1 1 0 0 0 0 -1 -1 -1 -1 Force (N) -2 -2 -2 -2 0 20 40 60 0 20 40 60 0 20 40 60 0 20 40 60 1 1 1 1 0 0 0 0 -1 -1 -1 -1 -2 -2 -2 -2 0 20 40 60 0 20 40 60 0 20 40 60 0 20 40 60 Time (10 msec) Conclusion: Position, velocity, and force can be reasonably well predicted for up to 300 msec after the last sampled data point.

41. MURI High- Level Control 3.0 Estimating the Effect of Du(t) on Arm Impedance • Du is the change in the input to the muscles as a result of our perturbation. • While Du cannot be measured directly, we know that it depends on a number of time-delayed, possibly adaptive error feedback systems. • Time-delayed error feedback from the spinal reflexes • Time-delayed error feedback from the forward model based cortical pathways • Input from inverse model based “open-loop” controller

42. MURI High- Level Control 3.1 Time-delayed effect of Du(t) on Arm Impedance In general, a time delay d in a feedback loop reduces apparent viscosity and adds apparent mass to a system. Example:

43. MURI High- Level Control 3.2 Estimating Du(t) in Terms of Measurable Quantities 1. Effect of Spinal Reflexes

44. MURI High- Level Control 3.2 Estimating Du(t) in Terms of Measurable Quantities 2. Effect of the Inverse model

45. MURI High- Level Control 3.2 Estimating Du(t) in Terms of Measurable Quantities 3. Effect of the Forward Model

46. MURI High- Level Control 3.2 Estimating Du(t) in Terms of Measurable Quantities 4. Effect of Adaptation of the Forward Model Predictable changes in impedance should occur as a function of the kind of model that the system learns as it practices movements in an unstructured environment. If learning is via a forward model, the apparent viscosity must decrease as compared to values obtained before the controller had adapted.

47. MURI High- Level Control Measuring Impedance of the Moving System 0.15 0.1 Y (m) 0.05 0 Perturb the movement in different directions -0.05 -0.1 -0.05 0 0.05 0.1 0.15 X (m)

48. MURI High- Level Control Impedance Early into the Movement 12 10 8 Interaction force 6 Perturb Impedance Controlled force 4 2 Force (N) 0 positionx50 (m) velocityx25 (m/sec) -2 acceleration (m/s 2 ) -4 -6 Inertial dynamics -8 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Time into the movement (s)

49. MURI High- Level Control Impedance in the Middle of the Movement 12 10 Interaction force 8 6 Impedance controller’s force Perturb 4 2 Force (N) 0 Positionx50 (m) Velocity x 25 (m/s) -2 -4 2 Acceleration (m/s ) -6 Dynamic force -8 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Time into the movement (s)

50. MURI High- Level Control Impedance Near End of Movement 12 10 Interaction force 8 6 Impedance Controlled force 4 Perturb Force (N) 2 0 velocityx25 (m/sec) -2 positionx50 (m) -4 acceleration (m/s2) Inertial dynamics -6 -8 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Time into the movement (s)