Neurological Modeling & Cooperation: Automatic Acquisition of Triggered Reactions, a Physiological Approach

1. Neurological Modeling & Cooperation:Automatic Acquisition of Triggered Reactions, a PhysiologicalApproach Cooperative Control of Distributed Autonomous Vehicles in Adversarial Environments2001 MURI: UCLA, CalTech, Cornell, MIT Mao/Massaquoi/Dahleh/Feron May 14, 2001 UCLA

2. Basic Route • Impression • Useful, complex group behavior is based on a combination of relatively simple, perhaps identical Triggered Programmed-Reactions existing within a collection of nominally autonomous agents • Hypothesis • The physiological basis for general behavioral TPRs is the same as that for TPRs used for elemental body movement control/postural regulation

3. Maintenance of upright standing and herding are functionally equivalent in body reference frame d X g d s s s s Examples

4. Observations • Both postural defense, herd containment and dancing via triggered reactions require • Assessment of continuous (though perhaps only piecewise, intermittent) sensory information • Selection of stereotyped movements (motion primitives) that are appropriately scaled and timed to project beyond the anticipated motion of the target • Learning based on goals and reinforcement as dictated by environment and higher control levels results Goals, constraints, Reward/failure Selection, Timing, Scaling Assessment, Prediction Multichannel sensory information Partially pre-programmed behavior

5. Observations (ct'd) • Presumably, scaling, timing and selection alsoautomatically learn to take into account supportive or obstructive features of environment, e.g. • Traction/motion characteristics of floor • responsiveness of target Or • Presence or absence of multiple actuators (e.g. ankles and hips when falling forward, hips only when falling backward) • Presence or absence of other herders on one side vs. another • General sensitivity to environment may be physiological substrate for functionally useful group-aware behavior

6. Modeling Assumptions • Natural motor control system can be represented as a hierarchy consisting of a high level, largely conscious, discrete state-machine-type ‘computer’ and a low/intermediate level, largely unconscious, continuous signal processing controller. • In between are structures enabling the development of flexible, simple, semi-conscious ‘motor programs’ (behaviors) that address/adhere to the goals and constraints provided by the high level computer

7. Natural Sensorimotor Control • That is, our Interest: • Understand Control, Assessment and Learning at the interface between higher and intermediate/low functional levels of natural sensorimotor system Discrete Behavior Control, Assessment & Adaptation (conscious/preconscious?) MURI Continuous Action Control, Assessment & Adaptation (subconscious?) Action Production Action Monitoring Environment

8. Natural Sensorimotor Control • More specifically, • Natural Sensorimotor control Hierarchy • High level Goals (conscious) • e.g. win point vs. conserve energy • Strategic Planning/Decisions (conscious) • e.g. return to right rear baseline • Tactical Objectives (preconscious/”overlearned’?) • e.g. contact ball with racket face having particular orientation and velocity • Tactical Assessment/Planning/Decisions (preconscious/’overlearned’?/development of “motor program”) • assess/predict ball trajectory, spin, body location in court • use forehand, assume particular posture, generate specific trajectory MURI

9. Natural Sensorimotor Control • Natural Control Hierarchy (cont’d) • Action (force, position) generation& on-line control (subconscious) • Action (continuous trajectory) improvement (optimization?) with practice (subconscious motor learning) • Behavior (discrete program, trajectory) improvement (optimization?) with practice (conscious--> preconscious: ‘tactical motor learning’, ‘motor programming’) • Behavior improvement (optimization?) with practice (conscious: ‘strategic motor learning’, ‘gamesmanship’) MURI

10. Natural Sensorimotor Control SENS • Natural Sensorimotor Control System MTR (parietal) ASSOC ST (frontal) ASSOC MT BG Interface between high and intermediate/low control levels involves sensorimotor and association cortices (especially frontal) and the Basal Ganglia. These link ‘automatic’ behavior and reward. Cerebellum likely contributes optimization Cbl

11. Human motor control principal information flow (adapted from V. Brooks, 1986) “highest level” PLANS (strategy) “middle level” (“high” and “intermediate”) PROGRAMS (tactics) “lower level” ACTION (force, velocity) Caudate & GP Putamen & GP “Motor Servo” Brainstem or Spinal Cord Segment Frontal & Parietal Assoc Ctx Mtr Ctx Neural signals ------------------ executive sensory consciousness gradient Im Ant Cbl L Ant Cbl Muscle & tendon, Joints, skin M. Cbl Flocc Cbl Body Force/ Motion Vestib Visual

12. MURI Goals • MURI to specifically study ‘Programming’ of Triggered Reaction Loops “highest level” PLANS, ALGORITHMS (free assoc, strategy) “high level” PROGRAMS (discrete control) (tactics:trajectories, cues) “intermediate level” CONTROL (continuous control) (stability, tracking, stiffness, scaling, movement time) Caudate & GP (Basal Ganglia) Putamen & GP, SN (Basal Ganglia) Motor Servo (proprioceptive) Frontal & Parietal Assoc Ctx Frontal & Parietal Peri- Sensorimotor Ctxs Mtr Ctx Im Ant Cbl L Post Cerebellum L Ant Cbl TPR Loop Circuitry M. Cbl Flocc Cbl

13. Proposed MURI project (Year 1) • Acquisition of triggered motor reactions Video monitor showing virtual targets and environment Robot arm Implementing virtual targets and environment

14. Proposed MURI project questions: with respect to physiological structures known or suspected to be involved in TPRs (Year 1) • What are the motion primitives? • How are they generated, scaled, timed, triggered? • What and how is continuous sensory information used? • How is prediction performed … evidence for internal models? • How is reinforcement/suppression mediated? • What is the statistical nature of the learning and programming?

15. Background studies and resources • Existing models for intermediate and low/level motor control based on cerebellar and sensorimotor cortical physiology • Robot arm laboratory • Access to human subjects including those with diseases of the basal ganglia and cerebellum

16. Research funded by NASA, ONR Beyond Year 1 • Useful, complex group behavior may emerge from relatively simple, perhaps identical Triggered Programmed-Reactions existing within a collection of nominally autonomous agents • Link to emergent group behavior possible via experimental observations / prior and similar approaches in Air Traffic Control (eg Mao, Feron and Bilimoria, IEEE ITS, 06/01)