Reachability-based Controller Design for Switched Nonlinear Systems

1. Reachability-based ControllerDesign for Switched NonlinearSystems EE 291E / ME 290Q Jerry Ding 4/18/2012

2. Hierarchical Control Designs • To manage complexity, design of modern control systems commonly done in hierarchical fashion • e.g. aircraft, automobiles, industrial machinery • Low level control tend to use continuous abstractions and design methods • ODE model • Stability, trajectory tracking • Linear/Nonlinear control methods • High level control tend to use discrete abstractions and design methods • Finite state automata, discrete event systems • Logic specifications of qualitative behaviors: e.g. LTL • Model checking, supervisory control

3. Challenges of Interfacing Layers of Control • Problem becomes more difficult at interface: • Closed loop behavior results from composition of discrete and continuous designs • Discrete behaviors may not be implemented exactly by continuous controllers • Continuous designs may be unaware of high level specifications • In safety-critical control applications, specifications often involves stringent requirements on closed-loop behavior • Current design approaches involve a mixture of heuristics and extensive verification and validation

4. Hybrid Systems Approach • Capture closed-loop system behavior through hybrid system abstraction

5. Hybrid Systems Approach • Formulate design methods within the framework of hybrid system theory • Challenges: • Nonlinear dynamics, possibly with disturbances • Controlled switching: switching times, switching sequence, switching policy • Autonomous switching: discontinuous vector fields, state resets

6. Reachability-Based Design for Switched Systems • Consider subclass of hybrid systems with: • Controlled switches, no state resets • Fixed mode sequence • Variable mode sequence • Nonlinear continuous dynamics, subject to bounded disturbances • Design controllers to satisfy reachability specifications • Reach-avoid problem: Given target set R, avoid set A, design a controller to reach R while avoiding A • Methods based upon game theoretic framework for general hybrid controller design • [Lygeros, et al., Automatica, 1999] • [Tomlin, et al., Proceedings of the IEEE, 2000]

7. Outline • Switched Systems with Fixed Mode Sequences: • Design of Safe Maneuver Sequence for Automated Aerial Refueling (AAR) • Switched Systems with Variable Mode Sequences: • Sampled-data switched systems • Controller synthesis algorithm for reach-avoid problem • Application example: STARMAC quadrotor experiments

8. Automated Aerial Refueling Procedures

9. Discrete Transitions Detach 1 Rejoin High Level Objective: Visit waypoint sets Ri, i = 1,…,6, in sequence Precontact Postcontact Detach 2 Start Contact End

10. Continuous Dynamics Relative States: x1, x2 = planar coordinates of tanker in UAV reference frame x3 = heading of tanker relative to UAV Controlled inputs: u1 = translational speed of UAV u2 = turn rate of UAV Disturbance inputs: d1 = translational speed of Tanker d2 = turn rate of Tanker Low Level Objective: Avoid protected zone A around tanker aircraft

11. Maneuver Sequence Design Problem • Given waypoint sets Ri, protected zone A, design continuous control laws Ki(x) and switching policies Fi(x) such that • 1) The hybrid state trajectory (q, x) passes through the waypoint sets qi× Riin sequence • 2) The hybrid state trajectory (q, x) avoids the protected zones qi× A at all times • Design approach: • Select switching policy as follows: in maneuver qi, switch to next maneuver if waypoint Ri is reached • Use reachable sets as design tool for ensuring • safety and target attainability objectives for each maneuver • compatibility conditions for switching between maneuvers

12. Capture sets and Unsafe sets

13. Computation of Reachable Sets • Use terminal condition to encode avoid set • Unsafe set computation (Mitchell, et al. 2005): • Let be the viscosity solution of Then • Capture set computation similar • Numerical toolbox for MATLAB is available to approximate solution • [Ian Mitchell, http://www.cs.ubc.ca/~mitchell/ToolboxLS/, 2007]

14. Maneuver Design Using Reachability Analysis • For mode qN • 1) Design a control law to drive RN -1 to RN • 2) Compute capture set to first time instant N such that

15. Maneuver Design Using Reachability Analysis • For mode qN • 3) Compute unsafe set, and verify safety condition • Modify control law design as necessary

16. Maneuver Design Using Reachability Analysis • For modes qk, k < N • 3) Iterate procedures 1-3 recursively • For q1 , R0 = X0 , where X0 is the initial condition set

17. Properties of Control Law • Continuous control laws designed in this manner satisfy a reach-avoid specification for each maneuver: • Reach waypoint set Ri at some time, while avoiding protected zone A at all times • Furthermore, they satisfy a compatibility condition between maneuvers • This ensures that whenever a discrete switch take place, the specifications of next maneuver is feasible • Execution time of refueling sequence is upper bounded by

18. Specifications for Aerial Refueling Procedure • Target Sets of the form • Avoid sets of the form • Control laws of the form

19. Capture Set and Unsafe Set Computation Result Precontact (Mode q2) Time Horizon

20. Simulation of Refueling Sequence Input bounds Unsafe Set For Detach 1 Collision Zone Target Set Radius Target Set Collision Set Radius Capture Set For Detach 1

21. Accounting for Disturbances • Capture sets and unsafe sets can be modified to account for fluctuations in tanker velocity using disturbance set Unsafe set for contact maneuver without disturbances Collision Zone In UAV Coordinates Reachable set slice at relative angle 0 Unsafe set for contact maneuver with 10% velocity deviation Rescaled coordinates: distance units in tens of meters

22. Outline • Switched Systems with Fixed Mode Sequences: • Design of Safe Maneuver Sequence for Automated Aerial Refueling (AAR) • Switched Systems with Variable Mode Sequences: • Sampled-data switched systems • Controller synthesis algorithm for reach-avoid problem • Application example: STARMAC quadrotor experiments

23. Switched System Model – Dynamics Discrete State Space Continuous State Space Continuous Dynamics Reset Relations

24. Switched System Model – Inputs • Sampled-data system for practical implementation • Quantized input for finite representation of control policy Switching Signal Piece-wise constant Continuous Input Disturbance Time-Varying 0 T 2T 3T 4T 5T

25. Switched System Model – Control and Disturbance Policies • On sampling interval [kT, (k+1)T], define One step control policy One step disturbance strategy (k+1)T (k+1)T kT kT

26. Outline • Switched Systems with Fixed Mode Sequences: • Design of Safe Maneuver Sequence for Automated Aerial Refueling (AAR) • Switched Systems with Variable Mode Sequences: • Sampled-data switched systems • Controller synthesis algorithm for reach-avoid problem • Application example: STARMAC quadrotor experiments

27. Problem Formulation • Given: • Switched system dynamics; for simplicity, assume that • Target set R • Avoid set A Target set Avoid set Mode Mode

28. Problem Formulation • Compute set of states (q, x) that can be controlled to target set while staying away from avoid set over finite horizon • Call this reach-avoid set Target set Avoid set Reach-avoid set Mode Mode

29. Problem Formulation • For any (q, x) in the reach-avoid set, automatically synthesize a feedback policy that achieves the specifications Target set Avoid set Reach-avoid set Mode Mode

30. One Step Capture and Unsafe sets • For each , compute one step capture and unsafe sets assuming • over one sampling interval • One step capture set • One step unsafe set where is solution of on

31. Reach-avoid Set Computation – Step 1 • For each , compute one step reach-avoid set using set difference Mode Mode For sets represented by level set functions The set difference is represented by

32. Reach-avoid Set Computation – Step 2 • Compute feasible set for one step reach-avoid problem, by taking union over Mode Mode For sets represented by level set functions The set union is represented by

33. Reach-avoid Set Computation – Iteration • Iterate to compute the reach-avoid set over [0,NT] • By induction, can show that Initialization: for to end Return:

34. Reach-avoid control law synthesis • At time k < N Step 1: Obtain state measurement Step 2: Find minimum time to reach

35. Reach-avoid control law synthesis • At time k < N Step 3: Find a control input such that Step 4: Apply input and iterate steps 1-3

36. Explicit Form of Control Laws • Explicit control laws given by for where • Number of reachable sets required is given by Number of quantization levels in mode qi Length of time horizon Number of discrete modes

37. Outline • Switched Systems with Fixed Mode Sequences: • Design of Safe Maneuver Sequence for Automated Aerial Refueling (AAR) • Switched Systems with Variable Mode Sequences: • Sampled-data switched systems • Controller synthesis algorithm for reach-avoid problem • Application example: STARMAC quadrotor experiments

38. High Level Control Gumstix PXA270, or ADL PC104 Carbon Fiber Tubing Low Level Control Atmega128 Fiberglass Honeycomb GPS Novatel Superstar II Sensorless Brushless DC Motors Axi 2208/26 Electronic Speed Controllers Castle Creations Phoenix-25 Inertial Meas. Unit Microstrain3DM-GX1 UltrasonicRanger Senscomp Mini-AE Battery Lithium Polymer STARMAC Quadrotor Platform

39. Experiment Setup • Objectives: • Drive a quadrotor to a neighborhood of 2D location in finite time, while satisfying velocity bounds • Disturbances: model uncertainty, actuator noise • System model

40. Reach-avoid Problem Set-Up • Target Set: +/- 0.2 m for position, +/- 0.2 m/s for velocity • Avoid Set: +/- 1 m/s for velocity • Time Step: 0.1 seconds, 25 time steps • Pitch and roll commands: • Disturbance bounds:

41. Reach-avoid Set - Plots

42. Reach-avoid Set - Plots Reach-avoid at Time Step 1 for All Inputs

43. Reach-avoid Set - Plots

44. Experimental Results

45. Experimental Results

46. Experimental Results • Moving car experiment

47. References • John Lygeros, Claire Tomlin, and S. Shankar Sastry. Controllers for reachability specifications for hybrid systems. Automatica, 35(3):349 – 370, 1999. • Claire J. Tomlin, John Lygeros, and S. Shankar Sastry. A game theoretic approach to controller design for hybrid systems. Proceedings of the IEEE, 88(7):949–970, July 2000. • Jerry Ding, Jonathan Sprinkle, S. Shankar Sastry, and Claire J. Tomlin. Reachability calculations for automated aerial refueling. In 47th IEEE Conference on Decision and Control, pages 3706–3712, Dec. 2008. • Jerry Ding, Jonathan Sprinkle, Claire Tomlin, S. Shankar Sastry, and James L. Paunicka. Reachability calculations for vehicle safety during manned/unmanned vehicle interaction. AIAA Journal of Guidance, Control, and Dynamics, 35(1):138–152, 2012.

48. References • Jerry Ding and Claire J. Tomlin. Robust reach-avoid controller synthesis for switched nonlinear systems. In 49th IEEE Conference on Decision and Control (CDC), pages 6481–6486, Dec. 2010. • Jerry Ding, Eugene Li, Haomiao Huang, and Claire J. Tomlin. Reachability-based synthesis of feedback policies for motion planning under bounded disturbances. In IEEE International Conference on Robotics and Automation (ICRA), pages 2160 –2165, May 2011.