Backwards Reachable Set

2. Backwards Reachable Set All states for which, for all possible control actions, there is a disturbance action which can drive the system state into a region G(0) in time t Backwards Reachable Set Reachability as game: disturbance attempts to force system into unsafe region, control attempts to stay safe

3. Reachable Set Propagation Theorem [Computing ]: where is the unique Crandall-Evans-Lions viscosity solution to: [Mitchell, Bayen, Tomlin 2005]

4. Backwards Reachable Set: Safety unsafe Backwards Reachable Set Safety Property can be encoded as a condition on the system’s reachable set of states In blue, system will stay safe In red, system may become unsafe On boundary, apply control to stay out of red

5. ‘evader’ (control) ‘pursuer’ (disturbance) Example 1: Aircraft Collision Avoidance Two identical aircraft at fixed altitude & speed: y v y u x v d

6. y x y Continuous Reachable Set

7. safety filter’s input modification evader’s actual input unsafe set collision set pursuer evader evader’s desired input pursuer’s input Collision Avoidance Filter Simple demonstration • Pursuer: turn to head toward evader • Evader: turn to head right Movies…

8. Collision Avoidance Control • http://www.cs.ubc.ca/~mitchell/ToolboxLS/

9. Overapproximating Reachable Sets Exact: Approximate: Overapproximative reachable set: [Khrustalev, Varaiya, Kurzhanski] • Polytopic overapproximations for nonlinear games • Subsystem level set functions • “Norm-like” functions with identical strategies to exact [Hwang, Stipanović, Tomlin] ~1 sec on 700MHz Pentium III (vs 4 minutes for exact)

11. Computing Reach Sets for Hybrid Systems modes 2 3 K 1 1 2 iterations 3 n safe unsafe unsafe

12. Reach Sets: uncontrollable predecessor modes 2 3 K 1 1 “safe” 2 iterations 3 n uncontrolled transition unsafe

13. Reach Sets: controllable predecessor modes 2 3 K 1 1 “safe” 2 iterations 3 n controlled transition safe

14. Reach Sets: Variational Inequality States which reach G without hitting E first: modes 2 3 K 1 1 2 iterations 3 where n subject to

15. Reach Sets: Iterate modes 2 3 K 1 1 2 iterations 3 n

16. Can separation assurance be automated? Requires provably safe protocols for aircraft interaction Must take into account: • Uncertainties in sensed information, in actions of the other vehicle • Potential loss of communication • Intent, or non-intent

17. unsafe set with choice to maneuver or not? Example 2: Protocol design unsafe set without maneuver safe unsafe ? unsafe set with maneuver

18. Protocol Safety Analysis safe with switch • Ability to choose maneuver start time further reduces unsafe set unsafe with or without switch safe without switch unsafe to switch

19. controlled transition (s1) q5 qs forced transition safe at present always safe safe to s1 SAFE q3 q4 safe at present will become unsafe safe to s1 safe at present always safe unsafe to s1 q1 q2 qu safe at present will become unsafe unsafe to s1 unsafe at present will become unsafe unsafe to s1 UNSAFE Implementation: a finite automaton • It can be easier to analyze discrete systems than continuous: use reachable set information to abstract away continuous details q5 qu q3 q4 q2 q1

20. Example 2: Closely Spaced Parallel Approaches San Francisco Airport 750 ft separation

21. Example 3: Closely Spaced Approaches EEM Maneuver 1: accelerate EEM Maneuver 2: turn 45 deg, accelerate EEM Maneuver 3: turn 60 deg [Rodney Teo] evader

22. Sample Trajectories Segment 2 Segment 3 Segment 1

23. Tested on the Stanford DragonFly UAVs Dragonfly 2 Dragonfly 3 Ground Station

24. Tested at Moffett Federal Airfield Accelerate and turn EEM Put video here North (m) East (m) Separation distance (m) EEM alert Above threshold time (s)

25. Tested at Moffett Federal Airfield Coast and turn EEM Put video here North (m) East (m) Separation distance (m) EEM alert Above threshold time (s)

26. Tested at Edwards Air Force Base T-33 Cockpit [DARPA/Boeing SEC Final Demonstration: F-15 (blunderer), T-33 (evader)]

27. Photo courtesy of Sharon Houck; Tests conducted with Chad Jennings

28. Implementation: Display design courtesy of Chad Jennings, Andy Barrows, David Powell R. Teo’s Blunder Zone is shown by the yellow contour Red Zone in the green tunnel is the intersection of the BZ with approach path. The Red Zone corresponds to an assumed 2 second pilot delay. The Yellow Zone corresponds to an 8 second pilot delay

29. R. Teo’s Blunder Zone is shown by the yellow contour Red Zone in the green tunnel is the intersection of the BZ with approach path. The Red Zone corresponds to an assumed 2 second pilot delay. The Yellow Zone corresponds to an 8 second pilot delay

30. Map View showing a blunder The BZ calculations are performed in real time (40Hz) so that the contour is updated with each video frame.

31. Map View with Color Strips The pilots only need to know which portion of their tunnel is off limits. The color strips are more efficient method of communicating the relevant extent of the Blunder zone

32. Experimental Platform: STARMAC The Stanford Testbed of Autonomous Rotorcraft for Multi-Agent Control

33. Example 4: Collision Avoidance Pilots instructed to attempt to collide vehicles

34. Example 5: Aircraft Autolander Aircraft must stay within safe flight envelope during landing: • Bounds on velocity (), flight path angle (), height () • Control over engine thrust (), angle of attack (), flap settings • Model flap settings as discrete modes • Terms in continuous dynamics depend on flap setting body frame wind frame inertial frame

36. Autolander: Synthesizing Control For states at the boundary of the safe set, results of reach-avoid computation determine • What continuous inputs (if any) maintain safety • What discrete jumps (if any) are safe to perform • Level set values and gradients provide all relevant data

37. TOGA TOGA flaps retracted maximum thrust flaps retracted maximum thrust flare flare flaps extended minimum thrust flaps extended minimum thrust rollout rollout flaps extended reverse thrust flaps extended reverse thrust slow TOGA flaps extended maximum thrust Application to Autoland Interface • Controllable flight envelopes for landing and Take Off / Go Around (TOGA) maneuvers may not be the same • Pilot’s cockpit display may not contain sufficient information to distinguish whether TOGA can be initiated existing interface controllable TOGA envelope intersection revised interface controllable flare envelope

38. Aircraft Simulator Tests • Setup • Commercial flight simulator, B767 pilot • Digital video of primary flight display • Maneuver • Go-around at low speed, high descent rate • Goal • Determine whether problematic behavior predicted by our model is possible in aircraft flight simulator

39. Aircraft Simulator Results • Produced unexpected behavior • Non-standard procedure; Unable to duplicate • Validated types of problems addressed by this method

40. Backwards Reachable Set: Safety unsafe Backwards Reachable Set Safety Property can be encoded as a condition on the system’s reachable set of states In blue, system will stay safe In red, system may become unsafe On boundary, apply control to stay out of red

41. Backwards Reachable Set: Capture Backwards Reachable Set desired Capture property can also be encoded as a condition on the system’s reachable set of states

42. Maneuver sequencing, “Reachavoid” Maneuver sequencing is accomplished by stringing together capture sets, starting from the target set and working backwards Target Set Unsafe Set Avoid sets can be combined with capture sets to guarantee safety

43. Impulse Example 5: Quadrotor Back-Flip Recovery Drift • Divide flip into three modes • Difficult problem: • Hitting some target sets while avoiding some unsafe sets • Solution: • Analyze rotational dynamics and vertical dynamics separately

44. Back-flip: Method (1) Recovery Drift Impulse • Identify target region in rotational state space for each mode • Use reachable sets to calculate capture basinfor each target • Dynamic game formulation accounts for worst-case disturbances • Verify that target of each mode is contained by capture basin of next mode

45. Back-flip: Method (2) • Identify unsafe region in vertical state space for final mode • Use reachable sets to propagate unsafe set for each mode • Dynamic game formulation accounts for worst-case disturbances • Verify that control keeps state out of unsafe set

47. Back-Flip: Results

48. Back-Flip: Results • Assumptions Validated • Safety Guaranteed • Reachability Demonstrated

49. Example 6: Automated aerial refueling Desired Target Set

50. Capture Set and Unsafe Set Computation Result