Cooperative and Noncooperative Operations of Swarms

1. Cooperative and Noncooperative Operations of Swarms Hoam Chung, David Shim, Mike Eklund, Shankar Sastry University of California, Berkeley www.swarms.org

2. Cooperative Operations of Swarms Heterogeneous formation flight of MD500s and UH-60s • Various helicopter formations are now being used in many applications • Some level of automation during formation flight can reduce pilot stress and fatigue • Few research results on autonomous helicopter formation exist due to helicopter’s complicated dynamic properties, and technical difficulties SWARMS

3. Mesh Controller • Mesh Controller tasks: • Obtain the leader and 2 neighboring helicopters’ current positions • Compute mesh stable trajectories based on the acquired position information and send commands to the navigation computer Neighbor 2 Wireless Token Ring Mesh Controller Flight Computer Leader RS-232 On UAV Neighbor 1 SWARMS

4. 2001 Experiment SWARMS

5. 2001 Experiment With leader info Without leader info SWARMS Animation by A. Pant and X. Xiao

6. Mesh Stable Controllers are OK, but… • The use of leader information improves the performance of the autonomous formation flight • For a heterogeneous mesh, an extension of mesh stability theory should be considered • “Mesh Stability” does not mean the “Safety” It’s a starting point for autonomous formation flight SWARMS

7. Model Predictive Control • Computes control inputs using real-time optimization • Shows better performance than non-predictive controls • Can consider various safety constraints in on-line manner • Easily accommodates adaptive disturbance rejection algorithms SWARMS

8. Model Predictive Control • Structure of MPC Compute control inputs minimizing gap errors considering helicopter dynamics at every sampling time optimization can deal with various constraints Positions of neighboring vehicles Control inputs Information of formation velocities desired gaps Weather conditions/Mission characteristics SWARMS

9. Simulation Scenario 1 Heli0 t n Heli1 Heli2 • 3DOF Point mass model • Homogeneous formation • Echelon right (45 deg. off lead) • Forward flight at 67.5 mi/h • Disturbance on 2nd helicopter • No safety constraints, no explicit disturbance rejection Heli3 SWARMS * Formation from FM 1-112 Attack Helicopter Operations, Headquarters, Dept. of the Army

10. Animation Animation generated by MATLAB SWARMS

11. Simulation Mesh stability is achieved without any explicit disturbance rejection algorithm SWARMS

12. Bi-directional Information Flow • For safer autonomous formation, the communication between neighbors should be bi-directional • In case of mesh stability concept, it’s difficult to deal with bi-directional information Information flow What will happen if directions of disturbances are reversed? SWARMS

13. Bi-directional Information Flow t • In case of MPC, simple redefinition of • error signal can deal with bi-directional • information flow j-1 n • For example, we can redefine the error vector of jth helicopter so that - Keep the center between j-1 and j+1 in tangential direction - Keep the desired gap in normal direction j • This flexibility of MPC allows various formations in 3D space with enhanced safety j+1 SWARMS

14. Forming a Formation • Adding vehicles one by one Merging procedures on the vehicle B: Establish communication with vehicle A Acquire variables about existing formation from vehicle A Compute a merging trajectory and track it Finish merging procedure if the gap error is within a certain bound Engage formation controller A B SWARMS

15. Forming a Formation • Adding vehicles group by group A B b + a b a Merging procedures on the group B: Vehicle b establishes communication with vehicle a in A Vehicle b acquires variables about leading formation from vehicle a Compute merging trajectory Propagate acquired variables and computed trajectory through B Track the merging trajectory Finish merging procedure SWARMS

16. Terminating a Formation • Terminating a formation one by one Termination procedures on the vehicle B: Compute a trajectory to get more gap from the existing formation Notify termination schedule to vehicle A Track the computed trajectory Send “Separation Completed” to vehicle A and close the communication channel Disengage formation controller and give control back to pilots A B SWARMS

17. Terminating a Formation • Terminating a formation group by group B A b a b a Termination procedures on b: Compute a trajectory to get more gap from the leading formation Propagate the computed plan to followers Notify termination schedule to vehicle a Track the computed trajectory Send “Separation Completed” to vehicle a and close the communication channel between a and b SWARMS

18. Modifying a Formation in the Air • Modification of a formation • MPC is basically a tracking controller • By manipulating local formation variables(gap info), reconfiguration of a formation without reorganization can be easily achieved SWARMS

19. Simulation Scenario 3 t Heli0 Heli1 n Heli2 Heli4 Heli3 Heli5 Heli6 • 3DOF Point mass model • Heterogeneous formation • 3D Vee formation (45 deg. off lead, 5m gaps in n, t, and z) • Forward flight at 67.5 mi/h, 5m(about 1.7 rotor radius) spacing • Disturbances on the leader and the last follower in right wing • No safety constraints, no explicit disturbance rejection SWARMS

20. Simulation of formation split and rejoin SWARMS

21. Formation Rejoining • Consider a situation that a vehicle is approaching to the existing 3D Vee formation for joining 1 Safe region 2 SWARMS

22. Formation Rejoining • Objectives for a perfect formation rejoining • A joining vehicle is positioned at predefined location in the formation • When it finishes the procedure, its velocities and heading should be close enough with those of the entire formation • During the procedure, the joining vehicle should remain in a safe region • The motion of the future neighbor acts like a disturbance during the joining procedure • For the vehicle in the formation, the first priority is maintaining the formation • Disturbances deteriorate ideal navigation conditions always exist SWARMS

23. Formation Rejoining • The formation joining problem can be regarded as a differential gaming under input/state constraints • Following question should be answered: • Does RHC scheme guarantee reachability under disturbances? • If so, how close is the reachable set rendered by RHC to that by infinite-horizon problem? SWARMS

24. Finite-horizon Differential Game SWARMS

25. Finite-horizon Differential Game • The reachable set by the solution of FHODG problem is identical with that of a modified infinite-horizon problem • As becomes small, the reachable set of RHC approaches to that of infinite-horizon solution with SWARMS

26. Finite-horizon Differential Game • This lemma plays important role in designing a receding horizon controller satisfying the condition SWARMS

27. Finite-horizon Differential Game • The reachable set can be enlarged by introducing longer prediction horizon • These theorems and lemma tell us that, if the FHODG is feasible with some prediction length L, then it guarantees a successful formation rejoining SWARMS

28. Works in Progress • A RHC scheme will be designed for 3D nonlinear kinematics plus linear dynamics model • Various numerical methods are now being investigated • Continuation method – Ravio et al. • Piecewise linear approximation and SQP – Fabien • Lagrange multipliers method – Sutton and Bitmead • For reducing computational burdens, the performance of open-loop and Stackelberg solutions under RHC scheme will be evaluated • The algorithm will be implemented and tested on BEAR hardware-in-the-loop systems SWARMS

29. Collision Avoidance using MPC • Five helicopters are given destination points. • The shortest (optimal) trajectory will lead to a collision. • Each vehicle can detect other vehicles position only within the sensing/communication region. • Can each vehicle fly safely and optimally? Unsafe Desired Trajectory Resolved by NMPTC with Collision Avoidance SWARMS

30. Collision Avoidance using MPC • Two UAVs are intentionally set on a head-on crash course • Model-predictive control-based trajectory planner computes safe trajectories with sufficient clearance in real time • Each vehicle’s current coordinate is used for MPC at each computation SWARMS May 2003

31. Collision Avoidance using MPC SWARMS

32. Obstacle Avoidance System • Dynamic path planning: real-time path generation using model predictive control • Sensing: onboard 3D laser scanner or preprogrammed obstacle maps • Experiment system: Berkeley UAV architecture implemented on Yamaha industrial helicopter platform with 3D laser scanner SWARMS

33. Richmond Field Station, UC Berkeley, Richmond, California Ground Station Obstacles Original path Vehicle Launching Pt. Adjusted path by MPC Urban Flight Experiment 10’ X 10’ Easy-up Canopy • 6 canopies to simulate urban environment • Secured by stakes at four corners • Resistant to wind gust of rotor downwash • Sufficient distances each other for helicopters to fly through SWARMS

34. Urban Navigation Experiment SWARMS

35. Non Cooperative Actions of Swarms: David Shim, Jongho Lee, Mike Eklund, Jonathan Sprinkle, Shankar Sastry

36. Aerial Pursuit-Evasion in MPC framework • Pursuer wants to position itself in a good position to “shoot down” the evader, e.g., follow the target’s tail and align its heading with the relative position vector, XE-XP • Evader wants to shake off the pursuer, e.g., get out of the hotspot • Pursuer and evader avoid colliding into each other within a closed 3-D space • State variables such as roll and pitch angles are constrained SWARMS

37. Aerial Pursuit-Evasion in MPC framework • Pursuer and Evaderin a closed 3-D space with the additional cost • Cost function also includes collision avoidance between aircraft and other obstacles including terrain Illustration by Mike Eklund and Jonathan Sprinkle SWARMS

38. ANIMATION 2004 SWARMS

39. Multiplayer PEGs: Proposed Solution • A close analogy is football: • Multi player • Initial (global) strategies well defined • Limited (local) coordination after the snap • What can we learn? • How can we apply this? • How far does the analogy go? SWARMS Back

40. Multiplayer PEGs • Preseason (Off-line precomputed strategy) • Play book: • Evaluate strategies and configurations that will maximize chance of success based on best estimate of other team’s tactics • Practice and preseason games: • Test playbook and find problems • Game time (On-line adaptive strategy) • Choose play based on best knowledge and experience • Line up (in best detection configuration, not necessarily static) • Execute the play • Active and reactive actions (respond to detected evader) • Local communication • Adapt to evolving behavior • Learn from experience, repeat as necessary (Learning by Doing) SWARMS Back