Multi-agent Collaborative Flight Experiment Karl Hedrick UC Berkeley

1. Multi-agent Collaborative Flight Experiment Karl Hedrick UC Berkeley

2. CIRPAS, Camp Roberts, CA Operated by the Naval Post Graduate School

3. Collaborative UAV Flight Test • GOALS of the August, 2006 Experiment Thrust 1. Distributed collaboration with limited communications a.) multi-vehicle, multi-step task allocation b.) limited ground-air and air-air com c.) user task cancellation/reallocation d.) agent created tasks e.) task prioritization f.) no fly zone filter and re-planner g.) Simple human/UAV team interface Thrust 2. Vision-based river following. a.) Ability to identify and search for the desired structure (river). b.) Ability to accurately track the river once identified. c.) Ability to accurately map the boundaries of the river.

4. Collaboration Research Goals In General • Studydistributed mechanisms for the collaboration of Unmanned Aerial Vehicles (UAV) • Generalize a large number of missions under one framework • Surveillance/Mapping • Border Patrol • Search & Rescue • Convoy Protection, etc. • Emphasis on Robustness rather than Optimization: • Addition and Deletion of Tasks • Addition and Removal/Failure of Agents • Limited and/or Failed communication

5. C3UV Collaboration Software Agent in range of user User • GOALS • Transmit desired mission from user to agents • Provide user with fused information from agents • Decompose and assign tasks among agents in response to dynamic mission definition • Accomplish tasks in an efficient and robust manner Mission state est. Mission state estimate Agent out of range Command station New tasks Cancel tasks

6. Communication Infrastructure Piccolo Autopilot Piccolo Autopilot PC104 PC104 User 2.4 GHz ethernet New tasks Cancel tasks 900 MHz radio Piccolo Groundstation Command station

7. The Mission • User defines mission in terms of tasks • Philosophy • “The user specifies whathe or she would like accomplished. • The system decides howto do so efficiently.”

8. Mission Commander Interface

9. Graphical User Interface • C:\Documents and Settings\Sonia\Desktop\CommanderSDK.exe

10. Monitor Area Task Task details Goal Detect any velocity-bounded intruder that cannot leave a defined box Trajectory UAV path depends on maximum speed of intruder; path becomes a “lawnmower” trajectory if maximum speed is zero

11. Guaranteed Search Task Task details Goal Detect any intruder traveling from a known start point with a bounded velocity Simulation Result 500 of 500 intruders detected in simulation

12. Avoiding a no fly zone: active filter Task Controller No fly zone Visit task Desired autopilot command No Fly Zone Filter Safe autopilot command Autopilot

13. BLCC- Berkeley “Language” for Collaborative Control • Define the mission and communicate it to team members • Define the “state” of each agent • Define the mission “state” • Allow for faults • Allow for conflict resolution • Define the information to be communicated between agents.

14. Agents (UAVs) • Transition Logic: Governs transitions of tasks and subtasks • Communication: Deconflicts plans and synchronizes information between agents vs. • Planner(ex. path-planner): calculates cost, generates plan and chooses “todo” • Low-level Controller (ex. waypoint tracker)

15. Task-Point List • Every process and each agent communicates primarily through the task-point list • A task-point list exists for each task and is manipulated by each process to generate a desired mode/task/mission 1 2 3 4

16. Real-Time Task Allocation Algorithms Multiple agents/Multiple Tasks • Emphasis on real-time computation and robustness to communication limitations and agent failure. • Recent progress on optimal and sub-optimal algorithms

17. Task Allocation • Given n UAVs and m tasks, how do we assign tasks to UAVs? • Assume that each task is simply a point to be visited, with some time spent at that point. • Neglect UAV turn rate constraints – assume constant velocity • For each UAV, let a tour be an ordered set of targets that it will visit • Let the cost of tour be the total time required to complete. For a constant velocity UAV with no turn rate constraint, this time corresponds to distance. • Often this is posed as an instance of the multiple traveling salesman problem

18. Multiple Traveling Salesman • The Multiple Traveling Salesman Problems focuses on minimizing total cost. For n UAVs, with the cost of a tour for UAV j = Tj • Our problem differs: we should focus on minimizing the max cost of any tour • Given that we’re working with constant velocity UAVs, the cost in fuel of having a UAV circle is the same as having it do some work. • For our problem, this corresponds to a minimum clock time problem. This problem is often referred to as the min-max Vehicle Routing Problem.

19. Min-Max Vehicle Routing • The min-max Vehicle Routing Problem is NP hard. There is no polynomial time solution. • We’d like to develop algorithms find near-optimal solutions to the min-max problem. • In practice, we’d like to build algorithms that are robust to communication losses; with perfect communication, we’d like these algorithms to achieve near optimal performance.

20. The Greedy Algorithm • In constructing a tour, let the UAV with the lowest cost function for its partial tour choose the next task. • This algorithm leads to balanced tours among UAVs: all UAVs perform tours of roughly equal cost. • For the min-max VRP, optimal solutions will contain tours balanced to within the maximum distance between any two tasks. • This is a fast algorithm that creates balanced tours • Sub-optimal • 2 questions: • How well does this work? • How do we implement this in a distributed system with limited communication?

21. Approximate Multi-step Distributed Min-Max Vehicle Routing Algorithm 0 Communication and 1 Computation

22. Example continued… 1 Communication and 1 Computation

23. Example continued… 1 Communication and 2 Computation

24. Example continued… 2 Communication and 2 Computation

25. Collaboration Architecture Information Base Communication Computation Decision Functions Integrates MSEA and MSEB Retains most up-to-date information Internal State ISA Complete Redo Fault Mission State EstimateMSEA Choose Task Execution Agent A MSEB MSEA Receive Broadcast User Plan

26. How we implement in software Collaboration Sensors Sensor Data Mission Plan Aircraft Commands A2A Wireless Piccolo Interface Datahub Collaboration info Telemetry Mission Plan AC Commands User Commands High Level Telemetry Task Execution Payload Interface Controllers Safe Flight Controller

27. Sig Rascal 110 airframe • Balsa frame remote control aircraft kit with 110” wingspan • Modifications: • 32 cc gasoline engine with vibration isolation mounts • Dual fuel tanks for 60 min flight time • Carbon fiber reinforcement to support payload • 26 lb takeoff weight • Piccolo avionics system

28. Berkeley Air Force

29. PC104 stack and payload tray • PC104 with 700 MHz Pentium III processor • 2 GB flash memory (16 GB on vision plane) • Bidirectional 1 Watt amplifier for 802.11b communication • Vibration isolating suspension • Wireless analog video transmitter

30. UC Berkeley UAV Platform: Autopilot, HIL Sim Piccolo Avionics Ground Station Actuators and Sensors Signals Hardware In Loop (HIL) Simulation Computer • Low Level Guidance and Control Provided by Cloud Cap Technology’s Piccolo Avionics Module and Corresponding Ground Station System • Capable of operating multiple vehicles. C3UV has successfully flown four vehicles simultaneously. Wireless Link Simulated States Ground Station Computer Manual Control Console

31. UC Berkeley UAV Platform: Communications Ground Station Three Communications Channels: • Autopilot Avionics and Ground station: 900 MHz • Plane to Plane communication: 2.4 GHz • Video downlink: 1.2 GHz

32. Piccolo avionics system Piccolo Piccolo • MHz UHF radio • “Piccolo channel” • Autopilot commands and telemetry • Piccolo Operator Interface • Send waypoint commands • Monitor aircraft physical state Piccolo Ground station

33. User interfaces and Piccolo payload channel Piccolo Piccolo PC104 PC104 • MHz UHF radio • “Piccolo channel” • Autopilot commands and telemetry • “Payload channel” • Communication between • PC104s and GUIs • Mission Commander Interface • Create tasks • Monitor status of tasks and UAVs • Piccolo Operator Interface • Send waypoint commands • Monitor aircraft physical state Piccolo Ground station • Process Monitor Interface • Activate or idle processes • Monitor process states

34. Aircraft to aircraft communication Piccolo Piccolo PC104 PC104 Piccolo PC104 2.4 GHz 802.11b ad-hoc wireless ethernet Mission state estimates Task allocation data

35. Preview of collaboration experiment - Logistics Mission Commander Tent Status Monitor Tent Piccolo Operator Interface Read-only display of Mission Commander Interface Process Monitor Interface Mission Commander Interface Video downlink from aircraft • Aircraft launch process • Aircraft takes off under manual control • Pilot gives control to Piccolo autopilot- aircraft flies on preloaded waypoint loop • Onboard software is activated using Process Monitor Interface ** At this point the aircraft is controlled only via the Mission Commander Interface. All other personnel serve only as monitors.**

36. UC Berkeley UAV Platform: All Together Analog Video TX’s GPS Ant. Wing-Mounted Camera Air-Ground UHF Ant. Onboard computer and wireless communication radio

37. August ONR Demonstration

38. LESSONS LEARNED • Initially Air-Air com was so bad that mission performance was unacceptable. • Improved amplifier/antenna combined with robust architecture allowed us to accomplish complex missions successfully. • Ground-Air link needs to have a higher bandwidth for human/UAV interaction. • Required computation including task allocation and vision processing can be done on a Pentium III. • 3 UAV’s with 4-5 tasks is already too complex for humans without autonomy software. • Complex interaction between task allocation and priority system needs further analysis.

39. The End