ONR UCAV Project Overview

1. Exploration of Hybrid and Intelligent Control Architectures in Conjunction with Probabilistic Verification ONR UCAV Project Overview S. Shankar Sastry July 21, 1998 Electronics Research Laboratory University of California, Berkeley

2. Problem: Design of Intelligent Control Architectures for Distributed Multi-Agent Systems • An architecture design problem for a distributed system begins with specified safety and efficiency objectives for each of the system missions (surveillance, reconnaissance, combat, transport) and aims to characterize control, observation and communication. • Mission decomposition among different agents • Task decomposition for each agent • Inter-agent and agent—mother ship coordination • Continuous control and mode switching logic for each agent • Fault management • This research attempts to develop fundamental techniques, theoretical understanding and software tools for distributed intelligent control architectures with UCAV as an example.

3. Fundamental Issues for Multi-Agent Systems • Central control paradigm breaks down when dealing with distributed multi-agent systems • Complexity of communication, real-time performance • Risk of single point failure • Completely decentralized control • Has the potential to increase safety, reliability and speed of response • But lacks optimality and presents difficulty in mission and task decomposition • Real-world environments • Complex, spatially extended, dynamic, stochastic and largely unknown • We propose a hierarchical perception and control architecture • Fusion of the central control paradigm with autonomous intelligent systems • Hierarchical or modular design to manage complexity • Inter-agent and agent–ship coordination to achieve global performance • Robust, adaptive and fault tolerant hybrid control design and verification • Vision-based control and navigation

4. Autonomous Control of Uninhabited Combat Air Vehicles • UCAV missions • Surveillance, reconnaissance, combat, transport • Problem characteristics • Each UCAV must switch between different modes of operation • Take-off, landing, hover, terrain following, target tracking, etc. • Normal and faulted operation • Individual UCAVs must coordinate with each other and with the mothership • For safe and efficient execution of system-level tasks: surveillance, combat • For fault identification and reconfiguration • Autonomous surveillance, navigation and target tracking requires feedback coupling between hierarchies of observation and control

5. Research Objectives: Design and Evaluation of Intelligent Control Architectures for Multi-agent Systems such as UCAV’s Research Thrusts • Intelligent control architectures for coordinating multi-agent systems • Decentralization for safety, reliability and speed of response • Centralization for optimality • Minimal coordination design • Verification and design tools for intelligent control architectures • Hybrid system synthesis and verification (deterministic and probabilistic) • Perception and action hierarchies for vision-based control and navigation • Hierarchical aggregation, wide-area surveillance, low-level perception Experimental Testbed • Control of multiple coordinated semi-autonomous DV8 helicopters

6. Formal Methods Hybrid systems (continuous and discrete event systems) Modeling Verification Synthesis Probabilistic verification Vision-based control Semi-Formal Methods Architecture design for distributed autonomous multi-agent systems Hybrid simulation Structural and parametric learning Real-time code generation Modularity to manage: Complexity Scalability Expansion Methods Methods

7. Research Thrust 1: Intelligent Control Architectures Thrust 1: Intelligent Control Architectures • Coordinated multi-agent system • Missions for the overall system: surveillance, combat, transportation • Limited centralized control • Individual agents implement individually optimal (linear, nonlinear, robust, adaptive) controllers and coordinate with others to obtain global information, execute global plan for surveillance/combat, and avoid conflicts • Mobile communication and coordination systems • Time-driven for dynamic positioning and stability • Event-driven for maneuverability and agility • Research issues • Intrinsic models • Supervisory control of discrete event systems • Hybrid system formalism

8. Decentralized Observation, Communication and Control for Multi-Agent Systems • Given a strategic objective and local observation • What are the required information protocols with Human-centered system and other autonomous agents to command tactical control? • Given a distributed control problem and the local observation at each site, what is the inter-site communication (minimal) or coordination protocols required to solve this problem? • Given a cooperative mission • What is the strategic objective (possibly dynamic) of each autonomous agent? • How to distribute among the available agents a specified centralized control problem?

9. Decentralized Observation and Communication for Discrete Event Systems Agent Communication Channels A1 A2 A3 Plant (Lp) • The agents have partial observation but can exchange messages. • The plant has a set of unobservable distinguished events (failures). • OBJECTIVE: Design the inter-agent communication scheme required to detect and isolate the distinguished events

10. Synthesis of Inter-agent Communication for Decentralized Observation Theorem 1(Lp,áåpoi, åfiñiÎI) is decentrally diagnosable if there exists n Î Nsuch that for allsfÎåf, usfv ÎLp, |v| £ n, implies (w ÎLp) Ù ( i, Påpoi (w) = Påpoi(usfv)) Þ (sf Î w). If any two sufficiently long plant traces look the same to all the agents, then either they have no failures or have all the same failures. Synthesis: The communicate all plant observationssolution works. General drawback: Redundant information is communicated. L(f) may not be regular even though Lp is regular. Current focus: Minimal communication, protocol synthesis, trace abstraction Documentation: Draft paper available and sent to WODES’98

11. Decentralized Control of Discrete Event SystemsProblem Formulation • Each agent has a set of controllable events • Controllable events are a subset of the set of observable events • The next event is either an uncontrollable event from the plant, a controllable event enabled by an agent, or a message event scheduled by an agent • A control objective is specified by a language • Investigate the existence and synthesis of coordination protocols

12. Communication and Control Synthesis for DES models • Advantages: • Will synthesize symbolic, event-driven, inter-agent communication over a finite message set • Very simple models permitting logical or combinatorial analysis and insights • AHS example: Worked for most coordinating maneuvers other than stability properties for vehicle following • Limitation: No formal way to capture continuous dynamics • The semantics of an event is generally some alignment or safety conditions in velocity, position, and euler angles with respect to targets or other agents • SOLUTION: Distributed control of hybrid systems systems

13. UCAV Control Architecture Intelligent Control Architecture • Mission Planning • Resource Allocation Mission Control Strategic Objective • Generating Trajectory Constraints • Fault Management Strategic Layer Inter-UCAV Coordination Trajectory Constraints • Flight Mode Switching • Trajectory Planning Sensor Info on Targets, UCAV’s Tactical Layer Replan Trajectory • Trajectory Tracking • Set Point Control Environmental Sensors Regulation Layer Actuator Commands Tracking errors UCAV Dynamics

14. Preliminary Control Architecture for Coordinating UCAVs • Regulation Layer (fully autonomous) • Control of UCAV actuators in different modes: stabilization and tracking • Tactical Layer (fully autonomous) • Safe and efficient trajectory generation, mode switching • Strategic Layer (semi-autonomous) • Generating trajectory constraints and influencing the tasks of other agents using UCAV-UCAV and UCAV-ship coordination for efficient • Navigation, surveillance, conflict avoidance • Fault management • Weapons configuration • Mission Control Layer (centralized) • Mission planning, resource allocation, mission optimization, mission emergency response, pilot interface

15. Research Thrust 2: Verification and Design Tools Thrust 2: Verification and Design Tools The conceptual underpinning for intelligent multi-agent systems is the ability to verify sensory-motor hierarchies perform as expected • Difficulties with existing approaches: • Model checking approaches (algorithms) grow rapidly in computational complexity • Deductive approaches are ad-hoc • We are developing hybrid control synthesis approaches that solve the problem of verification by deriving pre-verified hybrid system. • These algorithms are based on game-theory, hence worst-case safety criterion • We are in the process of relaxing them to probabilistic specifications.

16. Hybrid Control Synthesis and Verification Thrust 2: Verification and Design Tools • Approach • The heart of the approach is not to verify that every run of the hybrid system satisfies certain safety or liveness parameters, rather to ensure critical properties are satisfied with a certain safety critical probability • Design Mode Verification (switching laws) • To avoid unstable or unsafe states caused by mode switching (takeoff, hover, land, etc.) • Faulted Mode Verification (detection and handling) • To maintain integrity and safety, and ensure gradual degraded performance • Probabilistic Verification (worst case vs. the mean behavior) • To soften the verification of hybrid systems by rapprochement between Markov and Bayesian decision networks

17. Controller Synthesis for Hybrid Systems • The key problem in the design of multi-modal or multi-agent hybrid control systems is a synthesis procedure. • Our approach to controller synthesis is in the spirit of controller synthesis for automata as well as continuous robust controller synthesis. It is based on the notion of a game theoretic approach to hybrid control design. • Synthesis procedure involves solution of Hamilton Jacobi equations for computation of safe sets. • The systems that we apply the procedure to may be proven to be at best semi-decidable, but approximation procedures apply.

18. Research Thrust 3: Perception and Action Hierarchies Thrust 3: Perception and Action Hierarchies Design a perception and action hierarchy centered around the vision sensor to support surveillance, observation, and control functions • Hierarchical vision for planning at different levels of control hierarchy • Strategic or situational 3D scene description, tactical target recognition, tracking, and assessment, and guiding motor actions • Control around the vision sensor • Visual servoing and tracking, landing on moving platforms

19. What Vision Can Do for Control • Global situation scene description and assessment • Estimating the 3D geometry of the scene, object and target locations, behavior of the objects • Allows looking ahead in planning, anticipation of future events • Provides additional information for multi-agent interaction • Tactical target recognition and tracking • Using model-based recognition to identify targets and objects, estimating the motion of these objects • Allows greater flexibility and accuracy in tactical missions • Provides the focus of attention in situation planning

20. The vision system provides The control architecture needs Situation, 3D scene description Target recognition Continuous control Object tracking Guided motor action Motion detection, and optical flow Relation between Control and Vision • Higher level visual processing: precise, global information, computational intensive • Lower level visual processing: local information, fast, higher ambiguity Higher level Task decomposition for each agent Inter-agent,agent—mother ship coordination Lower level

21. Key Issues in Vision and Control: Deliver the Right Information at the Right Time • How to coordinate the planning stage with sensing stage • The planner should adjust to the speed and uncertainty of the vision system • The vision system should optimize its information flow from the lower level to the higher level, given the need of the planner • How to adjust the focus of attention • Selecting attention of visual processing in terms of the object locations, as well as level of abstraction • Fine tuning lower-level vision-motor control loop • A well-designed lower-level vision-motor control alleviates computation requirements of higher-level visual processing

22. Approach for Hierarchical Vision Processing • Use grouping to extract a compact description of the scene from lower processing • Reduces the computation complexity of higher-level reasoning, provides a basis for attention selection • Information estimated from “big picture” of the scene is less likely to be affected by noise in the sensor • Efficient computation algorithm which is able to capture the “big picture” of a scene has been developed • General results reported in CVPR’97, results on motion reported in ICCV’98

23. Approach for Hierarchical Vision Processing • Applying higher-level reasoning on the groups extracted • Model-based object recognition • Matching image groups to object models • 3D scene geometry estimation • Based on the motion correspondence found • Tracking and behavior analysis of objects • Applying Bayesian theory in selecting the right level of visual processing

24. Approach for Lower-level Vision—Motor Control • Vision-guided motor control • Use low-level image, motion flow information in formulating motor control law • Tracking in the 3D coordinates • Use optical flow equations to build a model of the scene in 3D space • Look-ahead control law to allow for visual processing time • Tracking in the image plane (2D) • Track objects (such as the landing pad) in image frame • Relate image measurement (such as image location of the pad, curvature of the lane marker) to motor control law

25. Research Contributions • Fundamental Research Contributions • Design of hybrid control synthesis and verification tools that can be used for a wide range of real-time embedded systems • Design of vision and control hierarchies for surveillance and navigation • Hierarchical vision for planning at different levels of control hierarchy • Control around the vision sensor • Our multi-agent control architecture can be used for many applications • ONR applications • UCAVs, simulated battlefield environment, distributed command and control, automatic target recognition, decision support aids for human-centered systems, intelligent telemedical system • General engineering applications • Distributed communication systems, distributed power systems, air traffic management systems, intelligent vehicle highway systems, automotive control

26. Research Schedule FY 00 FY 99 FY 98 O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J Intelligent Control Architectures Multi-Agent Decentralized Observation System Performance Evaluation of UCAV Architecture Preliminary UCAV Architecture Final UCAV Architecture Verification Tools Probabilistic Verification Theory Software Tools for Synthesis and Verification Hybrid Control Synthesis Methods Perception Smart Aerobots 3D Simulation Label Recognition C++ QNX Real-Time Visual Situation Assessment Terrain Following Control Scheme Vision System for Autonomous Takeoff/Landing Integrated System for Target Recognition and Terrain Following for Multiple UCAVs Public Tests Cal Day Demo April 17 Cal Day Demo Robotic Helicopter Competition Aug 12-13, Richland, WA UCAV Architecture Demo Robotic Helicopter Competition

27. Deliverables Task Duration Deliverables Intelligent Control Architectures Specification Tools 7/97 - 7/98 software, technical reports Design Tools 7/97 - 9/99 software, technical reports Architecture Evaluation Environment 7/97 - 7/00 software, technical reports UCAV Application 7/97 - 7/00 experiments, technical reports Verification Tools Design Mode Verification 7/97 -12/98 software, technical reports Faulted Mode Verification 7/97 - 9/99 software, technical reports Probabilistic Verification 9/97 - 9/99 technical reports Perception Surveillance 7/97 - 9/99 software, experiments Hierarchical Vision 7/97 - 7/00 software, technical reports Visual Servoing 9/97 - 7/00 experiments, technical reports UCAV Application 7/97 - 7/00 experiments, technical reports

28. Measures of Program Success • FY97-98 • Design of preliminary UCAV architeture • Design of hybrid control synthesis methods • Design of multi-agent decentralized observation system • FY98-99 • Development of probabilistic verification theory • Final UCAV architecture design • Vision and control for terrain following, take-off and landing for single UCAV • FY99-00 • Performance evaluation of UCAV architecture • Integration of vision and control for multiple coordinating UCAVs • Final version of the software tools for • Hybrid control synthesis and verification, and • Decentralized observation and control • Demonstration of UCAV architecture using the helicopter testbed

29. FY97-98 Accomplishments • Controller synthesis for hybrid systems. Developed algorithms and computational procedures for designing verified hybrid controllers optimizing multiple objectives • Multi-agent decentralized observation problem. Designed inter-agent communication scheme to detect and isolate distinguished events in system dynamics • SmartAerobots. 3D virtual environment simulation. Visualization tool for control schemes and vision algorithms—built on top of a simulation based on mathematical models of helicopter dynamics • Label recognition: prototype in Matlab, then in C++ (QNX real-time)

30. Berkeley Team Name Role Tel E-mail Shankar Sastry Principal (510) 642-7200 sastry@robotics.eecs.berkeley.edu Investigator (510) 642-1857 (510) 643-2584 Jitendra Malik Co-Principal (510) 642-7597 malik@cs.berkeley.edu Investigator Datta Godbole Research (510) 643-5806 godbole@robotics.eecs.berkeley.edu Engineer (510) 231-9582 John Lygeros Postdoc (510) 643-5795 lygeros@robotics.eecs.berkeley.edu Jianbao Shi Postdoc (510) 642-9940 jshi@cs.berkeley.edu Omid Shakernia Graduate Student (510) 643-2383 omids@robotics.eecs.berkeley.edu

31. Teaming and Interdependency • Collaboration with Prof. Varaiya (Berekeley) in designing a hierarchical control architecture for coordinating UCAVs • Collaboration with Prof. Russell (Berkeley) in developing probabilistic design and analysis tools • Collaboration with Prof. Zadeh (Berkeley) on soft computing tools for control of UCAVs and mode transition methods for DV 8 developed using fuzzy control • Collaboration with Prof. Speyer (UCLA) on fault detection and handling methods • Collaboration with Prof. Morse (Yale) on vision-guided navigation • Informal conversations with Prof. Anderson (ANU), Prof. Hyland (Michigan) and visit to Naval Post Graduate School • Pending: more formal collaborations with Profs. Narendra, Morse