Software-Enabled Control HWIL and Flight Tests*

1. Software-Enabled Control HWIL and Flight Tests* Gary Balas Aerospace Engineering and Mechanics University of Minnesota balas@aem.umn.edu www.aem.umn.edu 2 March 2005 * The theoretical development of the algorithms was funded by NASA Langley under the NASA Cooperative Agreement NCC-1-03028. Dr. Celeste M. Belcastro is the technical contract monitor. The application to the DARPA SEC fixed-wing capstone demonstration was funded as part of the Software Enabled Control Program, Dr. John Bay Program Manager. The contract number is USAF/AFMC F33615-99-C-1497, Dale Van Cleave was the Technical Contract monitor.

2. Software Enabled Control (SEC) The objective of SEC is to co-develop advanced real-time control system algorithms and the software services and infrastructure necessary to implement them on distributed embedded processors in a robust and verifiable way Dr. John Bay DARPA Information Technology Office

3. Distributed Monitoring, Modeling, and Control SEC C(s) Software-Enabled Control (SEC) • Program Objectives: • Control Systems for innovative vehicles • UAVs, rotorcraft, fighters • Increase automation for extreme maneuvers • Assured stability for flight mode transition • Improve disturbance rejection and fault tolerance • Automatic control reconfiguration • Redundancy management • Provide reusable middleware for coordinated, embedded software control on multiple aircraft types • Modernize flight control with adaptive, distributed computing Maneuverability Human pilots Vehicle capability Traditional control methods Speed SEC provides control systems for innovative vehicles that exceed the capability of conventional flight controllers On-board control and fault management • Multiple levels of control: • Vehicle management (including flight-critical systems) • Mission management (including route planning) • Multi-vehicle (e.g. automatic formation flight) Moller Proprietary

4. S/W Components OCP Middleware Operating system API API API Board support package VX-Works Hardware (CPU, memory, I/O) POSIX BSP DY4, Power PC Examples SEC Technologies • Active State Models: Prediction & Assessment • Dynamically exploit on-line system and environmental data to improve reference models • Predict effects over very large state and mode spaces • Rapidly assess damage • On-Line Control Customization: Adaptation • Enable precise mode transition • Support control re-parameterization and reconfiguration during operation due to environmental disturbances, interference, and damage • Accommodate dynamic coordination requirements • Coordinated Multi-Modal Control • Achieve global stability, maximize system and mission performance • Provide joint fault detection, isolation, and recovery • Enable distributed control implementation for physically separated components • Open Control Platform (OCP): Software • Provide reusable control middleware and tool support for building controllers from SEC technologies • Provide parametric open systems framework necessary to support SEC active state model, hybrid/coordinated, and adaptive multi-modal control • Provide flexible experimental platform for SEC control research and demonstration • High Confidence Software Control Systems • Assure safety and reliability under fault conditions • Design for verification, validation, and certifiability Gain altitude using main rotor collective Control reconfiguration for autorotation Tail rotor failure Safe Landing

5. Control Application Software Components OCP Middleware API Operating system API API Board support package VX-Works Hardware (CPU, memory, I/O) POSIX BSP DY4, PowerPC (EXAMPLES) Open Control Platform (OCP) The Open Control Platform (OCP) is a program task as well as the vehicle for contractor integration into demonstrations and experiments in flight control systems API Examples: Optimal Control multi-criteria receding horizon distributed Hybrid Control switching services blending planning Distributed Processing distributed systems multi-vehicle Fault Management detection and isolation reconfiguration error recovery Active Models model servers estimators multi-model management

6. Real-Time Services Bold Stroke SEC Tasks and Milestones Multi-Modal Control Active State Models On-Line Control Customization High-Confidence Systems Boeing SRI Honeywell MIT Cal Tech Rockwell Northrop Grumman Honeywell U Washington Vanderbilt OGI U Minnesota Northrop Grumman Draper Labs Vanderbilt Cal Tech U Minnesota UC Berkeley U Minnesota Northrop Grumman Stanford Ga Tech TRANSITION Legacy Technology Prediction Switching Adaptation Confidence OPEN CONTROL PLATFORM Boeing, GaTech, UC Berkeley, Honeywell Milestones API for switching svcs. Predictive models oper. Hybrid multi-model svcs Integrated model svcs. Mode triggering defs. CLF and LPV control Hybrid stability, single sys. Customization svcs. Hybrid run-time svcs High-level multi-mode API Multi-mode run-time svcs. Multi-vehicle hyb. control Hybrid model checking Formal specification lang. Integrated fault mgt. Sensor/act reconfig.

7. UMN/UCB SEC team members • University of Minnesota • Gary Balas, Raktim Bhattacharya, Tamas Keviczky, Praveen Vijayaraghavan, Katherine Zhou, Deenadayalan Narayanaswamy, Nachiket Bapat, Yohannes Ketema Ph.D., George Papageorgiou Ph.D., Richard Russell, Aron Cooper, Capt. Paul Blue, Francesco Borrelli Ph.D, Matt Payne, Ryan Ingvalson, Riccardo Oreste Natale Ph.D., Prof. Hector Rotstein, David Weyrauch • University of California, Berkeley • Andy Packard, Alpay Kaya, Zachary Jarvis-Wloszek, Sean Estill, Ken Hsu, Kunpeng Sun

8. SEC Technical issues addressed by UMN/UCB Increase autonomy, reliability and aggressiveness of UAVs for a range of potential missions through advances in control and trajectory generation. Objective: Development of software based control technologies that use dynamic information about the controlled system to adapt, in real-time, to changes in the systems and its operating environment. This will enable UAVs capable of • extreme performance • accommodation to changes in mission/tactical goals, environmental conditions, failures and constraints • use of diverse information from on and off-vehicle sensors

9. Technical Approach Specific technologies developed: • Receding horizon control (RHC) • 3D time-stamped position reference trajectory tracking. • Respecting aircraft constraints. • Integrated and independent trajectory generation and inner-loop control • Anytime control algorithms • Fault detection algorithms • Seamless integration with OCP • Implementation of control algorithms using RT-ARM • RHC API to interface with the OCP • Interface with online optimization solver within the OCP.

10. Receding Horizon Control Algorithm

11. Receding Horizon Autopilot Scheme • Exploits a priori reference information, accomplishes higher level control objectives. • Acts as a system/mission-state dependent variable inner-loop command pre-filter. • Ensures that the aircraft’s inputs are held within saturation limits even in the presence of wind gusts. • Respects flight envelope constraints and system-state dependent maneuver limits.

12. Receding Horizon Controller (autopilot) • Relies on an explicit discrete-time process model for prediction. • Limited information and insight about actual inner control loop by means of linearized models at certain flight conditions. Discretized linear inner closed-loop Disturbance model augmentation • Outputs: • Commands: – Tracking performance (y): – thrust altitude, speed – pitch rate – Maneuvering limits (z): vertical acceleration – Actuators (u): thrust and thrust rate, elevator defl. and rate

13. Receding Horizon Control

14. Linear and adaptive RHC schemesbased on [Maciejowski, 2001] • Uses a discrete-time LTI or linear, parameter-varying (LPV) internal model that depends on the actual flight condition. • Prediction model is LTI over prediction horizon, but changes every time a new control input is calculated. • Internal model input, input rate, output and state constraints can be imposed. • Unconstrained problems, analytic solution (least-squares). • Constrained problems lead to QP problems, efficient solvers. • Desired maneuver limits are imposed as soft constraints to alleviate infeasibility problems.

15. Receding Horizon Control: Problem Formulation

16. RHC problem formulation and cost function Optimization problem solved at each sampling time: subject to • Flight condition dependent inner closed-loop dynamics • (augmented with estimated disturbance state) • Flight envelope constraints • Maneuvering limits • Actuator constraints

17. RHC problem formulation and cost function • Using a linear internal model, the problem becomes a Quadratic Program: with the appropriate matrices that depend on the linear prediction model, the measured states and signal limits. • The  slack variable represents an -norm penalty (soft constraint) on flight envelope and maneuvering constraint violations. • This type of soft constraint is an “exact penalty function”, i.e. its value is nonzero only if the original problem with hard constraints is infeasible.

18. RHC Summary • Real-time implementation is feasible. • On-line constraint modification allows straightforward incorporation of system-state dependent, and time-varying constraints. • Linear prediction models should be flight condition dependent, either in the form of a selection of LTI models or an LPV system description. • Using LPV models for prediction, the modest complexity of the predictive control problem can still be retained (QP) with improved accuracy and extended operation limits, which are comparable even to the nonlinear solution.

19. Receding Horizon Control API

20. Every Major Frame • PRE: User-code uses “set()” methods to map input port data (model parameters, modes, faults and state estimates) into problem data (dynamics, costs, constraints). • RHCAPI retrieves problem data, formulates constrained quadratic program. • OPT (anytime task): RHCAPI solves the constrained Quadratic Program. If infeasible, relax and solve again. • POST: User code interprets optimal solution and problem data to issue commands

21. Model Data: Cost Data: Horizon Length: Signal Estimate: Initial State Estimate: Input Subspace: Constraint Data: Constraint Relaxation: RHC API Component: Problem Formulation At each time step, solve with subject to:

22. Relaxing the constraints Each constraint is associated with a relaxation group (G0, G1, …) Constraints in G0 cannot be relaxed The unit cost of relaxing any/all constraints in G1 is The unit cost of relaxing any/all constraints in G2 is and so on… Each individual constraint is linear in the decision variable . Express the constraint as . Let denote the original cost. The relaxed problem is:

23. RHC-API

24. RHC-API Timing • PRE Hard Real-Time (HRT) task starts at beginning of a major frame (2Hz). • Starts a timer, after 450 ms calls POST, other HRT task. • When PRE finishes, flag set enabling anytime task OPT to call optimizer. • If optimization converges, flag is set to indicate optimizer no longer running. • When POST gets called, it checks the optimizer running flag. • If optimizer is no longer running, processes optimization results and sends control commands. • If optimizer is still running, implements baseline solution control commands.

25. Fault Detection

26. Fault Detection (FD) Goals • Design an FD filter such that: • Input to filter: plant input/outputpair u-y. • Output from filter: “residual” signal r. • r is independent from the input u. • r is insensitive to plant noise and model mismatch (MM). • r is sensitive to the faults in the plant. • Channel of interest for FD: u y r u r Nominal Plant FD Filter Faulty Plant FD Filter

27. Faulty plant Fault Block Wf DemoSim/ Aircraft Fault Detection Challenges Challenge1: Black Box Simulation, Demosim. • No Access to internal signals. • Closed-loop system dynamics - i.e. T-33/UCAV autopilot Solution: Generate own fault implementation. • Inject fault as multiplicative input error. • Identified by simulating the response of an aircraft using a faulted aileron actuator (fault block Wf )

28. Wf Fault weight Wp MM weight Wn Noise weight P0 Nominal plant model Wf Wp Wn P0    Fault Detection Challenges Fault Detection Challenges Challenge 2: DemoSim complexity • Unknown internal nonlinearities (saturations, rate limits, quantization levels, etc). • Linear models fail to capture these dynamics. • Model mismatch looks like a fault. Solution: Incorporate model mismatch into design. • Model plant mismatch as input multiplicative uncertainty. • Optimally design a filter to account for this uncertainty. Model mismatch noise

29. Model Mismatch For some input signals, model mismatch dominates fault!

30. FD Simulation Results: Pulse Example Challenge: distinguish model mismatch from fault Solution: input-dependent threshold signal

31. Threshold Function s • s < 0: no fault • s > 0: fault • Two tuning parameters: • Add “forgetting factor” to integral, removes effect of Model Mismatch (MM). • Scale noise term Pulse example revisited Fault on Fault detected MM

32. UMN/UCB SEC Final Exam

33. Final Exam Airplanes

34. Our Customer DARPA AFRL IFSC and VACC Government Collaborators Air Force Flight Test Center at Edwards Air Force Base NASA Dryden Flight Research Center Boeing Teams Phantom Works Network Centric Operations J-UCAS F-15E1 SEC Controls Technology Teams University of California-Berkeley California Institute of Technology University of Colorado Honeywell Massachusetts Institute of Technology University of Minnesota Northrop Grumman Stanford University The Demonstration Team

35. SEC Ground Station andFlight Demo Visitor RV

37. UMN/UCB Final Exam Sequence of Events • Piloted approach to Ingress Point with specified flight condition • Start recording data (REC-ON) • Enable commands (“Fly to initial condition”) (CMD-ON) • Engage RHC controller (START) • Track reference trajectory (scan test range for targets) • Invoke Pop-up threat if desired (SND OBST) • Avoid pop-up threat and fly over target • Initiate fault simulator (FTSM_ON) • Detect fault and reconfigure controller • Finish mission at Egress Point • Disengage controller (CMD-OFF) • Stop recording data (REC-OFF)

38. Flight envelope of demonstration experiment

39. Flight test range atNASA Dryden Flight Research Center

40. Experiment Plan

41. Results • Matlab/Simulink/DemoSim Simulations • Hardware-in-the-loop Simulations • Flight Tests

42. High-fidelity simulations (Matlab/DemoSim) Comparison of results in different wind conditions

43. Comparison of results in different wind conditions

44. Comparison of results in different wind conditions True airspeed limits were saturated with -60 ft/s (~18 ms/) East wind

45. Comparison of results in different wind conditions True airspeed limits were saturated with -60 ft/s (~18 ms/)East wind

46. Comparison of results in different wind conditions True airspeed limits were saturated with -60 ft/s (~18 ms/)East wind

47. Comparison of results in different wind conditions

48. Hardware-in-the-loop simulations

49. Hardware-in-the-loop simulations