The Theory of Fast and Robust Adaptation

1. THE THEORY OF FAST AND ROBUST ADAPTATION Naira Hovakimyan Department of Aerospace and Ocean EngineeringVirginia Polytechnic Institute and State University This talk was originally given as a plenary at SIAM Conference on Control and Its Applications San Francisco, CA, June 2007 A. M. Lyapunov 1857-1918 G. Zames 1934-1997

2. Outline • Historical Overview • Limitations and the V&V Challenge of Existing Approaches • Paradigm Shift • Speed of Adaptation • Performance and Robustness • Rohrs’ Example • Various aerospace applications • Future Directions of Research

3. Motivation • Early 1950s – design of autopilots operating at a wide range of altitudes and speeds • Fixed gain controller did not suffice for all conditions • Gain scheduling for various conditions • Several schemes for self-adjustment of controller parameters • Sensitivity rule, MIT rule • 1958, R. Kalman, self-tuning controller • Optimal LQR with explicit identification of parameters • 1950-1960 - flight tests X-15 (NASA, USAF, US Navy) • bridge the gap between manned flight in the atmosphere and space flight • Mach 4 - 6, at altitudes above 30,500 meters (100,000 feet) • 199 flights beginning June 8, 1959 and ending October 24, 1968 • Nov. 15, 1967, X-15A-3

4. First Flight Test in 1967 The Crash of the X-15A-3 (November 15, 1967) X-15A-3 on its B52 mothership X-15A-3 Crash due to stable, albeit non-robust adaptive controller! Crash site of X-15A-3

5. Robustness (measure of relative stability) • Performance (comparison with desired system behavior) Objectives of Feedback • Stabilization and/or Tracking

6. Historical Background • Sensitivity Method, MIT Rule, Limited Stability Analysis (1960s) • Whitaker, Kalman, Parks, et al. • Lyapunov based, Passivity based (1970s) • Morse, Narendra, Landau, et al. • Global Stability proofs (1970-1980s) • Morse, Narendra, Landau, Goodwin, Keisselmeier, Anderson, Astrom, et al. • Robustness issues, instability (early 1980s) • Egard, Ioannou, Rohrs, Athans, Anderson, Sastry, et al. • Robust Adaptive Control (1980s) • Ioannou, Praly, Tsakalis, Sun, Tao, Datta, Middleton,et al. • Nonlinear Adaptive Control (1990s) • Adaptive Backstepping, Neuro, Fuzzy Adaptive Control • Krstic, Kanelakopoulos, Kokotovic, Zhang, Ioannou, Narendra,Ioannou, Lewis, Polycarpou, Kosmatopoulos, Xu, Wang, Christodoulou, Rovithakis, et al. • Search methods, multiple models, switching techniques (1990s) • Martenson, Miller, Barmish, Morse, Narendra, Anderson, Safonov, Hespanha, et al.

7. Landmark Achievement: Adaptive Control in Transition • Air Force programs: RESTORE (X-36 unstable tailless aircraft 1997), JDAM (late 1990s, early 2000s) • Demonstrated that there is no need for wind tunnel testing for determination of aerodynamic coefficients • (an estimate for the wind tunnel tests is 8-10mln dollars at Boeing) Lessons Learned: limited to slowly-varying uncertainties, lack of transient characterization • Fast adaptation leads to high-frequency oscillations in control signal, reduces the tolerance to time-delay in input/output channels • Determination of the “best rate of adaptation” heavily relies on “expensive” Monte-Carlo runs Boeing question: How fast to adapt to be robust?

8. Stability, Performance and Robustness • Absolute Stability versus Relative Stability • Linear Systems Theory • Absolute stability is deduced from eigenvalues • Relative stability is analyzed via Nyquist criteria: gain and phase margins • Performance of input/output signals analyzed simultaneously • Nonlinear Systems Theory • Lack of availability of equivalent tools • Absolute stability analyzed via Lyapunov’s direct method • Relative stability resolved in Monte-Carlo type analysis • No correlation between the time-histories of input/output signals

9. The Theory of Fast and Robust Adaptation • Main features • guaranteed fast adaptation • guaranteed transient performance • for system’s both signals: input and output • guaranteed time-delay margin • uniform scaled transient response dependent on changes in • initial condition • value of the unknown parameter • reference input • Suitable for development of theoretically justified Verification & Validation tools for feedback systems

10. Implementable nom. cont. Small-gain theorem gives sufficient condition for boundedness: Result: fast and robust adaptation via continuous feedback! One Slide Explanation of the New Paradigm System: Nominal controller in MRAC: Desired reference system: This is the nominal controller in the L1 adaptive control paradigm!

11. Hurwitz, unknown System dynamics Ideal Controller Adaptive Controller Adaptive law: Bounded Barbalat’s lemma State predictor Same error dynamics independent of control choice Closed-loop state predictor with u Two Equivalent Architectures of Adaptive Control

12. Enables insertion of a low-pass filter Low-pass filter: C(s) Implementation Diagrams Reference system MRAC u cannot be low-pass filtered directly. State predictor based reparameterization No more reference system upon filtering!!!

13. Low-pass filter: C(s) bounded • Sufficient condition for stability via small-gain theorem Barbalat’s lemma Stability • Closed-loop • Solving:

14. Filtered ideal controller: • Closed-loop system with filtered ideal controller = non-adaptive predictor: Stability via small-gain theorem Plant with unknown parameter Reference system of MRAC Reference system Closed-loop Reference System

15. (MRAC) Guaranteed Transient Performance • System state: • Control signal:

16. Virtual Plant with clean output Plant with unknown parameter LTI System for Control Specifications Design system for defining the control specs Reference system achieved via fast adaptation Independent of the unknown parameter

17. Achieving Desired Specifications • Sufficient condition for stability • Performance improvement

18. Large adaptive gain Smaller step-size Faster CPU Guaranteed Performance Bounds • Use large adaptive gain • Design C(s) to render sufficiently small Desired design objective

19. Time-delay Margin and Gain Margin  Point Time-delay occurs Plant Lower bound for the time-delay margin Projection domain defines the gain margin

20. Main Theorem of L1 Adaptive Controller Fast adaptation, in the presence of , leads to guaranteed transient performance and guaranteed gain and time-delay margins: where is the time-delay margin of .

21. Tracking vs robustness can be analyzed analytically. The performance can be predicted a priori. Design Philosophy Adaptive gain – as large as CPU permits (fast adaptation) • Fast adaptation ensures arbitrarily close tracking of the virtual reference system with bounded away from zero time-delay margin Fast adaptation leads to improvedperformance and improvedrobustness. Low-pass filter: • Defines the trade-off between tracking and robustness Increase the bandwidth of the filter: • The virtual reference system can approximate arbitrarily closely the idealdesiredreference system • Leads to reduced time-delay margin

22. Robot Arm with Time-varying Friction and Disturbance Control design parameters Parametric uncertainties in state-space form: Compact set for Projection Disturbances in three cases:

23. Simulation Results without any Retuning of the Controller System output Control signal

24. With time-delay 0.1 Verification of the Time-Delay Margin With time-delay 0.02

25. Application of nonlinear L1 theory MRAC and L1 for a PI controller MRAC L1 The open-loop transfer functions in the presence of time-delay Time-delay margin Time-delay margin

26. Miniature UAV in Flight: L1 filter in autopilot The Magicc II UAV is flying in 25m/h wind, which is ca. 50% of its maximum flight speed Courtesy of Randy Beard, BYU, UTAH

27. Rohrs’ Example: Unmodeled dynamics System with unmodeled dynamics: Nominal values of plant parameters: plant dynamics Unmodeled dynamics Reference system dynamics: Control signal: Adaptive laws from Rohrs’ simulations:

28. Instability/Bursting due to Unmodeled Dynamics Instability Parameter drift System output Bursting Parameters System output

29. Rohrs’ Explanation for Instability: Open-Loop Transfer Function -135 -180 At the frequency 16.1rad the phase reaches -180degrees in the presence of unmodeled dynamics, reverses the sign of high-frequency gain, the loop gain grows to infinity instability At the frequency 8rad the phase reaches -139 degrees in the presence of unmodeled dynamics, no sign reversal for the high-frequency gain the loop gain remains bounded bursting

30. Rohrs’ Example with Projection for Destabilizing Reference Input Improved knowledge of uncertainty reduces the amplitude of oscillations

31. Predictor model System Adaptive Law Control Law Rohrs’ Example with L1 Adaptive Controller Design elements Adaptive laws:

32. No Instability/Bursting with L1 Adaptive Controller

33. Pref(s) + r(t) y(s) + u - + Pum(s) P(s) + + Ppr (s) + y(s) r(t) u + kg - + + Pum(s) P(s) L1 Difference in Open-Loop TFs MRAC MRAC cannot alter the phase in the feedforward loop Adaptation on feedforward gain Continuous adaptation on phase

34. Classical Control Perspective L1 MRAC Adaptation simultaneously on both loop gain and phase Adaptation only on loop gain No adaptation on phase

35. Stabilization of Cascaded Systems with Application to a UAV Path Following Problem UAV with Autopilot Path Following Kinematics Cascaded system (in collaboration withIsaac Kaminer, NPS)

36. Path following kinematics Path following kinematics Autopilot UAV Very poor performance Exponentially stable Path following controller Path following controller Path following kinematics Autopilot UAV Output feedback L1 augmentation L1 adaptive controller Path following controller Augmentation of an Existing Autopilot by L1 Controller

37. Flight Test Results of NPS (TNT, Camp Roberts, CA, May 2007) L1Adaptive Controller Augmented L1Adaptive Controller Outer-loop path-planning, navigation With adaptation, errors reduced to 5m Autopilot UAV Inner-loop Guidance and Control Without adaptation Errors  100m

38. Architecture for Coordinated Path Following • Path following: follow predefined trajectories using virtual path length • Coordination: coordinate UAV positions along respective paths using velocity to guarantee appropriate time of arrival Guaranteed stability of the complete system: use L1to maintain individual regions of attraction L adaptation 1 Path Onboard A/P Path following desired Pitch rate Generation + UAV (Outer loop) path Yaw rate (Inner loop) commands Network Normalized Velocity Coordination info Path lengths command L adaptation 1

39. Time-critical Coordination of UAVs Subject to Spatial Constraints: Hardware-in-the-Loop Flight Test Results Performance comparison with and without adaptation for one vehicle Simultaneous arrival of two vehicles with L1 enabled Mission scenarios:Sequential autolanding, ground target suppression, etc. • Arrival time is the same, but is not specified a priori Assumptions: • Vehicles dynamically decoupled • Underlying communication network topology is fixed in time (currently relaxed) • Each vehicle communicates only with its neighbors

40. Flight Control Design (Models provided by the Boeing Co.) Tailless Unstable Aircraft X-45A Autonomous Aerial Refueling • Uncertain dynamic environment • Receiver dynamics due to aerodynamic influences from tanker • Drogue dynamics due to aerodynamic influences from tanker and receiver Elevon/yaw vectoring control mixing • Compensation for pitch break uncertainty and actuator failures • Finite time to contact the drogue in highly uncertain dynamic environment.

41. MRAC X-45A: Transient Performance vs Robustness • Transient tracking not too sensitive to changes in adaptive gain beyond certain lower bound • Filter impacts the transient tracking • The filter can be selected to improve the margin at the price of transient tracking TDM=0.045 Vertical acceleration TDM=0.1 • Transient tracking compared for MRAC and L1 with two different filters • The performance of C1(s) (with better margins) still better than for MRAC

42. Aerial Refueling • Control objective: • Finite time to contact the drogue in highly uncertain dynamic environment. • Solution/Approach: • An adaptive method withoutretuning adaptationgains • for highly uncertain dynamic environment • with guaranteed transient performance • for finite time-to-contact • Barron Associates Tailless Aircraft Model • 6 DOF Flying-wing UAV, Statically unstable at low angle of attack • Aerodynamic data primarily taken from [1] • Vortex effect data from wind tunnel test [2] – ICE model • Tanker: KC-135R, Receiver: Tailless 65 degree delta wing UAV • Aerodynamic coefficients change as a function of relative separation • Varying more significantly with lateral and vertical separation [1] S.P Fears et. al., Low-speed wind-tunnel Investigation of the stability and control characteristics of a series of flying wings with sweep angles of 50 degree [2] W.B. Blake et. al., UAV Aerial Refueling – Wind Tunnel Results and Comparison with Analytical Predictions

43. Scaled Response - Starting from different initial positions - Scaled response No retuning! - Twice the vortex – model of a different tanker - Uniform transient and steady state response - Scaled control efforts thrust channel elevator channel aileron channel

44. Air-Breathing Hypersonic Vehicle (Boman model WP AFRL) • Control Challenge of Air-Breathing Hypersonic Vehicles • Vibration due to high speed and structural flexibility • Uncertainties caused by rapid change of the operating conditions • Integration of airframe dynamics and propulsion system • Coupling of attitude and velocity Desired path flight angle desired speed Control signal: Elevator (deg) Velocity profile System response Control Signal: Equivalent ratio

45. Current Status and Open Problems • Adaptive Output Feedback Theory • Partial results in this direction • Numerical discretization issues due to fast adaptation • Integration of stiff systems • Adaptive Observer Design • Control of distributed autonomous systems over ad-hoc communication network topologies • Performance evaluation on the interface of different disciplines • More work on its way towards final answers!!!

46. Conclusions • Fast adaptation no more trial-error based gain tuning! • performance can be predicted a priori • robustness/stability margins can be quantified analytically • Theoretically justified Verification & Validation tools for feedback systems at reduced costs With very short proofs! • What do we need to know? • Boundaries of uncertainties sets the filter bandwidth • CPU (hardware) sets the adaptive gain

47. Acknowledgments My group: • Chengyu Cao (theoretical developments) • Jiang Wang (aerial refueling) • Vijay Patel (UCAV, flight test support) • Lili Ma (vision-based guidance) • Yu Lei (hypersonic vehicle) • Dapeng Li (theoretical developments, vision-based guidance) • Amanda Young (cooperative path following) • Enric Xargay (cooperative path following) • Zhiyuan Li (discretization issues for flexible aircraft) • Aditya Paranjape (differential game theoretic approaches for VBG) Collaborations: • The Boeing Co. (E. Lavretsky, K. Wise, UCAV model, aerial refueling) • Wright Patterson AFRL, CerTA FCS program (ICE vortex) • Raytheon Co. (R. Hindman, B. Ridgely) • NASA LaRC (I. Gregory, SensorCraft) • Wright Patterson AFRL (D. Doman, M. Bolender, hypersonic model) • Randy Beard (BYU) • Isaac Kaminer (NPS) Sponsors: AFOSR, AFRL, ARO, ONR, Boeing, NASA