Myopic Unfalsified Control: A Gradient Approach to Adaptation

1. Myopic Unfalsified Control: A Gradient Approach to Adaptation Michael G. Safonov University of Southern California

2. Background: Unfalsified Control ONLY 3 elements candidate controllerhypotheses goals data COMPUTER SIEVE Unfalsified K FALSIFIED M. G. Safonov. In Control Using Logic-Based Switching, Spring-Verlag, 1996.

3. Motivation for Unfalsification: An Achilles heel of modern system theory has been the habit of ‘proof by assumption’ Theorists typically give insufficient attention to the possibility of future observations which may be at odds with assumptions.

4. Reality is what we observe. Galileo: open-eyed “data-driven” Two ways to learn & adapt: Observation vs. Introspection Reality is an ideal, observable only through noisy sensors. vs. Plato: introspective“assumption driven” MODELS approximate Observed Data Data approximates Unobserved TRUTH ‘Curve-Fitting’ ‘Probabilistic Estimation’

5. “Unfalsified: Data Driven” goals are modest: no guaranteed predictions of future stability just consistency of goals, decisions and data no troublesome assumptions, parsimonious formulation: DATA GOALS DECISIONS Fast, reliable, exact “Traditional: Assumption Driven” goals are ambitious: guaranteed future stability Cost & estimation convergence many troublesome assumptions “the ‘true’ plant is in model set”, “noise I.I.D.” “bounds on parameters, probabilities” …, “linear time-invariant, minimum-phase plant, order < N” Slow, quasi-static, approximate Observation vs. Introspectionin adaptive control Remarkably, some leading “assumption driven” control theorists have held that observed DATA inconsistent with ASSUMPTIONS should be ignored (cf. M. Gevers et al., "Model Validation in Closed-Loop", ACC, San Diego, 1999)

6. CONTROLLER SELECTOR DECISIONS given candidate controllers K DATA GOALS LEARNING FEEDBACK LOOPS COMPUTER SIEVE Unfalsified Controllers K FALSIFIED M. G. Safonov. In Control Using Logic-Based Switching, Spring-Verlag, 1996. Unfalsified Adaptive Control

7. A candidate controller K is FALSIFIED when the data (y,u) proves existence of a ‘fictitious’ that would violate performance spec’s if the K were in the loop. Controllers may remain UNFALSIFIED until the data proves otherwise. Or, fading memory may be used to reinstate previously falsified controllers u y r e y + Unknown Plant K S - Unfalsified Falsified gain Controllers Controllers Unfalsification Key points: ● A controller need not be in the loop to be falsified. ● Even a single data point can falsify many controllers.

8. + e u y r Unknown Plant K S - gain Trivial Example • Plant Data: at time t=0, (u,y)=(1,1) • Candidate K’s: u=Ke real gain • Goal: |e(t)/r(t)| < 0.1 for all r(t) Relations: e=u/K=1/K , r=y+e=y+u/K= 1+1/K => K is unfalsified if |1/(1+K)| < 0.1 • => unfalsified K’s: K>9 or K<-11

9. control input u(t) plant output y(t) proportional gain kP (t) integral gain kI (t) derivative gain kD (t), Jun & Safonov, CCA/CACSD ‘99 Simulation Example: ACC Benchmark time-history of unfalsified candidate control gains time-history of plant response and optimal unfalsified control gain Tsao & Safonov, IEEE Trans, AC-42, 1997. Evolution of unfalsified set Time responses

10. Specified target response bound Actual response Commanded response Brugarolas, Fromion & Safonov, ACC ’98 Brugarolas, Fromion and Safonov, ACC98 Simulation Example: Missile • Learns control gains • Adapts quickly to compensate for damage & failures • Superior performance • Unfalsified adaptive missile autopilot: • discovers stabilizing control gains as it flies, nearly instantaneously • maintains precise sure-footed control

11. Simulation Example: Unfalsified ‘PID Universal Controller’ • Unfalsified adaptive control loop stabilizes in real-time • Unstable Plant 30 Candidate PID Controllers: • KI =[2, 50, 100] • KD = [.5, .6] • KP = [5, 10, 25, 80, 110] Example: Adaptive PID Goal: ————— Jun & Safonov, CCA/CACSD ‘99

12. Other People’s Successes with Unfalsified Control • Emmanuel Collins et al. (Weigh Belt Feeder adaptive PID tuning, CDC99) • Kosut (Semiconductor Mfg. Process run-to-run tuning, CDC98) • Woodley, How & Kosut (ECP Torsional disk control, adaptive tuning, ACC99 • Razavi & Kurfess, Int. J ACSP, Aug. 2001

13. How Unfalsified Works ftp://routh.usc.edu/pub/safonov/PID_Demo_for_MATLAB6.zip

14. Inside the Unfalsification Algorithm Block Bank of Filters (one for each candidate K) Controllers Out Data In

15. Why doesn’t everybody use unfalsified control? • Unfalsified Adaptive Control is better: • fast • reliable • exact • Full and precise use of all available evidence • meets Russell’s criterion: “Intimate association between observation and hypothesis” • But, it’s not yet widely used. Why?

16. Unfalsified Adaptive Control:Some Invalid Concerns • Strange noise-free, model-free formulation • You can’t prove convergence or stability (unless you assert the usual adaptive control assumptions) • Claims seem to good to be true: • Fast, reliable • Handles noise without noise models • Works for non-minimum phase plants • No need for plant degree bounds • No need for plant models

17. Unfalsified Adaptive Control:A Valid Concern • Unfalsified Adaptive Control can be computationally intensive: • Computationally intensive when there are many candidate controllers • Requires global minimization of unfalsified cost levels • May require ‘gridding’ (like multi-model adaptive)

18. Solutions: How to address concerns over unfalsified? • Direct comparision with traditional methods? • Make unfalsified more like traditional methods? • make it slow, quasi-static (break its legs): • differential equation update, or • run-to-run controller update • make it myopic, near-sighted (break its spectacles): • do local optimization only, not global • use steepest descent, gradient-based approaches to limit computational complexity

19. Solution #1: Direct Comparison Traditional & Alternative Unfalsified • Traditional assumptions: • LTI minimum phase plant No • Known upper bound for plant order No • Known relative degree of the plant No • Known sign of the high frequency gain No • Alternative ‘universal controllers’ (Martensson, 1985; Fu et al., 1986): • Require less prior information YES • Search a set of controllers and switch to the one satisfying a given performanceYES • Assume the plant to be LTI No

20. ym + Error e(t) REFERENCE MODEL Wm(s) Unknown Plant + - r up yp - Controller K( ) yp EXAMPLE: Direct Comparision Narendra MRAC vs. Unfalsified MRAC A. Paul and M. Safonov, 2002 Candidate Controllers K( ) : where q1=[-2;-1;0;2;4] ; q2=[2;5.5;6;6.5;7;8] and q3 =[-10;-6;-5;-4;-3;2] Cost Function J(,t ): Unfalsified if J(,t) < threshold = 200 J(,t) = α e( ,t)2+ β s0t exp{-σ (t- τ)} (e ( , τ)2) d τ < 200 ; weights ==1 where e( ,t) = fict( ,t) * Wm(t) - yp(t) σ=.001 is ‘fading-memory forgetting-factor (sigma-modification) ’ is the ‘fictitious reference signal’

21. Example: SimuLink Simulation of Unfalsified ControlNarendra MRAC vs. Unfalsified Adaptive A. Paul and M. Safonov, 2002

22. Simulation Results: Narendra MRACvs. UNFALSIFIED A. Paul and M. Safonov, 2002 Simulation Parameters: Reference signal: 5 cost(t) + 10 cos (5t) Reference model: Wm = 1/ (s+1). Unknown plant, used in simulation: Wp = (s+1) / (s2-3s+2) Narendra MRAC vs. Unfalsified MRAC Error Signal : Switching sequence: Unfalsification process Narendra & Annaswamy, 1988 Convergence Of Controller parameters :

23. Direct Comparison: MRAC vs. Unfalsified T. C. Tsao and M. G. Safonov. Int. J. Adaptive Control and Signal Processing, 15:319–334, 2001. Tsao & Safonov, CCA/CACSC’99

24. Solution #2: Simplified Gradiant-Based ‘Myopic’ Unfalsified Controller (IFAC 2002) Jun and Safonov, IFAC 2002

25. Properties of ‘Myopic’ Unfalsified Adaptive Control Jun and Safonov, IFAC 2002 • Similar in most ways to traditional adaptive • Simple to implement, low computational load • Uses only local (myopic) analysis of data • Slow, quasi-static adaptation • Same as ‘MIT Rule’ if MRAC cost and no sdt • But even myopic unfalsified is better • Always fewer assumptions, often slightly faster • Data-driven scientific logic of unfalsification is inherently more reliable

26. Myopic Unfalsified Simulation (Example 1) Jun and Safonov, IFAC 2002

27. Myopic Simulation Result Jun and Safonov, IFAC 2002

28. Myopic Simulation (Example 2) Jun and Safonov, IFAC 2002

29. Simulation Result (Example 2) Jun and Safonov, IFAC 2002

30. Plant Data: at time t = 0, K ( 0 ) = 8; T = 10s; Sample time = 0.1 Unknown Plant: 1/s(s+2) with white noise; Candidate Controllers: K > 0; Cost Function:IFTrun-to-run Unfalsifiedfinite-memory + e u y r Unknown Plant K S - gain Direct comparison with Hjalmarsson’s gradient-based IFT: Wang and Safonov, 2002 Hjalmarsson et al., IEEE CDC, 1994 vs.

31. IFT Unfalsified IFT vs. Unfalsified:Simulation Setup Wang and Safonov, 2002

32. IFT vs. Unfalsified: Comparison Wang and Safonov, 2002 Experiment #1 Experiment #2 Experiment #3 IFT: Error Signal Squared Unfalsified: Error Signal Squared Experiment #1 Experiment #2 Experiment #3

33. Conclusions • Introspective models are not adequate for analyzing adaptation • To explain learning and adaptation, we need the open-eyed, data-driven scientific logic of unfalsification • Unfalsified control is better than traditional assumption-driven adaptive control, even when crippled andmyopic

34. Selected Referenceshttp://routh.usc.edu