Historical, Generic and Current Challenges of Adaptive Control

1. Historical, Generic and Current Challenges of Adaptive Control

2. Outline • Introduction • Some old problems of adaptive control • Generic and conceptual challenges • Topical Problems • Conclusions IFAC ALCOSP 2007 St Petersburg

3. Macro-view of presentation • Show some big mistakes in Adaptive Control history, especially by theoreticians • Show some pervasive adaptive control problems still not properly addressed • Show some newer developments considered-- with continuing regrettable inadequacies IFAC ALCOSP 2007 St Petersburg

4. Thanks Recent/current collaborators: Alexander Lanzon Andrea Lecchini Michel Gevers Xavier Bombois Franky De Bruyne Steve Morse Thomas Brinsmead Sunghan Cha Michael Rotkowitz Past collaborators: Rick Johnson Iven Mareels Bob Bitmead Robert Kosut Petar Kokotovic Richard Johnstone Shankar Sastry Marc Bodson Wee Sit Lee …and to the ALCOSP 2007 organizers for inviting me! IFAC ALCOSP 2007 St Petersburg

5. Outline • Introduction • Some old problems of adaptive control • What is adaptive control? • Bursting • Rohr’s Counterexample • MIT Rule • Iterative identification and control • Generic and conceptual challenges • Topical Problems • Conclusions IFAC ALCOSP 2007 St Petersburg

6. Adaptive Control Disturbance d Controller Plant + Output Input Reference r y u — • Plant is initially unknown or partially known, or is slowly varying. • There is an underlying performance index/optimization, e.g. IFAC ALCOSP 2007 St Petersburg

7. Adaptive Control(continued) Disturbance d Controller Plant + Output Input Reference r y u — • A non-adaptive controller maps the error signal r-y into u in a causal, time-invariant way e.g. • An adaptive controller is one where parameters are adjusted. IFAC ALCOSP 2007 St Petersburg

8. Possible Form of Adaptive Controller Identifier Control Law Calculation Plant Parameters Controller Parameters disturbance r Controller Plant y u - • Often 3 time scales: • Underlying plant dynamics (with fixed parameters) • Time scale for identifying plant • Time scale of plant parameter variation • Nonlinearity is present! IFAC ALCOSP 2007 St Petersburg

9. Possible Form of Adaptive Controller Identifier Control Law Calculation Plant Parameters Controller Parameters disturbance r Controller Plant y u - • Often 3 time scales: • Underlying plant dynamics (with fixed parameters) • Time scale for identifying plant • Time scale of plant parameter variation • Nonlinearity is present! And now to the problems: Bursting Rohr’s Counterexample Iterative Identification and controller redesign MIT rule IFAC ALCOSP 2007 St Petersburg

10. Bursting Phenomenon • Plant output should follow unit step input. • Plant output yk well behaved till time 3400 • Recovery occurs by time 3500 IFAC ALCOSP 2007 St Petersburg

11. Bursting Phenomenon u(t) y(t) • Bursting phenomena can come after 1 week. Their existence had not been predicted • Why do they occur? How could they be stopped? • From measurements of u(•), y(•), one should be able to identify b and c • If u = constant, can only identify b/c-the DC gain • Adaptive controllers contain identifier of b and c IFAC ALCOSP 2007 St Petersburg

12. Bursting Phenomenon (Explanation) • Control law is designed based on estimates of b and c can accidentally implement unstable closed loop. • Instability then enriches the signals, giving improved identification. Stabilising control law is reapplied. • Practical issue: must either turn off adaptation or drive the system with “rich” input. • Set-point control does not provide a rich input! IFAC ALCOSP 2007 St Petersburg

13. Rohr’s Counterexample • Ingredients: • A true system with high frequency dynamics uncaptured by a low complexity model • Nonpersistently excited input • An adaptive control task • Observation: Instability • Claim of authors: • All adaptive control theorems require an ‘SPR’ condition • Unmodelled high frequency dynamics must destroy SPR • Therefore, no adaptive control system can ever work!!! • Explanation (not offered by the authors): • Mixture of MIT-rule like error (later) and bursting IFAC ALCOSP 2007 St Petersburg

14. Iterative Identification & Controller Redesign • A frequently advanced approach to adaptive control design is iterative identification and controller redesign. • One iteration comprises • (re) identifying the plant with the current controller • redesigning the controller to achieve the design objective on the basis of the identified model, • implementing it on the real plant • This is like adaptive control, with a long wait between controller updates, due to careful identification. It can lead to instability! • Explanation will come. IFAC ALCOSP 2007 St Petersburg

15. MIT Rule Problem kc(t) kpZp(s) yp(t) r(t) + e(t) - kmZm(s) ym(t) • Zp(s) is approximately known and modelled by Zm(s), kmis known, kpis positive and unknown, but kc(t) is known and adjustable • Problem: find a rule using e(t) to adjust kc(t) to cause e(t) to go to zero • Problem source: kpdepends on dynamic air pressure for aircraft. • Simple gradient descent algorithm (gain g) can be found IFAC ALCOSP 2007 St Petersburg

16. Example of performance g  • Unshaded region • is stable • Sine wave input at frequency  • Plant is (s+1)-1 IFAC ALCOSP 2007 St Petersburg

17. Performance: second example g  • Unshaded region • is stable • Sine wave input • at frequency  • Plant is e-s(s+1)-1 • while model is still • (s+1)-1 IFAC ALCOSP 2007 St Petersburg

18. Explaining Instability • First instability mechanism is interaction of excited plant dynamics with adaptive dynamics, made worse at high gain g • Problem is fixed by separating time scales (adapting slowly), with averaging theory justifying • Second separate instability mechanism comes from exciting at frequencies where Zp(s) and Zm(s) are significantly different. • No part of the adaptation mechanism was designed to deal with this. • Understanding of cause allowed technique to be safely used elsewhere, including radio telescope design in Australia. IFAC ALCOSP 2007 St Petersburg

19. Outline • Introduction • Some old problems of adaptive control • Generic and conceptual challenges • Impractical control objectives • Transient instability • Suddenly unstable closed loops • Changing experimental conditions • Topical Problems • Conclusions IFAC ALCOSP 2007 St Petersburg

20. Impractical objective Disturbance d Controller Plant + Output Input Reference r y u — • Plant is initially unknown or partially known, or is slowly varying. • There is an underlying performance index, e.g. • The closed loop with known plant may have phase margin of 2°. The index may not be practically achievable AND--separate point-- you may not know! IFAC ALCOSP 2007 St Petersburg

21. Impractical objective Disturbance d Controller Plant + Output Input Reference r y u — • Good features of an adaptive control algorithm are • It will identify that an objective is infeasible • It will gracefully terminate • Windsurfer approach to adaptive control does this • It would be desirable if all adaptive control algorithms had this feature!!! The index may not be practically achievable AND--separate point-- you may not know! IFAC ALCOSP 2007 St Petersburg

22. Transient instability • Theorem: Consider the plant X, and the adaptive control law Y. Under conditions A,B,C, as t, the parameter estimates do this nice thing, all signals are bounded, and performance [approaches that of known plant case, or something similar]. • This does not rule out: • The plant input assuming a value of 106 before it settles down to its steady state value of 1 • Nice bounds are hardly ever available • Editorial comment: • It is misleading and it has been counterproductive to tell people (without qualification) that stability is proven • It would be desirable to prove practical bounded gain results and stop pretending the stability proofs are enough!!! IFAC ALCOSP 2007 St Petersburg

23. Suddenly unstable closed loops • Consider a plant-controller combination which suddenly goes unstable • May be due to a fault • May be due to connecting the wrong controller • The practical problem is to change the controller to eliminate the instability in a very short time • Virtually no adaptive control theorem considers this scenario • An additional complication is that the closed-loop signal may no longer be rich; the instability dominates. • Old but still relevant data may save the day IFAC ALCOSP 2007 St Petersburg

24. Changing Experimental Conditions • In adaptive control, at each time instant • There is a model of the plant (which may be a good model) • There is a certain controller attached to the plant • If the plant model is a good one, a simulation of the model and controller will perform like the actual plant and controller • In the adaptive part of adaptive control • The controller may be changed to better reflect a control objective • The calculation of the new controller is based on the current model--applying with the current controller IFAC ALCOSP 2007 St Petersburg

25. Changing Experimental Conditions Similar open-loop behaviours: and Open-loop Closed-loop IFAC ALCOSP 2007 St Petersburg

26. Changing Experimental Conditions • Moral: changing the controller may turn a good plant model into a bad one, or vice versa • If you change the controller significantly to suit the model, you might produce instability with the real plant, • This explains instability arising in iterative identification and controller redesign • It also explains it in Multiple Model Adaptive Control (later) • Safe adaptive control refers to • Ensuring you never, even temporarily, connect a destabilising controller • It may require you also to never, even temporarily, connect a controller which gives very poor performance • It would be desirable if all adaptive control algorithms had this safety feature!!! IFAC ALCOSP 2007 St Petersburg

27. Outline • Introduction • Some old problems of adaptive control • Generic and conceptual challenges • Topical Problems • Multiple Model Adaptive Control • “Model Free” Adaptive Control • Validating safety with closed-loop data • Conclusions IFAC ALCOSP 2007 St Petersburg

28. Multiple Model Adaptive Control • Imagine a bus on a city street. The equations of motion of the bus have parameters depending on • the load • The friction between tyres and road • In many plant models a (frequently small) number of physically-originating parameters are changeable/unknown. • Learning a parameter vector  from measurements with an equation of the form may be too hard, especially for nonlinear plants IFAC ALCOSP 2007 St Petersburg

29. Multiple Model Adaptive Control • An alternative approach (MMAC) is as follows: • Suppose that the unknown parameter  lies in a bounded simply connected region. Call the unknown plant . • Choose a set of values in this region, with associated plants P1,.......,PN. • Design (in advance) nice controllers for P1,.......,PN. • Call them C1,......,CN . • Run an algorithm which at any instant of time estimates (via the measurements) the particular Pi which is the best model to explain the measurements from . Call the associated parameter • Connect up IFAC ALCOSP 2007 St Petersburg

30. Multiple Model Adaptive Control Supervisor noise Unknown or Partially known Plant P Controller i y u Input + - •   Supervisor studies effect of using present controller and decides whether    or not to switch controller •   Desirable outcome: after a finite number of switchings, the best controller    for the plant is obtained. IFAC ALCOSP 2007 St Petersburg

31. Why the name “multiple model”? • Underlying precept is that the plant coincides with or is near one of N nominal plants P1,.......,PN • Controller i, denoted Ci, is a good controller for Pi(and possibly plants “near”Pi) • All the preceding is valid in principle for nonlinear     plants! IFAC ALCOSP 2007 St Petersburg

32. Deciding the Best Model for P u y1 Multi- y estimator yN + r + Controller k Plant P - • Multi-estimator is a device which produces N outputs • if (and only if with complicated signals) (The controller is irrelevant in this condition) Multi- estimator LINEAR PLANTS NOW! _ Plant P Controller k IFAC ALCOSP 2007 St Petersburg

33. Supervision using Multi-estimator u Multi- estimator Controller J _ Plant P P I C I Multi- y1 estimator y yN r + Controller J Plant P - • Idea of algorithm: study for some small a > 0, and k=1,…,N. If the smallest occurs for k = I, say that P is best modelled by Switch in This may lead to switching in a destabilising controller! IFAC ALCOSP 2007 St Petersburg

34. Example • Plant is 3rd order, stable, with non-minimum phase zero in [1,10] and DC gain in [.2,2]. • Control objective: extend bandwidth beyond open loop plant, with closed loop transfer function close to 1 in magnitude. • Non-minimum phase zero is a limiter. • 441 plant models chosen, with DC gain and non-minimum phase zero each in 20 logarithmically space intervals. • Reference signal is wideband noise • Measurement noise and process (input) noise are present IFAC ALCOSP 2007 St Petersburg

35. Example of Temporary Instability IFAC ALCOSP 2007 St Petersburg

36. Example of Temporary Instability IFAC ALCOSP 2007 St Petersburg

37. Multiple Model Adaptive Control Difficulties • Should there be 7, 70 or 7000 models? How should one actually choose the models? • How can one avoid the (temporary) instabilities? These questions are actually linked. • They can be systematically resolved using notions of a -gap     metric (tested on the 441 model case) or performance-based     tools developed by Fekri and Athans  IFAC ALCOSP 2007 St Petersburg

38. Avoiding Instability: Safe Switching • The index of best model of P (out of P1,........PN) using CJ      is     NOT NECESSARILY the index of best controller to use on P. • Even if PI is the best model of P when CJ is connected, CI may not be in safe set of {CK }. So only switch when safe. • P may be best modelled by PI when CJ is used, but best modelled by PKwhen CI is used. • PI may in fact be a terrible model of P when CI is used. • Nontrivial fact: using crude estimation techniques one can obtain set of controllers {CK } which will retain stability. • can even retain similarity of performance. • Tool for this is Vinnicombe metric •     WARNING TO USER: If you switch controllers very frequently, you       can lose  stability. IFAC ALCOSP 2007 St Petersburg

39. “Model Free” Adaptive Control • Rule 1: Never try to estimate the plant, which may be nonlinear. Big claim 1:You don’t even need a model. • Use a finite set of controllers, where one at least is guaranteed satisfactory (stability and performance) for any plant that could be encountered • At any instant of time evaluate performance with current controller and other possible controllers: • Determine external input that would have to be used with each candidate controller to get SAME plant input and output as actual • Compute the performances for each case and compare. • Switch when current performance is not best. • First catch (repairable): controllers must be ‘invertible’ (minimum phase and bi-proper) • Other catches follow…. IFAC ALCOSP 2007 St Petersburg

40. “Model Free” Adaptive Control • Rule 1: Never try to estimate the plant, which may be nonlinear. Big claim 1: You don’t even need a model. • Use a finite set of controllers, where one at least is guaranteed satisfactory (stability and performance) for any plant that could be encountered • You can’t get a finite set of controllers where one at least is fine without knowing something about the plant. “Model Free” is a misnomer in this sense. • At any instant of time evaluate performance with current controller and performances that would result with each other possible controller, • if the external input were such that the same plant input and output resulted. • Switch when current performance is not best. • It is doubtful that one would wish to compare performances for different external inputs, and identical plant inputs and outputs. IFAC ALCOSP 2007 St Petersburg

41. “Model Free” Adaptive Control • At any instant of time evaluate performance with current controller and performances that would result with each other possible controller, • if the external input were such that the same plant input and output resulted. • Switch when current performance is not best. • It is doubtful that one would wish to compare performances for different external inputs, and identical plant inputs and outputs. • You CANNOT check that other controllers are or are not stabilising. You can only actually evaluate ‘performance’ in advance IF the possible controller is stabilizing. • The method can even result in REPEATED connection of the same destabilizing controller, in aggregate for a long time. • It illustrates again the need for safeadaptive control--i.e. ability to confirm before switching that a new controller will not destabilise. IFAC ALCOSP 2007 St Petersburg

42. Validating stability with new controller C2 (modified) z r C1 Plant y u May be OK to have C1 and C2 in forward or feedback paths only. • C1 is stabilizing • C2 is candidate to replace C1 • Experiment gives transfer function T from r to z (no stability     problem, will need richness, can have noise) • C2 is stabilizing (in place of C1) if Nyquist plot of T does  not     encircle origin. IFAC ALCOSP 2007 St Petersburg

43. Validating Safety with Closed-loop data (cont.) • The validation test uses limited amount of noisy closed-loop data to guarantee [P, C2] will be stable • It assumes no a priori information about the plant • except linear and time-invariant • It can potentially address transient instability problem • The experiments can tolerate significant error • It uses the phase response only up to a finite frequency • A small controller change implies a smaller frequency band over which T must be estimated IFAC ALCOSP 2007 St Petersburg

44. Outline • Introduction • Some old problems of adaptive control • Generic and conceptual challenges • Topical Problems • Conclusions IFAC ALCOSP 2007 St Petersburg

45. Conclusions • The common but not the only thread has been: unplanned instability--but the causes are all different. IFAC ALCOSP 2007 St Petersburg

46. Conclusions • The common but not the only thread has been: unplanned instability--but the causes are all different. IFAC ALCOSP 2007 St Petersburg

47. Macro-view of presentation • There are big mistakes revealed in Adaptive Control history, especially by theoreticians • There are pervasive adaptive control problems still not properly addressed, of great practical significance. IFAC ALCOSP 2007 St Petersburg

48. IFAC ALCOSP 2007 St Petersburg