PID and Model Predictive Control in a Networked Environment

1. PID and Model Predictive Control in a Networked Environment MS Thesis Defense Graham Alldredge June 28th, 2007 Department of Electrical and Computer Engineering Case Western Reserve University

2. Outline • Introduction and Previous Work • Simulation Methodologies • PID Control in NCS • Control theory, time-delay system (TDS) background • h-τ space stability regions for PID control in NCSs • Model Predictive Control (MPC) in NCS • Smith predictor and NCS stability regions • Forward predictor

3. Outline, cont’d • Play-Back Buffers with MPC • Comparisons to unbuffered PID control for systems with bounded random delays • Play-Back Buffer Design • Optimal play-back delay: optimization via simulation and an analytical method • Choosing PI gains • Comparisons to unbuffered PID control • Conclusions and Future Work

4. Network NCS Basics Reference Signal • Feedback control loop closed over a computer network • Cheap and practical Controller Plant Actuators Sensors • Drawbacks • Discretization • Time-varying, random loop delays • Packet loss

5. Select Previous Work • Nilsson (1998): Delay models and assumptions • Zhang (2001) & Hartman (2004): Stability regions in h-τ space and algorithms to find them (top) • Xiao et al. (2000): Jump-linear system modeling for systems with time-varying delays • Pohjola (2006): PID gains by optimization for NCSs (bottom) • Liberatore (2006): Play-back buffers • Control Theory: Stability analysis for time-delay systems

6. Simulation Framework • Assumptions • Constant sampling period of plant output • Time-invariant controller • Thus, variable time delay models the total RTT

7. Simulation Methodologies • TrueTime • Simulink-based co-simulation tool for real-time kernels, network dynamics E.g., Step tracking using PID control of a first-order plant with network-induced, time-varying round-trip delays Plant output and reference signal Loop delays

8. Simulation Methodologies • Queue Block for Time-Varying Delays • Created to precisely simulate the effect of time-varying delays on discrete signals, including out-of-order arrival drops

9. PID Control in NCS • Background: Basic TDS/NCS Concepts • PID Tuning Rules for TDS (O’Dwyer 2006): Roughly minimize IAE for a range of delays • Delay Margin: The smallestadditional delay whichdestabilizes the system

10. PID Control in NCS • Background: Basic TDS/NCS Concepts • Stability Windows: Intervalsof delay for which a TDS isstable • NCS Stability Regions:Simultaneously describe theeffects of delay anddiscretization on a controlsystem

11. PID Control in NCS • State-Space Model for Discrete PID Control of Continuous Plants

12. PID Control in NCS • NCS Stability Regions for PID Control of First-Order Plants Changing PID gains individually Increasing τd

13. PID Control in NCS • NCS Stability Regions for PID Control of Second-Order Plants Changing KP Changing KD, creating stability windows

14. Model Predictive Control in NCS • Smith Predictor: • Effectively remove loop delay using a plant model • State-space model

15. Model Predictive Control in NCS • Stability Regions for Smith Predictor Numerically calculated stability region and confirmation using simulation Demonstrating the effects of different gains

16. Model Predictive Control in NCS • Forward Predictor • Stability region from simulation • Practical implementation issues, e.g., choosing integration approximation Rectangular Trapezoidal

17. Play-Back Buffers • MPC performs poorly with time-varying delays • Play-back buffers hold samples until a specified play-back time is reached (Liberatore 2006) • The loop delay becomes more deterministic, significantly improving MPC performance • Drawback: an effective increase in loop delay

18. MPC with Play-Back Buffer Controller reacts after play-back delay Gains can be as aggressive as sampling time allows Ideal transient response Unbuffered PID Controller Controller reacts immediately Design gains for the delay distribution More conservative transient response Comparison of Controllers

19. Delay Distribution for Comparison • Bounded Interval Distribution: Beta • Out-of-order drops are removed

20. Choosing PID Gains • Find an “effective delay” using gains for a fixed-delay system from the tuning rule to minimize IAE

21. Performance Comparisons • Costs are compared by subtraction • No randomness in cost for forward predictor/play-back buffer • Randomness in cost for unbuffered PID controller is smoothed over several steps/phases When τmin is low and τrange is high, unbuffered PID control is superior

22. Performance Comparisons • Isocurves of equal cost are particularly interesting features of these cost surfaces • As βincreases (i.e., distribution is closer to τmin), unbuffered PID control is superior over a larger parameter range

23. Performance Comparisons • Isocurves for a faster plant – unbuffered PID control has superior performance over wider range • Isocurves for a slower plant – the aggressiveness of MPC is more valuable

24. Play-Back Buffer Design • Find the optimal play-back delay below the maximum delay • Semi-infinite interval delay distribution (Liberatore 2006): • Body: Shifted Gamma Distribution on [τmin,+∞) • Tail: Pareto Distribution (heavy-tailed) on [K,+∞) • Drops removed with 99% probability τmin= 50 ms, λ = 1000

25. Optimal Play-Back Delay • Optimization via Simulation • Used step tracking for costs • Cost function has expected behavior • Many step responses are averaged, always including one with transient affected by a long loop delay • Demonstrating relationships between the optimal play-back delay and distribution parameters • Seeking a connection between the optimal play-back delay and the delay distribution

26. Optimal Play-Back Delay • Analytical Method • Define a cost associated with a loop delay and a play-back delay using the worst-case delay spike • Delay spike cost as a function of loop delay for different play-back delays • For each play-back delay, weight the cost from each loop delay using the delay distribution, and compare this “play-back cost” to those from simulation

27. Optimal Play-Back Delay • Comparing the Methods • Results as r increases • CDF values of the optimal play-back delay

28. Choosing PI Gains • Can performance be improved using more conservative gains and a more aggressive play-back delay? • Use optimization via simulation method

29. Choosing PI Gains • For some plants and delay distributions, the difference in performance is small

30. Final Comparison: MPC & Optimal Play-Back vs. Unbuffered PID • Model predictive control with an optimal play-back buffer always outperformed an unbuffered PID controller for this first-order plant

31. Conclusions • Two simulation methodologies reviewed: TrueTime, own queue block implementation (used in experiments) • Reviewed continuous-time, time-delay system concepts • Developed a state-space model for discrete PID control of a continuous-time plant • Calculated NCS stability regions for PID control of first- and second-order systems • Demonstrated the extension of TDS concepts into h-τ space • Reviewed two model predictive controllers: the Smith predictor and a forward predictor • Constructed a state-space model for a discrete Smith predictor and presented NCS stability regions • Analyzed implementation issues for forward predictor

32. Conclusions, cont’d • Explored the benefits and drawbacks of removing all uncertainty in the delay using a play-back buffer • Compared the performance to unbuffered PID control for first-order plants and saw relationships with the delay distribution • Developed simulation and analytical methods for determining the optimal play-back delay for delays from a heavy-tailed distribution • Demonstrated that the most aggressive gains will usually yield the best performance, even if the play-back delay must be more conservative • Showed superior performance for the play-back scheme, compared to unbuffered PID control for a first-order plant

33. Future Work • Stability tools for systems with random delays • Discretization effects with play-back buffers, including co-simulation • Full implementation of Liberatore’s integrated play-back buffer scheme • Variable optimal play-back delay • Variable sampling time (e.g., using minimum attention results or as a consequence of congestion control) • Contingency control and its play-back delay • What happens to the results when delays are not independent (e.g., from a Markov process)?

34. Acknowledgements • Advisor: Dr. Michael Branicky • Dr. Vincenzo Liberatore (Honorary Advisor) • Dr. Wei Lin • Ahmad Al-Hammouri