Linear-Quadratic-Gaussian Problem

1. Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001

2. Outline • Linear-Quadratic Problem • Calculus of Variation Approach • Dynamic Programming Approach • Kalman Filter • Linear Feedback Control

3. N Linear-Quadratic Problem subject to linearsystem dynamics given the initial state x(0) where x(i) is the state variables at time i u(i) is the control variable at time i A(i) and B(i) are the cost matrices at time i

4. N Calculus of Variation Approach • Let

5. N Dynamic Programming Approach • Cost-to-gowithwhereand

6. N DP Approach (cont.) • Set • We have • LetandThen

7. N DP Approach (cont.) • By induction, we have the optimal solution to bewhereand with boundary conditionwith

8. N Static State subject to state transitionwith initial conditionwhereand  is noise

9. N Static State (cont.) • Set • Thus,

10. N Static State (cont.) • New Estimate = Old Estimate + Confidence Factor  Correction Term

11. N Dynamic State subject to state transition with disturbance (i)where and  is noise

12. N Dynamic State (cont.) • Cost-to-go • Setwhere • We have

13. N Dynamic System (cont.) • Cost-to-go • Substitute

14. N Dynamic System (cont.) • Set • Let Then, we have

15. N Dynamic System (cont.) • Cost-to-go • Substitute

16. N Dynamic System (cont.) • Set • Let and Then, we have

17. N With Control subject to state transitionwith disturbance (i)where and  is noise

18. N With Control (cont.) • Cost-to-go • Set • We have

19. N With Control (cont.) • Cost-to-go • Substitute

20. N With Control(cont.) • Set • Let Then, we have

21. N With Control(cont.) • Cost-to-go • Substitute

22. N With Control(cont.) • Set • Let andThen, we have

23. N Linear Feedback Control • The predicted statewith initial statebased on the estimator

24. N Linear Feedback Control (cont.) • Optimal control based on the estimated state:whereandwith boundary conditionwith

25. N Applications • Apollo program: control of the space craft. • Airplane controllers: autopilot. • Neighboring Optimal Control for Non-Linear Systems • Chemical process controller • Guidance and control

26. References: • Bryson, Jr., A. E. and Y.-C. Ho, Applied Optimal Control: Optimization, Estimation, and Control, Taylor & Francis, 1975. • Ho, Y.-C., Lecture Notes, Harvard University, 1997.