ENGINEERING OPTIMIZATION Methods and Applications

1. ENGINEERING OPTIMIZATION Methods and Applications A. Ravindran, K. M. Ragsdell, G. V. Reklaitis Book Review

2. Chapter 5: Constrained Optimality Criteria Part 1: Ferhat Dikbiyik Part 2:Yi Zhang Review Session July 2, 2010

3. Constraints: Good guys or bad guys?

4. Constraints: Good guys or bad guys? reduces the region in which we search for optimum.

5. Constraints: Good guys or bad guys? makes optimization process very complicated

7. Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem

8. Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem

9. Equality-Constrained Problems GOAL solving the problem as an unconstrained problem by explicitly eliminating K independent variables using the equality constraints

10. Example 5.1

11. What if?

12. Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem

13. Lagrange Multipliers Converting constrained problem to an unconstrained problem with help of certain unspecified parameters known as Lagrange Multipliers

14. Lagrange Multipliers Lagrange function

15. Lagrange Multipliers Lagrange multiplier

16. Example 5.2

18. Test whether the stationary point corresponds to a minimum positive definite

20. Example 5.3

23. max positive definite negative definite

24. Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem

25. Economic Interpretation of Lagrange Multipliers The Lagrange multipliers have an important economic interpretation as shadow prices of the constraints, and their optimal values are very useful in sensitivity analysis.

26. Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem

27. Kuhn-Tucker Conditions

28. NLP problem

29. Kuhn-Tucker conditions (aka Kuhn-Tucker Problem)

30. Example 5.4

31. Example 5.4

32. Example 5.4

33. Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem

34. Kuhn-Tucker Theorems Kuhn – Tucker Necessity Theorem Kuhn – Tucker Sufficient Theorem

35. Kuhn-Tucker Necessity Theorem • Let • f, g, and h be differentiable functions • x* be a feasible solution to the NLP problem. • and for k=1,….,K are linearly independent

36. Kuhn-Tucker Necessity Theorem • Let • f, g, and h be differentiable functions x* be a feasible solution to the NLP problem. • and for k=1,….,K are linearly independent at the optimum • If x* is an optimal solution to the NLP problem, then there exists a (u*, v*) such that (x*,u*, v*) solves the KTP given by KTC. Constraint qualification ! Hard to verify, since it requires that the optimum solution be known beforehand !

37. Kuhn-Tucker Necessity Theorem For certain special NLP problems, the constraint qualification is satisfied: When all the inequality and equality constraints are linear When all the inequality constraints are concave functions and equality constraints are linear ! When the constraint qualification is not met at the optimum, there may not exist a solution to the KTP

38. Example 5.5 x* = (1, 0) and for k=1,….,K are linearly independent at the optimum

39. Example 5.5 x* = (1, 0) No Kuhn-Tucker point at the optimum

40. Kuhn-Tucker Necessity Theorem Given a feasible point that satisfies the constraint qualification not optimal optimal If it does not satisfy the KTCs If it does satisfy the KTCs

41. Example 5.6

42. Kuhn-Tucker Sufficiency Theorem • Let • f(x) be convex • the inequality constraints gj(x) for j=1,…,J be all concave function • the equality constraints hk(x) for k=1,…,K be linear If there exists a solution (x*,u*,v*) that satisfies KTCs, then x* is an optimal solution

43. Example 5.4 • f(x) be convex • the inequality constraints gj(x) for j=1,…,J be all concave function • the equality constraints hk(x) for k=1,…,K be linear

44. Example 5.4 • f(x) be convex semi-definite

45. Example 5.4 • f(x) be convex • the inequality constraints gj(x) for j=1,…,J be all concave function v g1(x) linear, hence both convex and concave negative definite

46. Example 5.4 • f(x) be convex • the inequality constraints gj(x) for j=1,…,J be all concave function • the equality constraints hk(x) for k=1,…,K be linear v

47. Remarks For practical problems, the constraint qualification will generally hold. If the functions are differentiable, a Kuhn–Tucker point is a possible candidate for the optimum. Hence, many of the NLP methods attempt to converge to a Kuhn–Tucker point.

48. Remarks When the sufficiency conditions of Theorem 5.2 hold, a Kuhn–Tucker point automatically becomes the global minimum. Unfortunately, the sufficiency conditions are difficult to verify, and often practical problems may not possess these nice properties. Note that the presence of one nonlinear equality constraint is enough to violate the assumptions of Theorem 5.2

49. Remarks The sufficiency conditions of Theorem 5.2 have been generalized further to nonconvex inequality constraints, nonconvex objectives, and nonlinear equality constraints. These use generalizations of convex functions such as quasi-convex and pseudoconvex functions