Dr. Andrej Mošat`

1. Dr. Andrej Mošat` Prof. A. Linninger, Laboratory for Product and Process Design, M/C 063University of Illinois at Chicago 13 May 2010 Equality and inequality constrained optimization on continuous functions,a brief overview of optimization strategies

2. Constrained minimization formulation Motivation example: We seek to construct a cardboard box of maximum volume, given a fixed area of cardboard. General formulation is then:

3. Constrained minimization formulation Motivation example: We seek to construct a cardboard box of maximum volume, given a fixed area of cardboard. General formulation is then: Introducing the Lagrangian Function in a general constrained problem: It implies that: Reformulation of the cardboard problem: We obtain a unique solution:

4. Inequality constraints handling methods • Primal methods • Feasible direction • Active set • Gradient projection • Penalty and barrier methods • Penalty function • Barrier function • Exact Penalty Functions • Dual and cutting plane methods • Augmented Lagrangians • Primal-Dual methods • Interior Point Methods

5. Inequality constraints handling methods • Primal methods • Feasible direction • Active set • Gradient projection • Penalty and barrier methods • Penalty function • Barrier function • Exact Penalty Functions • Dual and cutting plane methods • Augmented Lagrangians • Primal-Dual methods • Interior Point Methods

6. Feasible direction method We seek a direction vector d and a nonnegative scalar alpha: Such that the segment xk, xk+1 lies completely within thew feasible set and F(xk+1 )< F(xk) Zoutendijk Algorithm: Fmin xk xk+1 Feasible set

7. Inequality constraints handling methods • Primal methods • Feasible direction • Active set • Gradient projection • Penalty and barrier methods • Penalty function • Barrier function • Exact Penalty Functions • Dual and cutting plane methods • Augmented Lagrangians • Primal-Dual methods • Interior Point Methods

8. Active set methods We seek a minimum of inequality constrained problem subject to a set of active constraints. Active constraint transformed into: g(x) = 0,  equality constrained problem Inactive constraint: g(x) < 0  must be checked. If g(x) !< 0, activate the constraint

9. Inequality constraints handling methods • Primal methods • Feasible direction • Active set • Gradient projection • Penalty and barrier methods • Penalty function • Barrier function • Exact Penalty Functions • Dual and cutting plane methods • Augmented Lagrangians • Primal-Dual methods • Interior Point Methods

10. Gradient projection methods We seek a point xk+1, which lies on the boundary of the feasible region and is a descent F(xk+1 )< F(xk) Problem formulation requires detailed discussion, see “Luenberger David G., Ye Yinyu - Linear and Nonlinear Programming (3rd edition, Springer), 2008” for details Fmin

11. Inequality constraints handling methods • Primal methods • Feasible direction • Active set • Gradient projection • Penalty and barrier methods • Penalty function • Barrier function • Exact Penalty Functions • Dual and cutting plane methods • Augmented Lagrangians • Primal-Dual methods • Interior Point Methods

12. Penalty methods Consider the constrained problem: Idea: transform the constrained problem into an unconstrained problem: Example:

13. Barrier methods Example of (not) robust sets of points, where it is possible to get to any boundary point approaching it from the interior: Idea: transform the constrained problem into an unconstrained problem by introducing a continuous barrier function into the formulation: A barrier function is a function B defined on the interior of S such that: (1) B is continuous, (2) B(x)>=0, (3) B(x) → Inf as x approaches the boundary of S. Robust Not robust S is robust,

14. Inequality constraints handling methods • Primal methods • Feasible direction • Active set • Gradient projection • Penalty and barrier methods • Penalty function • Barrier function • Exact Penalty Functions • Dual and cutting plane methods • Augmented Lagrangians • Primal-Dual methods • Interior Point Methods

15. Cutting plane methods Consider the constrained problem: Idea: transform the problem into a Linear Programming problem: Example:

16. Inequality constraints handling methods • Primal methods • Feasible direction • Active set • Gradient projection • Penalty and barrier methods • Penalty function • Barrier function • Exact Penalty Functions • Dual and cutting plane methods • Augmented Lagrangians • Primal-Dual methods

17. References Floudas C.A., "Nonlinear and Mixed-Integer Optimization : Fundamentals and Applications", Oxford University Press, (1995). Luenberger David G., Ye Yinyu - Linear and Nonlinear Programming (3rd edition, Springer), 2008, ISBN 978-0-387-74502-2 Jorge Nocedal, Stephen J. Wright, Numerical Optimization, Springer, New York, 1999, ISBN 0-387-98793-2 D. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Athena Scientific, Belmont, Massachusetts, 1996, ISBN 1-886529-04-3 D. Bertsekas, Nonlinear Programming, 2nd Edition, Athena Scientific, Belmont, Massachusetts