EML4552 - Engineering Design Systems II (Senior Design Project)

1. EML4552 - Engineering Design Systems II(Senior Design Project) Optimization Theory and Optimum Design Unconstrained Optimization (Lagrange Multipliers) Hyman: Chapter 10

2. y=f(x) df/dx=0 x Unconstrained Optimization • In 1-D the optimum is determined by:

3. Unconstrained Optimization • The condition for a local optimum can be extended to multi-dimensions x2 x1

4. Unconstrained Optimization • Condition for local optimum in unconstrained problem • However, most optimization problems are constrained

5. Optimization • Minimize (Maximize) an Objective Function of certain Variables subject to Constraints

6. Lagrange Multipliers • An analytical approach for solving constrained optimization problems • Particularly suited for problems in which the objective function and the constraints can be expressed analytical (even if highly non-linear) • Could be numerically implemented for more general cases • Will present the method through a simple example, it can be generalized for more complex problems

7. Lagrange Multiplers: Example • Determine the dimensions of a rectangular storage container to minimize fabrication costs, the container will hold a volume V, and be made of steel in the bottom (at a cost of S $/unit surface), and wood on the side (at a cost of W $/unit surface)

8. Lagrange Multipliers: Example • In ‘principle’, we could ‘solve’ for z in terms of x and y. Substitute back into the equation for cost to obtain C(x,y) and then apply the condition dC/dx=dC/dy=0 • This method, although correct in principle, could be very complex if we had many variables and constraints, or when the equations involved are difficult to solve (or involve numerical models) • A more general method is needed to approach constrained optimization problems.

9. Lagrange Multipliers: Example • Rewrite the constraint: • Define the Lagrangian as: • Notice that we have added “zero” to the objective function

10. Lagrange Multipliers: Example • Have turned a 3-D constrained problem into a 4-D unconstrained problem

11. Lagrange Multipliers: Example • The solution to the set of 4 equations in 4 unknowns is the optimum we seek. We need to solve the system, in this case:

12. Lagrange Multipliers: Example • Substituting:

13. Lagrange Multipliers: Example • Solutions: • x=y means the optimum occurs when the bottom of the container is ‘square’ • (the second solution can be shown to be the same condition x=y)

14. Lagrange Multipliers: Example • Substituting:

15. Lagrange Multipliers: General Case

16. Lagrange Multipliers: General Case

17. Lagrange Multipliers: General Case

18. Lagrange Multipliers: General Case

19. Other Optimization Methods • Step 1: Convert a constrained optimization problem into an unconstrained problem by use of ‘penalty’ functions

20. Other Optimization Methods • Step 2: Use a ‘search’ method to obtain the optimum (numerical probing of the objective function) • Random search • Directed search • Hybrid search • Combination of methods (‘decomposition’, sequential application, etc.)

21. Search Methods • The challenge is to create an ‘efficient’ search method that at the same time ensures we find the ‘global’ optimum and not just a local optimum • Random search • Steepest descent • “Simplex” (polyhedron) search • Genetic algorithm • Simulated annealing