L22 Numerical Methods part 2

1. L22 Numerical Methods part 2 • Homework • Review • Alternate Equal Interval • Golden Section • Summary • Test 4

3. Problem 10.4 Yes, descent direction

4. Prob 10.10 No, not a descent direction

5. Prob 10.19 Slope

6. Prob 10.19 cont’d

7. Prob 10.19 cont’d

8. Prob 10.30

9. The Search Problem • Sub Problem A Which direction to head next? • Sub Problem B How far to go in that direction?

10. Search Direction… Min f(x): Let’s go downhill! Descent condition

11. Step Size? How big should we make alpha? Can we step too “far?” i.e. can our step size be chosen so big that we step over the “minimum?”

12. We are here Which direction should we head? Figure 10.2 Conceptual diagram for iterative steps of an optimization method.

13. Some Step Size Methods • “Analytical” Search direction = (-) gradient, (i.e. line search) Form line search function f(α) Findf’(α)=0 • Region Elimination (“interval reducing”) Equal interval Alternate equal interval Golden Section

14. Nonunimodal functions Unimodal if stay in locale? Figure 10.5 Nonunimodal function f() for 0    0

15. Monotonic Increasing Functions

16. Monotonic Decreasing Functions continous

17. Unimodal functions monotonic decreasing then monotonic increasing monotonic increasing then monotonic decreasing Figure 10.4 Unimodal function f().

18. Some Step Size Methods • “Analytical” Search direction = (-) gradient, (i.e. line search) Form line search function f(α) Findf’(α)=0 • Region Elimination (“interval reducing”) Equal interval Alternate equal interval Golden Section

19. Analytical Step size Slope of line search= Slope of line at fmin Figure 10.3 Graph of f() versus .

20. Analytical Step Size Example

21. Alternative Analytical Step Size New gradient must be orthogonal to d for

22. Some Step Size Methods • “Analytical” Search direction = (-) gradient, (i.e. line search) Form line search function f(α) Findf’(α)=0 • Region Elimination (“interval reducing”) Equal interval Alternate equal interval Golden Section

23. “Interval Reducing”Region elimination “bounding phase” Figure 10.6 Equal-interval search process. (a) Phase I: initial bracketing of minimum. (b) Phase II: reducing the interval of uncertainty. Interval reduction phase”

24. 2 delta!

25. Successive-Equal Interval Algorithm “Interval” of uncertainty

26. More on bounding phase I Swan’s method Fibonacci sequence

27. Successive Alternate Equal Interval Assume bounding phase has found Min can be on either side of Point values… not a line But for sure its not in this region!

28. Successive Alt Equal Int Requires two function evaluations per iteration

29. Fibonacci Bounding Figure 10.8 Initial bracketing of the minimum point in the golden section method.

30. Golden section Figure 10.9 Graphic of a section partition.

31. Golden Section Example

32. Summary • General Opt Algorithms have two sub problems: search direction, and step size • Descent condition assures correct direction • For line searches…in local neighborhood… we can assume unimodal! • Step size methods: analytical, region elimin. • Region Elimination (“interval reducing”) Equal interval Alternate equal interval Golden Section