70 likes | 100 Views
Tutorial 5. Function Optimization. Line Search. Taylor Series for R n Steepest Descent. a. c. u. b. Line search runs as following. Let
E N D
Tutorial 5 Function Optimization. Line Search. Taylor Series for Rn Steepest Descent Tutorial 5-6
a c u b Line search runs as following. Let Be the scalar function of α representing the possible values of f(x) in the direction of pk. Let (a,b,c) be the three points of α, such, that the point of (constrained) minimum x’, is between a and c: a<x’<c. Then the following algorithm allows to approach x’ arbitrarily close: If b-a>c-b, u=(a+b)/2; If f(u)<f(b) (a,b,c)=(a,u,b) Else (a,b,c)=(u,b,c) Line search If b-a<c-b, u=(b+c)/2; If f(u)<f(b) (a,b,c)=(b,u,c) Else (a,b,c)=(a,b,u) Tutorial 5-6
The Taylor series for f(x) is ,where For the function of m variables, the expression is Taylor Series Tutorial 5-6
Consider the elliptic function: f(x,y)=(x-1)2+(2y-2)2and find the first three terms of Taylor expansion. 2D Taylor Series: Example Tutorial 5-6
-f’(0) 2 1 Consider the elliptic function: f(x,y)=(x-1)2+(2y-2)2and find the first three terms of Taylor expansion. Find the first step of Steepest Descent. Steepest Descent Tutorial 5-6
Consider the elliptic function: f(x,y)=(x-1)2+(2y-2)2and find the first three terms of Taylor expansion. Find the first step of Steepest Descent. Now, find a step a, in the direction of gradient, minimizing the function: Steepest Descent Tutorial 5-6
Consider the elliptic function: f(x,y)=(x-1)2+(2y-2)2and find the first three terms of Taylor expansion. Find the first step of Steepest Descent. Is it a minimum? Next step? Steepest Descent Tutorial 5-6