Technical College / Al-Najaf Communications Techniques Eng. Dpt.

1. Al-Mustansiriyah University College of Engineering Electrical Engineering Department Ph.D.courses /Comm. Eng./2018-2019 Technical College / Al-Najaf Communications Techniques Eng. Dpt. Technical Engineering College / Al-Najaf Communications Techniques Eng. Dpt. Computer Networks-4th Class-2011/2012 Computer Networks-4thClass-2015/2016 Optimization Lect.4: One Dimension Minimization Methods Mon. 15/10/2018

3. Al-Mustansiriya University College of Engineering Electrical Engineering Department PhD courses /Comm. Eng./2018-2019 Technical College / Al-Najaf Communications Techniques Eng. Dpt. Technical Engineering College / Al-Najaf Communications Techniques Eng. Dpt. Computer Networks-4th Class-2011/2012 Computer Networks-4thClass-2015/2016 Main Topics • Cubic Interpolation Method • Direct Root Method

4. c Cubic Interpolation Method • Linear interpolation is the simplest method of getting values at positions in between the data points. • Quadratic interpolation provide a smooth transition between adjacent segments. Linear Quadratic Cubic

5. c Cubic Interpolation Method • Cubic interpolation offers true continuity between the segments. • We need sufficient condition to have a regular curve, without the sharp edges. • We do that by using the derivative of the function which satisfy the condition, the slope on the leftis the same as the one on its right • The cubic interpolation method finds the minimizing step length in four stages Stage4 Stage1 Stage2 Stage3 Bounds on Normalization Refitting Approximation

6. c Cubic Interpolation Method Stage1: Normalization Recall that Where Note:This stage is not required if the one-dimensional minimization Stage2: To establish lower and upper bounds on the optimal step size , we need to find two points A and B at which the slope has different signs.

7. c Stage 3: Approximation 𝑇𝑜 𝑓𝑖𝑛𝑑 𝑡ℎ𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠 𝑎, 𝑏, 𝑐 and d 𝑖𝑛 Evaluate at points named A and B C1

8. c C1 Stage 3: Approximation C2

9. c C2 The necessary condition for the minimum of is Stage 3: Approximation The sufficiency condition for the minimum of leads to the relation By substituting the expressions for we obtain: End

10. c Stage 4: The value of found in stage 3 is the true minimum of and may not be close to the minimum of Hence the following convergence criteria can be used before choosing : If the criteria stated is not satisfied, a new cubic equation is used

11. c Example// Find the minimum of by the cubic interpolation method. Sol. Since this problem has not arisen during a multivariable optimization process, we can skip stage 1. To find at which is nonnegative, we start with = 0.4 and We evaluate the derivative at , 2, 4. This gives

12. c Thus we find that * Iteration 1 To find the value of and to test the convergence criteria, we first compute as

13. c Hence By discarding the negative value, we have Convergence criterion: If is close to the true minimum, , then should be approximately zero. Since

14. c Since this is not small, we go to the next iteration or refitting. As and Thus

15. c

16. c

17. c Direct Root Methods * The direct root methods seek to find the root (or solution) of the equation, = 0. * The necessary condition for to have a minimum of is that = 0. Three root-finding methods • The Newton method • The quasi-Newton method • and the Secant methods

18. c The Newton Method We consider the quadratic approximation of the function f (λ) at using the Taylor’s series expansion: By setting the derivative of Eq. 3 equal to zero for the minimum of ), we obtain If denotes an approximation to the minimum of Eq. 4 can be rearranged to obtain an improved approximation as

19. c The Newton Method The iterative process given by Eq. 5 can be assumed to have converged when the derivative, , is close to zero: Example// Find the minimum of the function using the Newton method with the starting point . Use

20. c The Newton Method Sol. The first and second derivatives of the function are given by

21. c The Newton Method

22. c The Newton Method

23. c Thank You Any Questions?