MATH 685/ CSI 700/ OR 682 Lecture Notes

1. MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations

2. Nonlinear Equations • Given a function f, we are looking for a value x, s.t. f(x)=0 (a root of the equation, or a zero of the function f). The problem is called root finding or zero finding.

3. Example

4. Existence/uniqueness

5. Examples in 1d

6. Example of a system in 2d

7. Multiplicity

8. Sensitivity and conditioning

9. Sensitivity and conditioning

10. Sensitivity and conditioning

11. Convergence rate

12. Convergence rate

13. Bisection method

14. Example: bisection iteration

15. Bisection method

16. Fixed-point iterations

17. Examples

18. Example: fixed point problems

19. Examples: FPI

20. Example: FPI

21. Convergence of FPI

22. Newton’s method

23. Newton’s method

24. Newton’s method

25. Convergence of Newton’s method

26. Newton’s method

27. Secant method

28. Secant method

29. Example

30. Higher-degree interpolation

31. Inverse interpolation

32. Inverse quadratic interpolation

33. Example

34. Linear fractional interpolation

35. Example

36. Safeguarded methods • Rapidly convergent methods for solving nonlinear equations may not converge unless started close to solution, but safe methods are slow • Hybrid methods combine features of both types of methods to achieve both speed and reliability • Use rapidly convergent method, but maintain bracket around solution • If next approximate solution given by fast method falls outside bracketing interval, perform one iteration of safe method, such as bisection • Fast method can then be tried again on smaller interval with greater chance of success • Ultimately, convergence rate of fast method should prevail • Hybrid approach seldom does worse than safe method, and usually does much better • Popular combination is bisection and inverse quadratic interpolation, for which no derivatives required

37. Zeros of polynomials

38. Systems of nonlinear equations Solving systems of nonlinear equations is much more difficult than scalar case because: • Wider variety of behavior is possible, so determining existence and number of solutions or good starting guess is much more complex • There is no simple way, in general, to guarantee convergence to desired solution or to bracket solution to produce absolutely safe method • Computational overhead increases rapidly with dimension of problem

39. Fixed-point iteration (FPI)

40. Newton’s method

41. Example

42. Example

43. Convergence of Newton’s method

44. Cost of Newton’s method

45. Secant updating methods

46. Broyden’s method

47. Broyden’s method

48. Example

49. Example

50. Example