Nonlinear Equations

1. Nonlinear Equations Jyun-Ming Chen

2. Contents • Bisection • False Position • Newton • Quasi-Newton • Inverse Interpolation • Method Comparison

3. Consider the problem in the following general form: f(x) = 0 Many methods to choose from: Interval Bisection Method Newton Secant … Solve the Problem Numerically

4. Recall the following theorem from calculus Intermediate Value Theorem (中間值定理) If f(x) is continuous on [a,b] and k is a constant, lies between f(a) and f(b), then there is a value x[a,b] such that f(x) = k Interval Bisection Method

5. Bisection Method (cont) • Simply setting k = 0 • Observe: • if sign( f(a) ) ≠ sign( f(b) ) • then there is a point x [a, b] such that f(x) = 0

6. non-trivial interval [a,b]: f(a) ≠ 0, f(b) ≠ 0 and sign( f(a) ) ≠ sign( f(b) ) sign(-2) = -1 sign(+5) = 1 Definition

7. Start with a non-trivial interval [a,b] Set c(a+b)/2 Three possible cases: ⑴ f(c) ＝ 0, solution found ⑵ f(c) ≠ 0, [c,b] nontrivial ⑶ f(c) ≠ 0, [a,c] nontrivial Keep shrinking the interval until convergence ⑴ → problem solved ⑵⑶ → a new smaller nontrivial interval Idea _______ ½ size

8. Algorithm What’s wrong with this code?

9. Remarks • Convergence • Guaranteed once a nontrivial interval is found • Convergence Rate • A quantitative measure of how fast the algorithm is • An important characteristics for comparing algorithms

10. Let: Length of initial interval L0 After k iterations, length of interval is Lk Lk=L0/2k Algorithm stops when Lk eps Plug in some values… Convergence Rate of Bisection This is quite slow, compared to other methods… Meaning of eps

11. How to get initial (nontrivial) interval [a,b] ? • Hint from the physical problem • For polynomial equation, the following theorem is applicable: roots (real and complex) of the polynomial f(x) = anxn + an-1xn-1 +…+ a1x + aο satisfy the bound:

12. Roots are bounded by Hence, real roots are in [-10,10] Roots are –1.5251, 2.2626 ± 0.8844i Example complex

13. Other Theorems for Polynomial Equations • Sturm theorem: • The number of real roots of an algebraic equation with real coefficients whose real roots are simple over an interval, the endpoints of which are not roots, is equal to the difference between the number of sign changes of the Sturm chains formed for the interval ends.

14. Sturm Chain

15. Example

16. Sturm Theorem (cont) • For roots with multiplicity: • The theorem does not apply, but … • The new equation : f(x)/gcd(f(x),f’(x)) • All roots are simple • All roots are same as f(x)

17. Sturm Chain by Maxima

18. Maxima (cont)

19. A method of determining the maximum number of positive and negative real roots of a polynomial. For positive roots, start with the sign of the coefficient of the lowest power. Count the number of sign changes n as you proceed from the lowest to the highest power (ignoring powers which do not appear). Then n is the maximum number of positive roots. For negative roots, starting with a polynomial f(x), write a new polynomial f(-x) with the signs of all odd powers reversed, while leaving the signs of the even powers unchanged. Then proceed as before to count the number of sign changes n. Then n is the maximum number of negative roots. Descarte’s Sign Rule 3 positive roots 4 negative roots

20. x2 defined as the intersection of x axis and x0f0-x1f1 Choose [x0,x2] or [x2,x1], whichever is non-trivial Continue in the same way as bisection Compared to bisection: x2=(x1+x0)/2 False Position Method

21. Determine intersection point Using similar triangles: False Position (cont)

22. Alternatively, the straight line passing thru (x0,f0) and (x1,f1) Intersection: simply set y=0 to get x False Position (cont)

23. Example

24. False Position • Always better than bisection?

25. tangent linethru(x0 , f0) (x0, f0) Also known as Newton-Raphson method Newton’s Method Graphical Derivation

26. Newton’s Method (cont) • Derived using Taylor’s expansion

27. Taylor’s Expansion (cont)

28. Old Barbarians used the following formula to compute the square root of a number a explain why this works: Finding square root of 1 (a=1) with x0 = 2 Example

29. Newton’s Method

30. Fast Inverse Square Root To understand the magic number 0x5f3759df, read: Chris Lomont or Paul Hsieh

31. Definition • Error of the ith iterate • Order of a method m, satisfies where Ek is an error bound of ek

32. Linear Convergence of Bisection root L0 a0 b0 L1 a1 b1 L2 a2 b2

33. Linear Convergence of Bisection (cont) • We say the order of bisection method is one, or the method has linear convergence

34. Quadratic Convergence of Newton • Let x* be the converged solution • Recall

35. Quadratic Convergence of Newton (cont) • Subtracting x*: Or we say Newton’s method has quadratic convergence

36. Example: Newton’s Method • f(x)= x3–3x2 –x+9=0 Worse than bisection !?

37. plot f(x) Plot xk vs. k xk 60 30 10 5 k -1.525 -1.525 -10 25 35 40 Why?

39. Case 1:

40. Case 2: Diverge to 

41. Recall Quadratic Convergence of Newton’s • The previous example showed the importance of initial guess x0 • If you have a good x0, will you always get quadratic convergence? • The problem of multiple-root

42. Example Prove this!! • f(x)=(x+1)3=0 • Convergence is linear near multiple roots

43. Multiple Root • If x* is a root of f(x)=0, then (x-x*) can be factored out of f(x) • f(x) = (x-x*) g(x) • For multiple roots: • f(x) = (x-x*)k g(x) • k>1 and g(x) has no factor of (x-x*)

44. Multiple Root (cont) Implication:

45. Remedies for Multiple Roots • where k is the multiplicity of the root • Get quadratic convergence! • Problem: do not know k in advance!

46. Modified Newton’s Method

47. Modified Newton’s Method (cont)

48. Example f(x)=(x–1)3sin((x – 1)2)

49. Quasi-Newton’s Method • Recall Newton: • The denominator requires derivation and extra coding • The derivative might not explicitly available (e.g., tabulated data) • May be too time-consuming to compute in higher dimensions

50. Quasi-Newton (cont) • Quasi: • where gk is a good and easily computed approx. to f’(xk) • The convergence rate is usually inferior to that of Newton’s