Numerical Methods

1. Numerical Methods Solution of Equation

2. What computers can’t do • Solve (by reasoning) general mathematical problems  they can only repetitively apply arithmetic primitives to input. • Solve problems exactly. • Represent all numbers. Only a finite subset of the numbers between 0 and 1 can be represented.

3. Root Finding Methods • Bisection Method • Secant Method • Modified Secant • Successive approximation • Newton Raphson method • Berge Vieta

4. Motivation • Many problems can be re-written into a form such as: • f(x,y,z,…) = 0 • f(x,y,z,…) = g(s,q,…)

5. Motivation • A root, r, of function f occurs when f(r) = 0. • For example: • f(x) = x2 – 2x – 3 has tworoots at r = -1 and r = 3. • f(-1) = 1 + 2 – 3 = 0 • f(3) = 9 – 6 – 3 = 0 • We can also look at f in its factored form. f(x) = x2 – 2x – 3 = (x + 1)(x – 3)

6. Finding roots / solving equations • General solution exists for equations such as ax2 + bx + c = 0 • The quadratic formula provides a quick answer to all quadratic equations. • However, no exact general solution (formula) exists for equations with exponents greater than 4. • Transcendental equations: involving geometric functions (sin, cos), log, exp. These equations cannot be reduced to solution of a polynomial.

7. Examples

8. Problem-dependent decisions • Approximation: since we cannot have exactness, we specify our tolerance to error • Convergence: we also specify how long we are willing to wait for a solution • Method: we choose a method easy to implement and yet powerful enough and general • Put a human in the loop: since no general procedure can find roots of complex equations, we let a human specify a neighbourhood of a solution

9. Bisection Method • Based on the fact that the function will change signs as it passes thru the root. • Suppose we know a function has a root between a and b. (…and the function is continuous, … and there is only one root) • f(a)*f(b) < 0 • Once we have a root bracketed, we simply evaluate the mid-point and halve the interval.

10. Bisection Method • c=(a+b)/2 f(a)>0 f(c)>0 a c b f(b)<0

11. Bisection Method • Guaranteed to converge to a root if one exists within the bracket. a = c f(a)>0 b c a f(c)<0 f(b)<0

12. Bisection method… • Check if solution lies between aand b…F(a)*F(b) < 0 ? • Try the midpoint m: compute F(m) • If |F(m)| < tolselectm as your approximate solution • Otherwise, if F(m) is of opposite sign to F(a) that is ifF(a)*F(m) < 0, then b = m. • Else a = m.

13. Bisection Method • Simple algorithm: Given: a and b, such that f(a)*f(b)<0 Given: error tolerance, err c=(a+b)/2.0; // Find the midpoint While( |f(c)| > err ) { if( f(a)*f(c) < 0 ) // root in the left half b = c; else // root in the right half a = c; c=(a+b)/2.0; // Find the new midpoint } return c;

14. Square root program • If the input c < 1, the root lies between c and 1. • Else, the root lies between 1 and c. • The (positive) square root function is continuous and has a single solution. c = x2 Example: F(x) = x2 - 4 F(x) = x2 - c

15. Example: bisection iteration

16. Problem with Bisection • Although it is guaranteed to converge under its assumptions, • Although we can predict in advance the number of iterations required for desired accuracy (b - a)/2n <e -> n > log((b - a ) / e ) • Too slow! Computer Graphics uses square roots to compute distances, can’t spend 15-30 iterations on every one! • We want more like 1 or 2, equivalent to ordinary math operation.

17. Secant method

18. Secant method

19. Secant Illustration F(x) = x2 - 10 • 1 (a=1, fa=-9) (b=10, fb=90) •  int = 1.8, fint = -6.7 • 2(a=10, fa=90) (b=1.8, fb= -6.7) • int = 0.88, fint = -9.22 3 (a=1.8, fa=-6.7) (b=0.88, fb=-9.22) • int = 4.25, fint = 8 4 (a=0.88, fa=-9.22) (b=4.25, fb=8) • Int =2.68, fint = -2.8 Etc… 2 3 1 4

20. double secant(double c, int iterations, double tol)… for ( int i = 0; i < iterations; i++) { double x = ( fa*b - fb*a ) / ( fa - fb ); double fx = func( x, c ); if ( fabs( fx ) < tol ) return x; a = b; fa = fb; b = x; fb = fx; } return -1; SECANT METHOD The change from ordinary regula is that the sign check is dropped and points are just “shifted over”

21. Example

22. Ex. • Use secant method to estimate the root of f(x) = ln x. • Start the computation with value of • xl = xi-1 = 0.5 • xu = xi = 5.0 • Solution The secant method

23. Modified Secant Method

24. Modified Secant Method Original Secant Method Modified Secant Method

25. Ex. • Use the secant method to estimate the root of f(x) = e-x – x. Use a value of 0.01 for  and start with x0 = 1.0. • Solution (true root = 0.56714329…) • First iteration • x0 = 1 f(x0) = -0.63212 • x1+ x0 = 1.01 f(x1+ x0) = -0.64578 • Calculate t = 5.3%

26. Newton’s Method • Open solution, that requires only one current guess. • Root does not need to be bracketed. • Consider some point x0. • If we approximate f(x) as a line about x0, then we can again solve for the root of the line.

27. Newton’s Method • Solving, leads to the following iteration:

28. Newton’s method

29. Finding a square-root • Example: 2 = 1.4142135623730950488016887242097 • Let x0 be one and apply Newton’s method.

30. Finding a square-root • Example: 2 = 1.4142135623730950488016887242097 • Note the rapid convergence • Note, this was done with the standard Microsoft calculator to maximum precision.

31. Newton’s method

32. Newton’s Algorithm • Requires the derivative function to be evaluated, hence more function evaluations per iteration. • A robust solution would check to see if the iteration is stepping too far and limit the step. • Most uses of Newton’s method assume the approximation is pretty close and apply one to three iterations blindly.

33. Division by Multiplication • Newton’s method has many uses in computing basic numbers. • For example, consider the equation: • Newton’s method gives the iteration:

34. Actual performance (tol = 1e-15)

35. Examples • Locate the first nontrivial root of sin x = x3 using Bisection method with the initial interval from 0.5 to 1. perform computation until error 2% • Determine the lowest positive root of: f(x) = 8 e-x sin (x) - 1 Using the Newton-Raphson method (three iterations, x0 = 0.3) and Using the secant method (four iterations, x-1 = 0.5 and x0 = 0.4). Using the modified secant method (three iterations, x0 = 0.3, d = 0.01).

36. Quiz • What do you know about mathematical model in solving engineering problem? • Use zero through third order Taylor series expansions to predict f(2.5) for f(x) = ln x using a base point at x = 1. Compute the true percent relative error for each approximation. • Determine the real root of f(x)= 5x3-5x2+6x-2 using bisection method. Employ initial guesses of xl = 0 and xu = 1. iterate until the estimated error a falls below a level of s = 15%