MATH 175: Numerical Analysis II

MATH 175: Numerical Analysis II Lecturer: Jomar Fajardo Rabajante IMSP, UPLB 2ndSem AY 2012-2013

5th Method: Newton’s Method/Newton-Raphson Iteration • http://www.youtube.com/watch?v=hhT25CO6wDI ____________________________________________ Theorem: Assume that r is a zero of the differentiable function f. Then if but then f has a zero of multiplicity q at r. The root is called simple if the multiplicity is one.

5th Method: Newton’s Method/Newton-Raphson Iteration Newton’s Method converges, but is slowing down near x=0. y=x^2, has root of multiplicity 2.

5th Method: Newton’s Method/Newton-Raphson Iteration Assume that Newton’s method converges to the root r. Usually, Newton’s method is quadratically(fast) convergent if the root r is simple. If the root is NOT simple Newton’s method converges linearly. Newton’s Method Convergence Theorem: Assume that the (q+1)-times continuously differentiable function f on [a,b]has a multiplicity q root at rє [a,b]. Then Newton’s method is locally convergent to r, and the error errork at iteration k satisfies

5th Method: Newton’s Method/Newton-Raphson Iteration Example: If q=3 Hence, Near convergence, this means that the error will decrease by 2/3 on each iteration (O((2/3)^k). For the previous methods, you can interpret lambda same as this. This interpretation is not applicable when lambda=0

5th Method: Newton’s Method/Newton-Raphson Iteration If f is (q+1)-times continuously differentiable on [a,b], which contains a root r of multiplicity q>1, then Modified Newton’s method converges locally and quadratically to r.

5th Method: Newton’s Method/Newton-Raphson Iteration Assuming Newton’s method is applied to a function with a zero of multiplicity q>1. The multiplicity of the zero can be estimated as the integer nearest to Apply this only if we do not know q.

An iterative method is called locally convergent to r if the method converges to r for initial guesses sufficiently close to r. Example: Newton’s Convergence Theorem guarantees convergence to a simple zero if f is twice differentiable and if we start the iteration sufficiently close to the zero.

Another Failure of Newton’s Method

5th Method: Newton’s Method/Newton-Raphson Iteration Stopping Criterion: If Newton’s method is linearly (i.e. not quadratically) convergent use (same as RegulaFalsi)

5th Method: Newton’s Method/Newton-Raphson Iteration Stopping Criterion: If Newton’s method is superlinearly (e.g. quadratically) convergent or if we do not know the order of convergence use

A GENERAL METHOD: FIXED POINT ITERATION Definition: An iteration of the form: is called a fixed point (or functional) iteration. And any x* such that is called a fixed point of g.

A GENERAL METHOD: FIXED POINT ITERATION Questions: Is secant method a fixed point iteration? Is Newton’s method a fixed point iteration? We are going to study the generalization of Secant and Newton’s methods and look at the origin of the sufficient conditions for the methods to converge to a zero of a function (e.g. Newton’s Method Convergence Theorem was proved using the Fixed Point Theorem)… Actually, everything that is true for Fixed Point Iteration is applicable to Secant and Newton’s Method.

A GENERAL METHOD: FIXED POINT ITERATION How to generate a fixed point iteration for solving roots? Example: 1st Method: Use Secant or Newton’s Transform Newton’s Transform:

A GENERAL METHOD: FIXED POINT ITERATION 2nd Method: Separate x in the equation. Examples:

A GENERAL METHOD: FIXED POINT ITERATION 2nd Method: What is the idea of this method? Solving the zero of Solving the intersection of

Fixed point: y-value is equal to x-value

x-value of the fixed point is the root of the original function

A GENERAL METHOD: FIXED POINT ITERATION Exercise: Let us go back to our original equation: From Derive this fixed point formula:

A GENERAL METHOD: FIXED POINT ITERATION Question: Can every equation f(x)=0 be turned into a fixed point problem? YES, and in many different ways… However, not all are converging!!! And actually, not all has fixed points (no intersection with y=x)!!!

A GENERAL METHOD: FIXED POINT ITERATION • How to do the iterations work? THE COBWEB DIAGRAM (Geometric representation of FPI) • draw a line segment vertically to the function • then draw a horizontal line segment to the diagonal y=x. • repeat.

DIVERGING!!! FPI fails!

What if we start here?

A GENERAL METHOD: FIXED POINT ITERATION • Intuitively, what do you think is the condition for convergence? (Hint: look at the slopes) CONVERGING!

|Slopes|>1 Slope=1 |Slopes|<1 |Slopes|<1 |Slopes|>1

A GENERAL METHOD: FIXED POINT ITERATION FIXED POINT THEOREM: Let the iteration function g be continuouson the closed interval [a,b] with g:[a,b][a,b]. Furthermore, suppose that g is differentiable on the open interval (a,b) and there exists a positive constant K<1 (capital letter K) such that |g’(x)|<K<1 for all xє(a,b). Then • The fixed point x* in [a,b] exists and is unique. • The sequence {xk} generated by xk=g(xk-1) converges to x* for any x0є[a,b]. Assignment: PROVE!!!

Example: xk=sqrt(xk-1) on [0.3,2]. We should be sure that there is a fixed point in [0.3,2] 1. g(x)=sqrt(x), continuous on [0.3,2]? YES! 2. g:[0.3,2][0.3,2]? YES! Why? This square root function is monotonically increasing. sqrt(0.3)=0.5477… and sqrt(2)=1.4142... So [sqrt(0.3),sqrt(2)] is a subset of [0.3,2]. 3. |g’(x)|=0.5/sqrt(x)<1 for all x in [0.3,2]? YES! Why? |g’(x)| is monotonically decreasing so |0.5/sqrt(0.3)|=0.91287…<1. Actually, K=0.5/sqrt(0.3) Hence, the fixed point in [0.3,2] exists and is unique. And if you use any starting value in [0.3,2], the iteration will converge to the fixed point. Try [0.1,2] would the theorem hold?

A GENERAL METHOD: FIXED POINT ITERATION Note that the hypotheses of the fixed point theorem are sufficient conditions for convergence of the iteration scheme, but not necessary. However, we may add for all xє(a,b): if |g’(x)|>1, then the iteration diverges. If |g’(x)|=1, no conclusion can be made. If |g’(x)|<1, then the fixed point is said to be attracting. If |g’(x)|>1, then the fixed point is said to be repelling.

A GENERAL METHOD: FIXED POINT ITERATION Assume that the hypotheses of the fixed point theorem are met, also assume g’ is continuous on (a,b). If g’(x*)≠0, then for any starting value in [a,b], the iteration will converge only linearly to the fixed point. Example: try to get g’(x) consider [a,b]=[-1,1]

A GENERAL METHOD: FIXED POINT ITERATION To obtain a higher-order convergence, the iteration function must have a zero derivative at the fixed point. The more derivatives of the iteration function which are zero at the fixed point, the higher will be the order of convergence.

A GENERAL METHOD: FIXED POINT ITERATION Stopping Criterion: If fixed point iteration is linearly convergent use

A GENERAL METHOD: FIXED POINT ITERATION Stopping Criterion: If fixed point iteration is superlinearly convergent or if we do not know the order of convergence use

A GENERAL METHOD: FIXED POINT ITERATION ANOTHER FIXED-POINT TRANSFORM: HALLEY’S METHOD (has cubic order of convergence for simple zeros) Halley’s Method is under the group of Householder’s Methods for root-finding.

A GENERAL METHOD: FIXED POINT ITERATION Fixed Point Iteration has many other applications other than root-finding. It is also used in the analysis of Discrete Dynamical Systems leading to the study of Chaos.

MATH 175: Numerical Analysis II