1 / 19

Chapter 7 Iterative Techniques in Matrix Algebra

Solve. Equivalently transform into . Then start the iteration from an initial guess and obtain the (convergent) sequence. Similar to the fixed-point iterations for solving f ( x ) = 0 ……. Idea. What to analyze?.

pbrandi
Download Presentation

Chapter 7 Iterative Techniques in Matrix Algebra

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Solve Equivalently transform into . Then start the iteration from an initial guess and obtain the (convergent) sequence . Similar to the fixed-point iterations for solving f(x) = 0 …… Idea What to analyze? Chapter 7Iterative Techniques in Matrix Algebra The accuracy can be controlled by number of iterations. Iterative techniques are practically used to solve sparse linear systems of equations. How to design an iterative scheme? Under what conditions that the sequence will converge? How fast can a method converge? Error estimation? 1/19

  2. Chapter 7 Iterative Techniques in Matrix Algebra -- Norms of Vectors and Matrices Definition: A vector norm on Rn is a function, || · ||, from Rn into R with the following properties for all and  C: Some popularly used norms: n v  n = 2 v || x || | x |  = || x || | x | 2 i = 1 i 1 i 1 / p Note: n v v = 1 i  = = p || x || max | x | || x || | x | p i  i   1 i n = 1 i  7.1 Norms of Vectors and Matrices Vector Norms /* positive definite */ /* homogeneous */ /* triangle inequality */ Euclidean norm 2/19

  3. Chapter 7 Iterative Techniques in Matrix Algebra -- Norms of Vectors and Matrices Definition: A sequence of vectors in Rn is said to converge to with respect to the norm || · || if, given any  > 0, there exists an integer N() such thatfor all k N(). Theorem: The sequence of vectors converges to in Rn with respect to the norm || · || if and only if for each i = 1, 2, …, n. Definition: If there exist positive constants C1 and C2 such that , then || · ||A and || · ||B are said to be equivalent. Theorem: All the vector norms on Rn are equivalent. HW: Read the proof of Theorem 7.7 on p.423 3/19

  4. Chapter 7 Iterative Techniques in Matrix Algebra -- Norms of Vectors and Matrices Definition: A matrix norm on the set of all n  n matrices is a real-valued function, || · ||, defined on this set, satisfying for all n  n matrices A and B and all  C: /* positive definite */ /* homogeneous */ /* triangle inequality */ Matrix Norms (4)* || AB ||  || A || · || B ||/* consistent */ When you have to analyze the error bound of AB – imagine you doing it without a consistent matrix norm… Oh haven’t I had enough of new concepts? What do I need the consistency for? In general, if we have || AB || || A || · || B || , then the 3 norms are said to be consistent. 4/19

  5. Chapter 7 Iterative Techniques in Matrix Algebra -- Norms of Vectors and Matrices associated with the vector norm || · ||p Some popularly used norms: Frobenius Norm Natural Norm /* operator norm */ /* spectral norm */ 5/19

  6. Chapter 7 Iterative Techniques in Matrix Algebra -- Norms of Vectors and Matrices  Show that Excuses for not doing homework I have the proof, but there isn't room to write it in this margin. 1 Proof (for ):  Show that Let row p be the maximum row, that is Take a special unit vector such that HW: p.429-430 #5(a), 7, 13   6/19

  7. Chapter 7 Iterative Techniques in Matrix Algebra -- Eigenvalues and Eigenvectors For any eigenvalue  of A with eigenvector and         Im Definition: We call an n  n matrix Aconvergent if for all i, j = 1, 2, …, n we have Re  7.2 Eigenvalues and Eigenvectors Spectral Radius  (A) Definition: The spectral radius (A) of a matrix A is defined by  (A) = max |  | where  is an eigenvalue of A. Theorem: If A is an n  n matrix, then  (A)  || A || for any natural norm || · ||. Proof: HW: p.436 #3 7/19

  8. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems A = Tj  7.3 Iterative Techniques for Solving Linear Systems Jacobi Iterative Method In matrix form: – U – L D Jacobi iterative matrix 8/19

  9. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems Algorithm: Jacobi Iterative Method Solve given an initial approximation . Input: the number of equations and unknowns n; the matrix entries a[ ][ ]; the entries b[ ]; the initial approximation X0[ ]; tolerance TOL; maximum number of iterations Nmax. Output: approximate solution X[ ] or a message of failure. Step 1 Set k = 1; Step 2 While ( k  Nmax) do steps 3-6 Step 3 For i = 1, …, n Set ; /* compute xk */ Step 4 If then Output (X[ ]); STOP; /* successful */ Step 5 For i = 1, …, n Set X 0[ ] = X [ ]; /* update X0 */ Step 6 Set k ++; Step 7 Output (Maximum number of iterations exceeded); STOP. /* unsuccessful */ Since A will not be changed during the iterations, we can reorder the equations so that aii 0. Otherwise A is singular. A bit wasteful, isn’t it? What if aii= 0? X(k+1) must wait till all the entries of X(k) are obtained. Hence two vectors are needed to store the results. 9/19

  10. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems Tg Gauss - Seidel Iterative Method Only one vector needs to be saved. … … … … In matrix form: Gauss-Seidel iterative matrix 10/19

  11. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems A mathematician about his colleague: " He made a lot of mistakes, but he made them in a good direction. I tried to copy this, but I found out that it is very difficult to make good mistakes. " Note: Neither of the methods are always convergent. And more, there are cases in which Jacobi method fails while Gauss-Seidel is convergent, and vice-versa. See Exercises 9 and 10. 11/19

  12. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems Theorem: The following statements are equivalent: (1) A is a convergent matrix; (2) limn|| An || = 0 for some natural norm; (3) limn|| An || = 0 for all natural norms; (4) (A) < 1; (5) limnAn for every ？ Convergence of Iterative Methods Sufficient condition: ||T|| < 1 Necessary condition:  O 12/19

  13. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems Theorem: For any Rn, the sequence defined by for each k  1, converges to the unique solution of if and only if (T) < 1. (T) < 1  for any (T) < 1  Given that (T) < 1, then Proof: p.443  13/19

  14. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems Theorem: If || T || < 1 for any natural matrix norm and is a given vector, then the sequence defined by converges for any Rn, to a vector Rn. And the following error bounds hold: Theorem: If A is strictly diagonally dominant, then for any choice of both the Jacobi and Gauss-Seidel methods give sequences that converge to the unique solution of Proof (Hint): Simply prove that for any |  |  1 we have | I  T |  0. That is,  cannot be an eigenvalue of the corresponding iteration matrix T. 14/19

  15. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems Examine Gauss - Seidel method from another angle: where ri(k) = r (k) = + w (k) (k – 1) i x x i i a Let . For certain choice of positive  , we can reduce the the norm of the residual vector and obtain faster convergence. Such methods are called Relaxation Methods. ii 0 <  < 1 /* Under- Relaxation methods */  = 1 /* Gauss - Seidel */  > 1 /* Successive Over- Relaxation methods */ Relaxation Methods /* residual */ 15/19

  16. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems In matrix form: Oooooh come on! It’s way too complicated to compute T , and you can’t expect me to get its spectral radius right! There’s gotta be a short cut … 16/19

  17. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems Theorem: If A is positive definite and tridiagonal, then (Tg) = [(Tj)]2 < 1, and the optimal choice of  for the SOR method is With this choice of , we have Theorem: (Kahan) If aii 0 for each i = 1, 2, …, n. Then (T)  |  – 1 |. This implies that the SOR method can converge only if 0 <  < 2. Theorem: (Ostrowski-Reich) If A is positive definite and 0 <  < 2, the the SOR method converges for any choice of initial approximation. HW: p.453 #13 17/19

  18. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems Example: Given and an iterative method Then:  For what values of  that the method will converge?  For what values of  that the method will have the fastest convergence? 1 = 1+  , 2 = 1+ 3  Convergency requires  ( T )<1 -2/3 <  < 0  -2/3 -1/3 0 Solution: Consider the eigenvalues of T = I +  A  For what values of  that (T) = max {| 1+ |,| 1+ 3|} assumes its minimum?  = - 1/2 18/19

  19. Chapter 7 Iterative Techniques in Matrix Algebra -- Iterative Techniques for Solving Linear Systems Lab 04. Compare Methods of Jacobi with Gauss-Seidel Time Limit: 1 second; Points: 3 Use Jacobi and Gauss-Seidel methods to solve a given n×nlinear system with an initial approximation . Note: When checking each aii, first scan downward for the entry with maximum absolute value (aii included). If that entry is non-zero, swap it to the diagonal. Otherwise if that entry is zero, scan upward for the entry with maximum absolute value. If that entry is non-zero, then add that row to the i-th row. 19/19

More Related