Tutorial 2-3. Contents

1. Tutorial 2-3. Contents • Solution of Systems of Linear Equations • Gaussian Elimination • LU Decomposition • Gram-Schmidt vector orthogonalization Tutorial 2-3

2. Linear Systems in Matrix Form (1)  Linear equation means that every variable is in the power 1 only. If the variables in the equation are ordered x1, … xn and the missing variables xi are written as … + 0·xi + … , then the linear equation can be written in the matrix notation, according to the matrix multiplication rules. Solution of linear equations was one of the reasons, for matrix notation invention. Tutorial 2-3

3. An inconsistent example: Geometric interpretation ERO:Multiply the first row with -2 and add to the second row Rank{A}=1 Rank{A|b}=2≠ Rank{A} Tutorial 2-3

4. Full-rank systems • If Rank{A}=n Det{A}  0  A-1 exists  Unique solution Tutorial 2-3

5. Pivotal element First step of elimination Tutorial 2-3

6. Pivotal element Second step of elimination Tutorial 2-3

7. Back substitution algorithm Tutorial 2-3

8. LU Decomposition A=LU Ax=b LUx=b Define Ux=y Ly=b Solve y by forward substitution ERO’s must be performed on b as well as A The information about the ERO’s are stored in L Indeed y is obtained by applying ERO’s to b vector Ux=y Solve x by backward substitution Tutorial 2-3

9. LU Decomposition by Gaussian elimination There are infinitely many different ways to decompose A. Most popular one: U=Gaussian eliminated matrix L=Multipliers used for elimination Compact storage: The diagonal entries of L matrix are all 1’s, they don’t need to be stored. LU is stored in a single matrix. Tutorial 2-3

10. An example: Linear System Rank{A}=n  A-1 exists  Unique solution Tutorial 2-3

11. Gaussian Elimination x3=3; -x2-2x3=-8; x2=2; x1+2*2+3*3=14; x1=14-4-9=1. Tutorial 2-3

12. LU Factorization L-1 U=L-1A L-1 A U Check that there were no mistakes: Tutorial 2-3

13. LU Factorization Check that there were no mistakes: L-1 A U x3=3; -x2-2x3=-8; x2=2; x1+2*2+3*3=14; x1=14-4-9=1. L-1 b L-1b Tutorial 2-3

14. the Gram-Schmidt procedure for vector orthogonalization • The purpose we can construct a set of orthonormal vectors uifrom a set of n-dimensional vectors vi. 1≤i≤m • And vican be represented by the linear combination of ui Tutorial 2-3

15. G-S procedure (cont.) • Select a vector from vi arbitrarily, say v1 • By normalizing its length, we obtain the first vector, say • Select v2 and subtract the projection of v2 ontou1, we get w2=v2– (v2•u1)u1 Tutorial 2-3

16. G-S procedure (cont.) • And normalizing • Now we can check, that Tutorial 2-3

17. G-S procedure (cont.) • The procedure continues by selecting v3 and subtract its projection on u1 and u2, we have w3=v3– (v3•u1)u1– (v3• u2)u2 7. Then, the orthonormal vector u3 is 8. By continuing this procedure, we shall construct the set of orthonormal vectors ui Tutorial 2-3

18. G-S procedure: Example Consider v1=(0,1,1); v2=(1,-1,0); v3=(1,0,1) They are not orthogonal: (vi,vj)≠0. Following G-S: Tutorial 2-3

19. Example (cont.) We can check, that: Tutorial 2-3

20. Example (cont.) Why? Tutorial 2-3