Lecture 6 Matrix Operations and Gaussian Elimination for Solving Linear Systems

1. Lecture 6Matrix Operationsand Gaussian Elimination for Solving Linear Systems Shang-Hua Teng

2. Matrix(Uniform Representation for Any Dimension) • An m by n matrix is a rectangular table of mn numbers

3. Matrix(Uniform Representation for Any Dimension) • Can be viewed as m row vectors in n dimensions

4. Matrix(Uniform Representation for Any Dimension) • Or can be viewed as n column vectors in m dimensions

5. Squared Matrix • An n by n matrix is a squared table of n2 numbers

6. Some Special Squared Matrices • All zeros matrix • Identity matrix

7. Matrix Operations • Addition • Scalar multiplication • Multiplication

8. Matrix Addition: Matrices have to have the same dimensions What is the complexity?

9. 2. Scalar Multiplication: What is the complexity?

10. 3. Matrix Multiplication Two matrices have to be conformal What is the complexity?

11. Matrix Multiplication Two matrices have to be conformal

12. The Laws of Matrix Operations • A + B = B + A (commutative) • c(A+B) = cA + c+B (distributive) • A + (B + C) = (A + B) + C (associative) • C(A+B) = CA + CB (distributive from left) • (A+B)C = AC+BC (distributive from right) • A(BC) = (AB)C (associative) • But in general:

13. Counter Example

14. Special Matrices • Identity matrix I • IA = AI = A • Square Matrix A

15. Elimination: Method for Solving Linear Systems • Linear Systems == System of Linear Equations • Elimination: • Multiply the LHS and RHS of an equation by a nonzero constant results the same equations • Adding the LHSs and RHSs of two equations does not change the solution

16. Elimination in 2D • Multiply the first equation by 3 and subtracts from the second equation (to eliminate x) • The two systems have the same solution • The second system is easy to solve

17. (3,1) 8y = 8 Geometry of Elimination Reduce to a 1-dimensional problem.

18. Upper Triangular Systems and Back Substitution • Back substitution • From the second equation y = 1 • Substitute the value of y to the first equation to obtain x-2=1 • Solve it we have: x = 3 • So the solution is (3,1)

19. How Much to Multiply before Subtracting • Pivot: first nonzero in the row that does the elimination • Multiplier: (entry to eliminate) divided by (pivot) Multiply: = 3/1

20. How Much to Multiply before Subtracting • Pivot: first nonzero in the row that does the elimination • Multiplier: (entry to eliminate) divided by (pivot) The pivots are on the diagonal of the triangle after the elimination Multiply: = 3/2

21. Breakdown of Elimination • What is the pivot is zero == one can’t divide by zero!!!! Eliminate x: No Solution!!!!: this system has no second pivot

22. Geometric Intuition(Row Pictures) • Two parallel lines never intersect (3,1) 8y = 8

23. Geometric Intuition(Column Picture) Two column vectors are co-linear!!!!

24. Geometric Intuition Geometric degeneracy cause failure in elimination!

25. Failure in Elimination May Indicate Infinitely Many Solutions • y is free, can be number! • Geometric Intuition (row picture): The two line are the same • Geometric Intuition (column picture): all three column vectors are co-linear

26. Failure in Elimination(Temporary and can be Fixed) • First pivot position contains zero • Exchange with the second equation Can be solved by backward substitution!

27. Singular Systems versus Non-Singular Systems • A singular system has no solution or infinitely many solution • Row Picture: two line are parallel or the same • Column Picture: Two column vectors are co-linear • A non-singular system has a unique solution • Row Picture: two non-parallel lines • Column Picture: two non-colinear column vectors

28. Gaussian Elimination in 3D • Using the first pivot to eliminate x from the next two equations

29. Gaussian Elimination in 3D • Using the second pivot to eliminate y from the third equation

30. Gaussian Elimination in 3D • Using the second pivot to eliminate y from the third equation

31. Now We Have a Triangular System • From the last equation, we have

32. Backward Substitution • And substitute z to the first two equations

33. Backward Substitution • We can solve y

34. Backward Substitution • Substitute to the first equation

35. Backward Substitution • We can solve the first equation

36. Backward Substitution • We can solve the first equation

37. Generalization • How to generalize to higher dimensions? • What is the complexity of the algorithm? • Answer: • Express Elimination with Matrices

38. Step 1Build Augmented Matrix Ax = b [A b]

39. Pivot 1: The elimination of column 1

40. Pivot 2: The elimination of column 2 Upper triangular matrix

41. Backward Substitution 1: from the last column to the first Upper triangular matrix

42. Expressing Elimination by Matrix Multiplication

43. Elementary or Elimination Matrix • The elementary or elimination matrix That subtracts a multiple l of row j from row i can be obtained from the identity entry by adding (-l) in the i,j position

44. Elementary or Elimination Matrix

45. Pivot 1: The elimination of column 1 Elimination matrix

46. The Product of Elimination Matrices