Gaussian Elimination

1. Gaussian Elimination Computer Engineering Majors Author(s): Autar Kaw http://numericalmethods.eng.usf.edu Transforming Numerical Methods Education for STEM Undergraduates

2. Naïve Gauss Eliminationhttp://numericalmethods.eng.usf.edu

3. Naïve Gaussian Elimination A method to solve simultaneous linear equations of the form [A][X]=[C] Two steps 1. Forward Elimination 2. Back Substitution

4. Forward Elimination The goal of forward elimination is to transform the coefficient matrix into an upper triangular matrix

5. Forward Elimination A set of n equations and n unknowns . . . . . . (n-1) steps of forward elimination

6. Forward Elimination Step 1 For Equation 2, divide Equation 1 by and multiply by .

7. Forward Elimination Subtract the result from Equation 2. − _________________________________________________ or

8. Forward Elimination Repeat this procedure for the remaining equations to reduce the set of equations as . . . . . . . . . End of Step 1

9. Forward Elimination Step 2 Repeat the same procedure for the 3rd term of Equation 3. . . . . . . End of Step 2

10. Forward Elimination At the end of (n-1) Forward Elimination steps, the system of equations will look like . . . . . . End of Step (n-1)

11. Matrix Form at End of Forward Elimination

12. Back Substitution Solve each equation starting from the last equation Example of a system of 3 equations

13. Back Substitution Starting Eqns . . . . . .

14. Back Substitution Start with the last equation because it has only one unknown

15. Back Substitution

16. THE END http://numericalmethods.eng.usf.edu

17. Naïve Gauss EliminationExamplehttp://numericalmethods.eng.usf.edu

18. Example: Surface Shape Detection To infer the surface shape of an object from images taken of a surface from three different directions, one needs to solve the following set of equations The right hand side are the light intensities from the middle of the images, while the coefficient matrix is dependent on the light source directions with respect to the camera. The unknowns are the incident intensities that will determine the shape of the object.

19. Example: Surface Shape Detection The solution for the unknowns x1, x2, and x3 is given by Find the values of x1, x2,and x3 using Naïve Gauss Elimination.

20. Example: Surface Shape Detection Forward Elimination: Step 1 Yields

21. Example: Surface Shape Detection Forward Elimination: Step 1 Yields

22. Example: Surface Shape Detection Forward Elimination: Step 2 Yields This is now ready for Back Substitution

23. Example: Surface Shape Detection Back Substitution: Solve for x3 using the third equation

24. Example: Surface Shape Detection Back Substitution: Solve for x2 using the second equation

25. Example: Surface Shape Detection Back Substitution:Solve for x1 using the first equation

26. Example: Surface Shape Detection Solution: The solution vector is

27. THE END http://numericalmethods.eng.usf.edu

28. Naïve Gauss EliminationPitfallshttp://numericalmethods.eng.usf.edu

29. Pitfall#1. Division by zero

30. Is division by zero an issue here?

31. Is division by zero an issue here? YES Division by zero is a possibility at any step of forward elimination

32. Pitfall#2. Large Round-off Errors Exact Solution

33. Pitfall#2. Large Round-off Errors Solve it on a computer using 6 significant digits with chopping

34. Pitfall#2. Large Round-off Errors Solve it on a computer using 5 significant digits with chopping Is there a way to reduce the round off error?

35. Avoiding Pitfalls • Increase the number of significant digits • Decreases round-off error • Does not avoid division by zero

36. Avoiding Pitfalls • Gaussian Elimination with Partial Pivoting • Avoids division by zero • Reduces round off error

37. THE END http://numericalmethods.eng.usf.edu

38. Gauss Elimination with Partial Pivotinghttp://numericalmethods.eng.usf.edu

39. Pitfalls of Naïve Gauss Elimination • Possible division by zero • Large round-off errors

40. Avoiding Pitfalls • Increase the number of significant digits • Decreases round-off error • Does not avoid division by zero

41. Avoiding Pitfalls • Gaussian Elimination with Partial Pivoting • Avoids division by zero • Reduces round off error

42. What is Different About Partial Pivoting? At the beginning of the kth step of forward elimination, find the maximum of If the maximum of the values is in the p th row, then switch rows p and k.

43. Matrix Form at Beginning of 2nd Step of Forward Elimination

44. Example (2nd step of FE) Which two rows would you switch?

45. Example (2nd step of FE) Switched Rows

46. Gaussian Elimination with Partial Pivoting A method to solve simultaneous linear equations of the form [A][X]=[C] Two steps 1. Forward Elimination 2. Back Substitution

47. Forward Elimination Same as naïve Gauss elimination method except that we switch rows before each of the (n-1) steps of forward elimination.

48. Example: Matrix Form at Beginning of 2nd Step of Forward Elimination

49. Matrix Form at End of Forward Elimination

50. Back Substitution Starting Eqns . . . . . .