Bruce Mayer, PE Licensed Electrical & Mechanical Engineer BMayer@ChabotCollege

1. Engr/Math/Physics 25 Chp8 LinearAlgebraic Eqns-2 Bruce Mayer, PE Licensed Electrical & Mechanical EngineerBMayer@ChabotCollege.edu

2. Learning Goals • Define Linear Algebraic Equations • Solve Systems of Linear Equation by Hand using • Gaussian Elimination • Cramer’s Method • Distinguish between Equation System Conditions: Exactly Determined, Overdetermined, Underdetermined • Use MATLAB to Solve Systems of Eqns

3. Cramer’s Method for Eqn Sys • Solves equations using determinants. • Gives insight into the • existence and uniqueness of solutions • Identifies SINGULAR (a.k.a. Divide by Zero) Systems • effects of numerical inaccuracy • Identifies ILL-CONDITIONED (a.k.a. Stiff) Systems

4. Cramer’s Method can Solve “Square” Systems; i.e., [No. Eqns] = [No. Unknowns] Consider Sq Sys Cramer’s Method – Illustrated-1 • Calc Cramer’s Determinant, Dc • Also Called the “Characteristic” or “Denominator” Determinant • Dc Determinant of the Coefficients

5. Now, to Find The Individual Solns, Sub The Constraint Vector for the Variable Coefficients and Compute the Determinant for Each unknown, Dk In this Example Find Dx, Dy, Dz as Cramer’s Method – Illustrated-2

6. Once We’ve Calculated all these Determinants, The Rest is Easy Cramer’s Method – Illustrated-3 • These Eqns Ilustrate the most UseFul Feature of Cramer’s Method Dc appears in all THREE Denominators

7. Since Cramer’s Method – Illustrated-4 • However, We can ANTICIPATE Problems if |Dc| << than the SMALLEST Coefficient • Completing the Example • Can ID “Condition” by Calculating Dc • SINGULAR Systems → Dc = 0 • ILL-CONDITIONED Systems → Dc = “Small” • Small is technically relative to the Dk

8. Calc the Determinants First Recall The SIGN pattern for Determinants Cramer’s Method – Illustrated-6 • Dc is LARGE → WELL Conditioned System • Find Dc

9. Solve using MATLAB’s det Function Cramer’s Method – Illustrated-7 >> EqnSys = [21 -9 -12 -33; -3 6 -2 3; -8 -4 22 50]; >> Dc = det(EqnSys(:,1:3)) Dc = 1146 >> Dx = det([EqnSys(:,4),EqnSys(:,2:3)]) Dx = 1146 >> Dy = det([EqnSys(:,1),EqnSys(:,4), EqnSys(:,3)]) Dy = 2292 All Row Elements of Cols 1-3 All Row Elements of Cols 4, 2-3 All Row Elements of Cols 1, 4, 3

10. Solve using MATLAB’s det Function Cramer’s Method – Illustrated-8 >> Dz = det([EqnSys(:,1:2),EqnSys(:,4)]) Dz = 3438 >> x = Dx/Dc x = 1 >> y = Dy/Dc y = 2 >> z = Dz/Dc z = 3 All Row Elements of Cols 1-2, 4

11. Cramer vs Homogenous: • In general, for a set of HOMOGENEOUS linear algebraic equations that contains the same number of equations as unknowns • a nonzero solution exists only if the set is singular; that is, if Cramer’s determinant is zero • furthermore, the solution is not unique. • If Cramer’s determinant is not zero, the homogeneous set has a zero solution; that is, all the unknowns are zero

12. Cramer’s Rule Summary • Cramer’s determinant gives some insight into ill-conditioned problems, which are close to being singular. • A Cramer’s determinant close to zero indicates an ill-conditioned problem.

13. UnderDetermined Systems • An UNDERdetermined system does not contain enough information to solve for ALL of the unknown variables • Usually because it has fewer equations than unknowns. • In this case an INFINITE number of solutions can exist, with one or more of the unknowns dependent on the remaining unknowns. • For such systems the Matrix-Inverse and Cramer’s methods will NOT work

14. UnderDetermined Example-1 • A simple UnderDetermined system is the equation • All we can do is solve for one of the unknowns in terms of the other; for example, x = 6 –3y OR y = −x/3 + 2 • An INFINITE number of (x,y) solutions satisfy this equation

15. UnderDetermined Example-2 • When there are more Unknowns than Equations, the LEFT-DIVISION method will give a solution with some of the unknowns set equal to ZERO • For Example >>A = [1, 3]; b = 6; >>solution = A\b solution = 0 2 • which corresponds to • x = 0 • y = 2

16. More UnderDetermined Systems • An infinite number of solutions might exist EVEN when the number of equations EQUALS the number of knowns • Predict by Cramer as: • For such systems the Matrix Inverse method and Cramer’s method will also NOT work • MATLAB’s left-division method generates an error message warning us that the matrix A is singular

17. Minimum Norm Solution • When det(A) = 0, We can use the PSEUDOINVERSE method to find ONE Solution, x, such that the Euclidean (or Pythagorean) Length of x is MINIMIZED • In MATLAB: x = pinv(A)*b • MATLAB will return the MINIMUM NORM SOLUTION →

18. Existence and Uniqueness • The set Ax = b with m equations and n unknowns has solutions if and only if • Rank[A] is the maximum number of LINEARLY INDEPENDENT rows of A • Linear Independence → No Row of Ais a SCALAR multiple of ANY OTHER Combinations of Rows

19. Consider Ax = b An INconsistent Example • Since The Ranks are Unequal → this system of equations is NOT solvable • Graphically • ERO: Multiply the 1st row by −2 and add to the 2nd row Rank[A]=1 Rank[Ab]=2

20. Existence and Uniqueness • Recall Rank for m-Eqns & n-Unknwns • Now Let r = rank[A] • If condition (1) is satisfied and if r = n, then the solution is unique • If condition (1) is satisfied but r < n, an infinite number of solutions exists and • r unknown variables can be expressed as linear combinations of the other n−r unknown variables, whose values are ARBITRARY

21. Homogenous Case • The homogeneous set Ax = 0 is a special case in which b = 0 • For this case rank[A] = rank[Ab] always, and thus the system in all cases has the trivial solution x = 0 • A nonzero solution, in which at least one unknown is nonzero, exists if and only if rank[A] < n (n  No. Unknowns) • If m < n, the homogeneous set always has a nonzero solution

22. ILLconditioned Systems • An ill-conditioned set of equations is a set that is CLOSE to being SINGULAR • The ill-conditioned status depends on the ACCURACY with which the solution calculations are made • When the internal numerical accuracy used by MATLAB is INsufficient to obtain a solution, then MATLAB prints a message to warn you that the matrix is close to singular and that the results might be INaccurate

23. OVERdetermined Systems - 1 • By Contrast an OVERdetermined system is a set of equations that has MORE independent equations than unknowns • For such a system the Matrix Inverse method and Cramer’s method will NOT work because the A matrix is not square • However, SOME overdetermined systems have exact solutions, and they can be obtained with the left division method x = A\b

24. OVERdetermined Systems - 2 • For OTHER OverDetermined systems, NO exact solution exists. • In some of these cases, the left-division method does not yield an answer, while in other cases the left-division method gives an answer that satisfies the equation set only in a “LEAST SQUARES” sense • When MATLAB gives an answer to an overdetermined set, it does NOT tell us whether the answer is Exact or Least-Squares in Nature • We need to check the ranks of A and [Ab] to know whether the answer is the exact solution.

25. Example  Under/Over Determined % Bruce Mayer, PE * 27Feb08 % ENGR25 * Under/OverDetermined Linear Systems % file = Over_Under_Lin_Sys_Example_0904.m % Solve Under & Over Determined Systems for [x1; x2; x3; x4] % % An UNDERdetermined case => 3-Eqns in 4-Unknwns Au = [2 -3 7 4; -9 2 -2 -7; 7 6 -5 -4] bu = [5; -9; 7] disp('Showing UNDERdetermined solution & check - Hit Any Key to continue') xu = Au\bu bu_chk = Au*xu pause % % An OVERdetermined case => 5-Eqns in 4-Unknwns Ao = [2 -3 7 4; -9 2 -2 -7; 7 6 -5 -4; 1 9 0 -6; 2 -5 -8 1] bo = [5; -9; 7; 2; -6] disp('Showing OVERdetermined solution & check') xo = Ao\bo bo_chk= Ao*xo Time For Live Demo

26. Uniqueness of solutions • A system has a unique solution If, and Only If (IFF): Rank[A]=Rank[Ab] = n • Where n is the ORDER of the system; i.e., the Number of UNKNOWNS • Such systems are called full-rank systems

27. OverDetermined Rank • To interpret MATLAB answers correctly for an OVERdetermined system, first check the ranks of [A] and [Ab] to see whether an exact solution exists  r[Ab] = r[A] = n (the no. of Unknowns) • If r[Ab] = r[A] > n then a unique soln does not exist, then you know that the left-division answer is a least squares solution • Can easily Chk: Axsoln = b (exactly)? • This is at least as easy as the rankcalc

28. Solving Linear Eqns Summary - 1 • If the number of equations (m) in the set equals the number of unknown variables (n), the matrix A is square and MATLAB provides two ways of solving the equation set Ax = b: • The matrix inverse method; solve for x by typing x = inv(A)*b. • The matrix left-division method; solve for x by typing x = A\b.

29. Solving Linear Eqns Summary - 2 • If A is square and if MATLAB does not generate an error message when you use one of these methods, then the set has a unique solution, which is given by the left-division method. • You can always CHECK the solution for x by typing A*x to see if the result is the same as b. • Use this to avoid the rankcalcs

30. Solving Linear Eqns Summary - 3 • If when you type A*x you receive an error message, the set is UNDERdetermined, and either it does not have a solution or it has more than one solution • In such a case, if you need more information, you must use the following procedures.

31. Solving Linear Eqns Summary - 4 • For UNDERdetermined and OVERdetermined sets, MATLAB provides three ways of dealing with the equation set Ax = b. (Note that the matrix inverse method will NEVER work with such sets.) • The matrix left-division method; solve for x by typing x = A\b • The pseudoinverse method; solve for x by typing x = pinv(A)*b • the Reduced Row Echelon Form (RREF) method. This method uses the MATLAB function rref to obtain a solution.

32. Solving Linear Eqns Summary - 5 • UNDERdetermined Systems • In an underdetermined system NOT enough information is given to determine the values of all the unknown variables. • An INFINITE number of solutions might exist in which one or more of the unknowns are dependent on the remaining unknowns • For such systems Cramer’s method and the matrix inverse method will not work because either A is not square or because |A| = 0.

33. Solving Linear Eqns Summary - 6 • Underdetermined Systems cont • The left-division method will give a solution with some of the unknowns arbitrarily set equal to zero, but this solution is NOT the general solution; it’s just ONE solution (of Trillions) • An infinite number of solutions might exist even when the number of equations equals the number of unknowns. The left-division method fails to give a solution in such cases • In cases that have an infinite number of solutions, some of the unknowns can be expressed in terms of the remaining unknowns, whose values are arbitrary. The rref function can be used to find these relations.

34. Solving Linear Eqns Summary - 7 • OVERdetermined Systems • An overdetermined system is a set of equations that has more independent equations than unknowns • For such a system Cramer’s method and the matrix inverse method will not work because the A matrix is NOT SQUARE • Some overdetermined systems have exact solutions, which can be obtained with the left-division method A\b.

35. Solving Linear Eqns Summary - 8 • OVERdetermined systems cont. • For overdetermined systems that have no exact solution, the estimated-answer given by the left-division method satisfies the equation set only in a LEAST SQUARES sense • When we use MATLAB to solve an overdetermined set, the program does not tell us whether the solution is exact. We must determine this information ourselves. The first step is to check the ranks of A and [Ab] to see whether a solution exists; if no solution exists, then we know that the left-division solution is a least squares answer.

36. FlowChart for LinSys Solution Can Construct Least-Squares Approximation

37. Descison Tree for LinSys Solns if Rank(A)==Rank(Ab) & Rank(A)==NoUnknowns disp('EXACT Solution') elseifRank(A)==Rank(Ab) & Rank(A)<NoUnknowns disp('INFINITE Solutions') elseifRank(Ab) > NoUnknowns disp('LEAST SQS Approximation') else disp('NOsolution; can least-sqs est.') end

38. All Done for Today GabrielCramer Born: 31 July 1704 in Geneva, Switzerland Died: 4 Jan 1752 in Bagnols-sur-Cèze, France

39. Engr/Math/Physics 25 Appendix Time For Live Demo Bruce Mayer, PE Licensed Electrical & Mechanical EngineerBMayer@ChabotCollege.edu

40. MATLAB LinSys Solver Code - 1 % Script file lineq.m % Solves the set Ax = b, given A and b. % Check the ranks of A and [A b]. if rank(A) == rank([A b]) % Then The ranks are equal. Size_A = size(A); % Does the rank of A equal the number of unknowns? if rank(A) == size_A(2) % Then the Rank of A equals the number of unknowns. disp(’There is a unique solution, which is:’) x = A\b % Solve using left division.

41. MATLAB LinSys Solver Code - 2 else % Rank of A does not equal the number of unknowns. disp(’There is an infinite number of solutions.’) disp(’The augmented matrix of the reduced system is:’) rref([A b]) % Compute the augmented matrix. end else % The ranks of A and [A b] are not equal. disp(’There are no solutions.’) end

42. MATLAB rank • rank • Rank of matrix • Syntax • k = rank(A) • Description • The rank function provides an estimate of the number of linearly independent rows or columns of a full matrix.