G-inverse and Solution of System of Equations and their Applications in Statistics

1. G-inverse and Solution of System of Equations and their Applications in Statistics Professor Dr. S. K. Bhattacharjee Department of Statistics University of Rajshahi, Bangladesh

2. System of Linear Equations

3. Linear Combination of Vectors

4. Vector Form

5. Solution of Linear System • A solution of a linear system is an assignment of values to the variables x1, x2, ..., xn such that each of the equations is satisfied. The set of all possible solutions is called the solution set. • A linear system may behave in any one of three possible ways: • The system has infinitely many solutions. • The system has a single unique solution. • The system has no solution.

6. Geometric Interpretation • For a system involving two variables (x and y), each linear equation determines a line on the xy-plane. Because a solution to a linear system must satisfy all of the equations, the solution set is the intersection of these lines, and is hence either a line, a single point, or the empty set. • For three variables, each linear equation determines a plane in three-dimensional space, and the solution set is the intersection of these planes. Thus the solution set may be a plane, a line, a single point, or the empty set. • For n variables, each linear equations determines a hyperplane in n-dimensional space. The solution set is the intersection of these hyperplanes, which may be a flat of any dimension

7. Solution of Two Vectors

8. Solution of Three Equations

9. Solution of Three Equations

10. Solution of Three Equations

11. Solution of Three Equations

12. Solution of Two Equations in Three Variables

13. Solution of Linear System • In general, the behavior of a linear system is determined by the relationship between the number of equations and the number of unknowns: • Usually, a system with fewer equations than unknowns has infinitely many solutions. Such a system is also known as an underdetermined system. • Usually, a system with the same number of equations and unknowns has a single unique solution. • Usually, a system with more equations than unknowns has no solution. Such a system is also known as an overdetermined system.

14. Examples • The system has exactly 1 solution. • Systems have 1 and only 1 solution when the two lines have different slope. Think about it, if the two lines have different slopes then eventually at some point they must meet. After all the lines are not parallel. • system has no solutions • Systems have no solution when the lines are parallel (ie have the same slope) and the lines have different y-intercepts. • As an example look at the following two lines • Line 1: y = 5x +13 • Line 2: y = 5x + 12 • The system has infinite solutions • Systems have infinite solutions when the lines are parallel and the lines have the same y-intercept. If two lines have the same slope (ie are parallel) and the same y-intercept, they are actually the same exact line. In other words, systems have infinite solutions when the two lines are the same line! • As an example consider the following two lines • Line 1: y = x +3 • Line 2: 2y = 2x +6 • These two lines are exactly the same line. If you multiply line 1 by two you get line 2.

15. Solution of Linear System

16. Two Variables Three Eq.

17. Example The equations 3x + 2y = 6 and 3x + 2y = 12 are inconsistent.

18. Methods of Solution • The Methods of finding the solution to systems of linear equations: • graph : by looking at where lines intersect (meet) on a graph • algebraic equation : by setting the equations of the system equal to each other then solving this equation. • substitution : by solving for one of the variables and substituting its value in to the other equation. • Elimination : Elimination involves algebraic manipulations of two or more equations. The end goal is to eliminate a variable by creating opposite coefficients (The examples below should clarify this straightforward approach).

19. Graphical Method

20. Algebraic Equation Method

21. Substitution Method • The Substitution Method • The substitution method involves algebraic substitution of one equation into a variable of the other. • A quick refresher on algebraic substitution: Refresher:Substitution • Equation 1 : x = 5 • Equation 2: y = x +2 • How to Substitute • 1) Use equation 1( x= 5) to substitute 5 for x in second equation • y = (5) + 2 • 2) So love for Y • y = 5+ 2 = 7

22. Example • Substitution Example Two • Line 1 : y=2x+1 • Line 2 : 2y=3x-2 • Step 1: Substitute one equation into the other • 2(2x+1)=3x-2 • Step 2: Now that you have a single variable equation, solve for that variable's equation • 4x+2 = 3x-2 • x+2= -2 • x= –4 • Step 3 : Once you have solved for the one variable insert that variable back into either equation to obtain the value of y at the solution. • Insert x= –4 to find y value y = 2(–4)+1= –7 • This example's solution is ( –4, –7).

23. Elimination Method • Elimination method is an algebraic method for solving systems. To use elimination you perform an operation on 1 equation then add the two equations so that one of the variables cancels. • Example of Elimination • Line 1: y = x + 1 • Line 2: y = –x

24. Elimination Method

25. Matrix Methods of Solution

26. Inverse Method • The Matrix form of a linear system of equations is Ax= b If the coefficient matrix is non-singular then the solution is: x = A-1b

27. Elementary Row Transformation Solving a system of linear equations by reducing the augmented matrix of the system to row canonical form

28. Example

29. Example

30. Homogeneous System

31. Homogeneous System • Homogeneous Systems are always Consistent • Every homogeneous system has at least one solution, known as the zero solution (or trivial solution), which is obtained by assigning the value of zero to each of the variables. The solution set has the following additional properties: • If u and v are two vectors representing solutions to a homogeneous system, then the vector sum u + v is also a solution to the system. • If u is a vector representing a solution to a homogeneous system, and r is any scalar, then ru is also a solution to the system.

32. Homogeneous System • Suppose that a homogeneous system of linear equations has m equations and n variables with n>m. Then the system has infinitely many solutions.

33. Example

34. Example

35. Example

36. Generalized Inverse

37. Generalized Inverse

38. Generalized Inverse

39. Method of Obtaining g-Inverse

40. Example

41. Method of Obtaining g- Inverse

42. Example

43. Properties of g Inverse of X’X The matrix X’X has an important role in statistics where it arises in least square equations X’Xb = X’y. When G is a generalized inverse of X’X: • G’ is also a generalized inverse of X’X • XGX’X=X, i.e., GX’ is a generalized inverse of X • XGX’ is invariant to G • XGX’ is symmetric whether G is or not. Let X+ be the Moore – Penrose inverse of X. Then XX+ = XGX’ but X+ may not be equal to GX’.

44. g-Inverse

45. g-Inverse

46. Properties • Every singular symmetric matrix (of order two or more) has both symmetric and non-symmetric generalized inverses. • Let A represent a matrix of full column rank and B a matrix of • full row rank. Then, (1) a matrix G is a generalized inverse of A if and only if G • is a left inverse of A. And, (2) a matrix G is a generalized inverse of B if and only • if G is a right inverse of B.

47. Properties • For any matrix A and any nonzero scalar k, (1/k)A− is a generalized inverse of kA. • For any matrix A,−A− is a generalized inverse of −A. • For any matrix A, (A−)’ is a generalized inverse of A’ . • For any symmetric matrix A, (A−)’ is a generalized inverse of A’ .

48. Properties • A linear system Ax = b is consistent if and only if • AA- b = b • or, equivalently, if and only if • (I − AA-)B = 0

49. The Moore-Penrose Inverse Given any matrix A, there is a unique matrix M such that (i) AMA=A (ii) MAM=M (iii) AM is symmetric (iv) MA is symmetric Then matrix M is called the Moore-Penrose inverse of A. M = L’(K’AL’)-1 K’

50. The Moore-Penrose Inverse