1 / 52

INDR 262 INTRODUCTION TO O PTIMIZATION METHODS

INDR 262 INTRODUCTION TO O PTIMIZATION METHODS. LINEAR ALGEBRA. Matrices. m and n are positive integers Order of matrix : m x n The number in the i th row and j th column of A is called the ij th element of A and is written a ij . Example. Equal Matrices .

morton
Download Presentation

INDR 262 INTRODUCTION TO O PTIMIZATION METHODS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. INDR 262INTRODUCTION TO OPTIMIZATION METHODS LINEAR ALGEBRA INDR 262MetinTürkay1

  2. Matrices • mandnarepositiveintegers • Order of matrix: mxn • The number in the ith row and jth column of A is called the ijth element of A and is written aij. INDR 262MetinTürkay2

  3. Example INDR 262MetinTürkay3

  4. Equal Matrices • Two matrices A and B are equal if and only if A and B are of the same order and for all i and j, aij=bij. • If A=B, then x=1, y=2, w=3, z=4 INDR 262MetinTürkay4

  5. Vectors • A list of n real numbers, say (a1, a2, …, an) is called an n-dimensional vector. An n-dimensional vector also maybe displayed as a 1 by n matrix. INDR 262MetinTürkay5

  6. Scalar Product of Two Vectors • The scalar product of vectors u = [u1 u2 … un] and is u1v1+u2v2+…+unvn INDR 262MetinTürkay6

  7. Example • uv = 1x2+2x1+3x2 = 10 INDR 262MetinTürkay7

  8. Notes • If u = [1 2 3] and , then uv is not defined because the vectors are of different dimensions. • Two vectors are perpendicular to each other if and only if their scalar product is equal to 0. • E.g., u = [1 -1] and. INDR 262MetinTürkay8

  9. Scalar Multiple of a Matrix • Given any matrix A and any scalar c, the scalar multiple of matrix A, cA, is obtained from the matrix A by multiplying each element of A by c. INDR 262MetinTürkay9

  10. Addition of two Matrices • Let A=[aij] and B=[bij] be two matrices with the same order (say mxn). Then, the matrix C=A+B is defined to be the mxn matrix whose ijth element is aij+bij. INDR 262MetinTürkay10

  11. The Transpose of a Matrix • Given any mxnmatrix the transpose of A (written AT) is the nxmmatrix INDR 262MetinTürkay11

  12. Example • For any matrix A, (AT)T=A. INDR 262MetinTürkay12

  13. Matrix Multiplication • Given two matrices A and B, the matrix product of A and B is defined if and only if the number of columns in A is equal to the number of rows in B. The matrix product C=AB is determined as follows: cij = scalarproduct of (rowi of Aandcolumnj of B) INDR 262MetinTürkay13

  14. Properties of Matrix Multiplication • Matrix multiplication is associative, i.e., A(BC)=(AB)C. • Matrix multiplication is distributive, i.e., A(B+C)=AB+AC. INDR 262MetinTürkay14

  15. Example INDR 262MetinTürkay15

  16. Matrices and Systems of Linear Equations • Consider a system of linear equations given by • x1, x2, …, xn are referred to as variables • aij’s and bi’s are constants • A set of equations like above is called a linear system of m equations in n variables INDR 262MetinTürkay16

  17. Solution • A solution to a linear system of m equations in n unknowns is a set of values for the unknowns that satisfy each of the systems m equations. INDR 262MetinTürkay17

  18. Example x1+2x2 = 5 2x1 - x2= 0 Solution Not a solution INDR 262MetinTürkay18

  19. Matrix Representation of Systems of Linear Equations Ax = b Augmented matrix INDR 262MetinTürkay19

  20. The Gauss-Jordan Method for Solving Systems of Linear Equations • Gauss-Jordan method is used to find solution(s) to systems of linear equations. A system of linear equations must satisfy one of the following cases: • Case 1: The system has no solution • Case 2:The system has a unique solution • Case 3:The system has an infinite number of solutions INDR 262MetinTürkay20

  21. Elementary Row Operations • An ero transforms a given matrix A into a new matrix A’ via one of the following operations. • Type 1 ero:A’ is obtained by multiplying any row of A by a nonzero scalar. • Type 2 ero:Begin by multiplying any row of A (say, row i) by a nonzero scalar c. For some ji, let row j of A’= c(row i of A) + row j of A, and let the other rows of A’ be the same as the rows of A. • Type 3 ero:Interchange any two rows of A. INDR 262MetinTürkay21

  22. Facts • If the matrix A’ is obtained from A via an ero, A’ and A are equivalent. • If the augmented matrix [A’b’] is obtained from [Ab] via an ero, the systems Ax=b and A’x=b’ are equivalent. • Any sequence of ero’s performed on the augmented matrix [Ab] corresponding to the system Ax=b will yield an equivalent linear system. INDR 262MetinTürkay22

  23. Gauss-Jordan Method • The Gauss-Jordan method solves a linear system of equations by utilizing ero’s in a systematic fashion. Step 1 To solve Ax=b, write down the augmented matrix [Ab]. Step 2 At any stage, define a current row, current column, and a current entry. Begin with row 1 as the current row, column 1 as the current column, and a11 as the current entry. INDR 262MetinTürkay23

  24. Gauss-Jordan Method a If a11 (the current entry) is nonzero, use ero’s to transform column 1 (the current column) to [1 0 … 0]T. Then, obtain the new current row, column, and entry by moving down one row and one column to the right, and go to Step 3. b If a11 (the current entry) equals 0, then do a Type 3 ero involving the current row and any row that contains a nonzero entry in the current column. Use ero’s to transform column 1 (the current column) to [1 0 … 0]T. Then, obtain the new current row, column, and entry by moving down one row and one column to the right, and go to Step 3. c If there are no nonzero numbers in the first column, obtain a new current column and entry by moving one column to the right. Then go to Step 3. INDR 262MetinTürkay24

  25. Gauss-Jordan Method Step 3 a If the new current entry is nonzero, use ero’s to transform it to 1 and the rest of the current column’s entries to 0. When finished, obtain a new current row, column, and entry. If this is impossible, stop. Otherwise, repeat Step 3. b If the current entry is 0, do a Type 3 ero with the current row and any row that contains a nonzero entry in the current column. Then, use ero’s to transform column entry to 1 and the rest of the current column’s entries to 0. When finished, obtain the new current row, column, and entry. If this is impossible, stop. Otherwise, repeat Step 3. c If the current column has no nonzero numbers below the current row, obtain a new current column and entry and repeat Step 3. If it is impossible, stop. INDR 262MetinTürkay25

  26. Gauss-Jordan Method Step 4Write down the system of equations A’x=b’ that corresponds to the matrix [A’b’] obtained when Step 3 is completed. Then, A’x=b’ will have the same set of solutions as Ax=b. • The Gauss–Jordan method converts the augmented matrix [Ab] into [A’b’] such that INDR 262MetinTürkay26

  27. Example • Solve the following system of linear equations using the Gauss-Jordan method. 2x1 + 2x2 + x3 = 9 2x1 - x2 + 2x3 = 6 x1 - x2 + 2x3 = 5 INDR 262MetinTürkay27

  28. Solution INDR 262MetinTürkay28

  29. Solution INDR 262MetinTürkay29

  30. Solution x1 = 1x2 = 2x3 = 3 INDR 262MetinTürkay30

  31. Analysis of the Solutions to Systems of Linear Equations • For any linear system, a variable that appears with a coefficient of 1 in a single equation and a coefficient of 0 in all other equations is called a basic variable. Any variable that is not a basic variable is called a nonbasic variable. x set of all of the variables in the system Ax=b xB set of all of the basic variables in the system Ax=b xN set of all of the nonbasic variables in the system Ax=b x = xBxN INDR 262MetinTürkay31

  32. Analysis of the Solutions to Systems of Linear Equations • The solution to A’x=b’ can be categorized in one of the three cases: Case 1:A’x=b’ has at least one row of the form [0 0 … 0c] and c0. Then, Ax=b has no solution. Case 2: When case 1 does not apply and xN=, then Ax=b has a unique solution. Case 3: When case 1 does not apply and xN, then Ax=b has infinite number of solutions. INDR 262MetinTürkay32

  33. Example • Case 1 does not apply since there are no rows of the form [0 0 … 0c] and c0. • Case 2 does not apply since, xB={x1, x2, x3} xN={x4, x5} • There are infinite number of solutions. INDR 262MetinTürkay33

  34. Example • Assign arbitrary values to the variables in xN; x4=c, x5=k. • Write down the equations in [A’b’], x1 + c + k = 3 x1 = 3 - c – k x2 + 2c = 2 x2 = 2 - 2c x3 + k = 1 x3 = 1 - k • It is easy to see that there are infinite number of values of c and k that will satisfy this system of equations. INDR 262MetinTürkay34

  35. Linear Combination • A linear combination of the vectors in V is any vector of the form c1v1+c2v2+…+ckvk, where c1, c2, …, ck are arbitrary scalars. • Example: V={[1,2], [2,1]} 2v1-v2 = 2([1 2]) – [2 1] = [0 3] 0v1-3v2 = 0([1 2]) – 3([2 1]) = [6 3] INDR 262MetinTürkay35

  36. Linear Independence & Linear Dependence • A set V of m-dimensional vectors is linearly independent if the only linear combination of vectors in V that equals 0 is the trivial linear combination. • A set V of m-dimensional vectors is linearly dependent if there is a nontrivial linear combination of the vectors in V that adds up to 0. INDR 262MetinTürkay36

  37. Example 1 • V={[1,0], [0,1]}Try to find a linear combination of vectors in V that yields 0. • c1([1 0]) + c2([0 1]) = [0 0] • In order to satisfy this, [c1c2] = [0 0] c1=c2=0 • The only linear combination of vectors in V that yields 0 is the trivial linear combination. Therefore, V is a linearly independent set of vectors. INDR 262MetinTürkay37

  38. Example 2 • V={[1,2], [2,4]}Try to find a linear combination of vectors in V that yields 0. • c1([1 2]) + c2([2 4]) = [0 0] [c12c1] + [2c24c2] = [0 0] c1 + 2c2 = 0 c1 = -2c2 2c1 + 4c2 = 0  2c1 = -4c2 • So, c1= 2 c2 = -1is one of the possible solutions. • There exists a nontrivial linear combination of vectors in V that yields 0. Therefore, V is a linearly dependent set of vectors. INDR 262MetinTürkay38

  39. The Rank of a Matrix • Let A be any mxn matrix, and denote the rows of A by r1, r2, …, rm. Also define R={ r1, r2, …, rm}. • The rank of A is the number of vectors in the largest linearly independent subset of R. • If for a matrix A with m rows, rank A=m; then the matrix is a collection of linearly independent set of vectors. If rank A<m; then the matrix contains a linearly dependent set of vectors. INDR 262MetinTürkay39

  40. Examples RankA = 3 RankB = 2 INDR 262MetinTürkay40

  41. The Inverse of a Matrix • A single linear equation in a single variable can be solved by multiplying both sides of the equation by multiplicative inverse of the variable coefficient. • Example: 4x=3  4-1(4x) = (4-1)3 x=3/4 • We can generalize this approach to square systems of linear equations (i.e., number of equations = number of unknowns). INDR 262MetinTürkay41

  42. Square and Identity Matrix • A square matrix is any matrix that has an equal number of rows and columns. • The diagonal elements of a square matrix are those elements aij such that i=j. • A square matrix for which all diagonal elements are equal to 1 and all non-diagonal elements are equal to 0 is called an identity matrix. INDR 262MetinTürkay42

  43. Inverse of A • For a given mxm matrix A, the mxm matrix B is the inverse of A if BA = AB = Im AA-1=I INDR 262MetinTürkay43

  44. Finding the Inverse of a Matrix with the Gauss-Jordan Method Step 1 Write down the mx2m matrix [AIm]. Step 2 Use ero’s to transform [AIm] into [ImB]. This will only be possible if rank A=m; in this case, B=A-1. Ifrank A<m, then A has no inverse. INDR 262MetinTürkay44

  45. Example INDR 262MetinTürkay45

  46. Using Matrix Inverse to Solve Linear Systems of Equations • Given a linear system of equations, Ax=b • Multiply both sides by A-1 AA-1x = A-1bx = A-1b INDR 262MetinTürkay46

  47. Example 2x1 + 5x2 = 7 x1+ 3x2 = 4 INDR 262MetinTürkay47

  48. Determinant of a Matrix • Any square matrix A has a number called the determinant of A (shown by set A or A). INDR 262MetinTürkay48

  49. Minor of A • If A is an mxm matrix; then for any values of i and j, the ijthminor of A (Aij) is the (m-1)x(m-1) submatrix of A obtained by deleting row i and column j of A. INDR 262MetinTürkay49

  50. Determinant of a Matrix • Let A be any mxm matrix, • det A = (-1)i+1ai1(det Ai1) + (-1)i+2ai2(det Ai2) + … +(-1)i+maim(det Aim) INDR 262MetinTürkay50

More Related