1 / 55

Introduction to Matrices Douglas N. Greve dgreve@mgh.harvard

Introduction to Matrices Douglas N. Greve dgreve@mgh.harvard.edu. Why Matrices?. Simplifies notation Simplifies concepts Grounding for general linear model (GLM) Simplifies implementation (eg, matlab, octave) Optimization. 2. 4. y. p. 3.3i. -7.4. What is the matrix?.

ktruitt
Download Presentation

Introduction to Matrices Douglas N. Greve dgreve@mgh.harvard

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Matrices Douglas N. Greve dgreve@mgh.harvard.edu

  2. Why Matrices? • Simplifies notation • Simplifies concepts • Grounding for general linear model (GLM) • Simplifies implementation (eg, matlab, octave) • Optimization

  3. 2 4 y p 3.3i -7.4 What is the matrix? • A matrix is a table of numbers, like a spread sheet • Each entry is called an “element” • There is a value in each element (no empty cells) • Elements can be whole numbers, positive, negative, fractions, real, imaginary, symbolic variables M M = M

  4. Parts of Matrices • Size/Dimensions • Elements • Groupings of Elements • Diagonals • Triangles • Vectors

  5. 2 4 y p 3.3i -7.4 Matrix Size/Dimension • Number of Rows • Number of Columns • Rows-by-Columns (RowsxCols) 2 4 y 1x3 2x3 2 p p 2x1 1x1

  6. Scalar versus Matrix • Scalar is just a number or value • No dimension or size g = -1 No brackets No dimension Not a 1x1 g = [-1] 1x1

  7. 2 4 y p 3.3i -7.4 Matrix Element • Indicated by its row and column number • Scalar value M(row,col) M = M(i,j) 2x3 Mij (1,2) (1,1) (1,3) M(2,3) = -7.4 (2,3) (2,1) (2,2)

  8. Diagonals 2 4 6 1 3 5 Row = Column + Offset 7 8 9 3x3 2 4 6 1 3 5 Row = Column - 2 (Second Upper Diagonal) } Row = Column - 1 (First Upper Diagonal) 7 8 9 Row = Column + 0 (Main Diagonal) Row = Column - 1 (First Lower Diagonal) } Off-Diagonals Row = Column - 2 (Second Lower Diagonal)

  9. Triangles 2 4 6 1 3 5 Upper Triangle 7 8 9 3x3 Lower Triangle

  10. Vector • Special Type of Matrix • Number of Rows or Columns = 1 • Often given lower-case variable names 2 4 y 1x3 Row Vector 2 p 2x1 Column Vector

  11. 2 2 2 4 4 4 x y x p p p 3.3i 3.3i 3.3i -7.4 -7.4 -7.4 A matrix consists of vectors 2x3 Two 1x3 row vectors Three 2x1 column vectors

  12. Special Matrices • Square • Diagonal • Identity • Triangular • Symmetric • Toeplitz

  13. Square Matrix • Rows = Columns 2 4 p 1x1 p 3.3i 2x2

  14. Diagonal Matrix • All off-diagonal elements = 0 • M(i,j)=0 if i != j 2 0 0 0 3 0 0 0 9 3x3

  15. Identity Matrix • Main diagonal has all 1s • All off-diagonal elements = 0 • Square • Symbol “I” 1 0 0 I= 0 1 0 0 0 1 3x3 • Any matrix multiplied by the identity is itself

  16. 2 0 0 1 3 0 2 4 6 7 8 9 0 3 5 3x3 0 0 9 3x3 Triangular Matrices Upper Triangular – all values below the main diagonal are 0 Lower Triangular – all values above the main diagonal are 0

  17. Symmetric Matrix • M(row,col) = M(col,row) • Reflect across main diagonal • Square 2 4 6 4 3 5 6 5 9 3x3

  18. Toeplitz Matrix All elements on a diagonal are the same 2 4 6 2 4 1 2 4 1 2 7 1 2 3x3 7 1 3x2 A causal, time invariant linear system is represented by an upper triangular Toeplitz matrix

  19. Ones Matrix • All elements are 1 • Any number of rows or columns 1 1 1 1 1 1 1x3 1 1 1 2x3 1 1 1 2x1 1x1

  20. Zeros Matrix • All elements are 0 • Any number of rows or columns 0 0 0 0 0 0 1x3 0 0 0 2x3 0 0 0 2x1 1x1

  21. Matrix Operations • Addition/Subtraction • Vector Multiplication • Matrix Multiplication • Multiplication by scalar • Transpose • Trace • Inverse • Pseudo Inverse

  22. Matrix Addition/Subtraction • Element-by-Element Addition/Subtraction • Size of two matrices must be the same • C(row,col) = A(row,col) +/- B(row,col) C = A + B (your first matrix equation!) 2 4 6 9 0 7 A = B = 1 3 5 1 -2 3.3 2x3 2x3 2 11 15 C = 2 1 8.3 2x3

  23. 0 1 7 Vector Multiplication • One Row Vector and One Column Vector • Same length, Results in 1x1 • Inner Product, Dot Product, “Scalar” Product • Like a correlation • “Multiply and accumulate” z = x.y= x*y = S xi*yi 2 4 6 x = 1x3 2 4 6 * * * 0 1 7 0+4+42 = 46 y = 3x1

  24. Matrix Multiplication • Series of vector multiplications • Same “inner dimension” 0 2 8 4 2 4 6 A = 1 6 2 5 B = 1 3 5 3 9 3 7 2x3 3x4 c11 c12 c13 c14 cij=Arowi*Bcolj C=A*B = c21 c22 c23 c24 2x4 Dimensions: (2x3) *(3x4)=(2x4)

  25. 0 2 4 6 1 3 Matrix Multiplication • Series of vector multiplications • Same “inner dimension” 0 2 8 4 2 4 6 A = 1 6 2 5 B = 1 3 5 3 9 3 7 2x3 3x4 c11 c12 c13 c14 c11= C=A*B = c21 c22 c23 c24 2x4 2*0+4*1+6*3= 22

  26. 4 2 4 6 5 7 Matrix Multiplication • Series of vector multiplications • Same “inner dimension” 0 2 8 4 2 4 6 A = 1 6 2 5 B = 1 3 5 3 9 3 7 2x3 3x4 22 c12 c13 c14 c12= C=A*B = c21 c22 c23 c24 2x4 2*4+4*5+6*7= 70

  27. 2 2 4 6 6 9 Matrix Multiplication • Series of vector multiplications • Same “inner dimension” 0 2 8 4 2 4 6 A = 1 6 2 5 B = 1 3 5 3 9 3 7 2x3 3x4 22 70 c14 c13 c13= C=A*B = c21 c22 c23 c24 2x4 2*2+4*6+6*9= 82

  28. 8 2 4 6 2 3 Matrix Multiplication • Series of vector multiplications • Same “inner dimension” 0 2 8 4 2 4 6 A = 1 6 2 5 B = 1 3 5 3 9 3 7 2x3 3x4 22 70 c14 82 c14= C=A*B = c21 c22 c23 c24 2x4 2*8+4*2+6*3= 42

  29. 0 1 3 5 1 3 Matrix Multiplication • Series of vector multiplications • Same “inner dimension” 0 2 8 4 2 4 6 A = 1 6 2 5 B = 1 3 5 3 9 3 7 2x3 3x4 22 70 42 82 c21= C=A*B = c21 c22 c23 c24 2x4 1*0+4*1+5*3= 18

  30. 4 1 3 5 5 7 Matrix Multiplication • Series of vector multiplications • Same “inner dimension” 0 2 8 4 2 4 6 A = 1 6 2 5 B = 1 3 5 3 9 3 7 2x3 3x4 22 70 42 82 c22= C=A*B = 18 c22 c23 c24 2x4 1*4+4*5+5*7= 54

  31. Outer Product (Vector Multiplication) 0 0 1 1 7 7 • One Row Vector one Column Vector • Same length (N) • Result is NxN matrix • Special case of Matrix Multiplication z = y*x 2 4 6 2 4 6 x = 1x3 1x3 0 0 0 y = 2 4 6 14 28 42 3x1 3x3 3x1

  32. Multiplication with a scalar • Multiply each element by scalar value 2 4 6 g = -1 A = No brackets No dimension Not a 1x1 1 3 5 2x3 -2 -4 -6 gA = -1 -3 -5 2x3

  33. Matrix Transpose • “Reflect” across diagonal • B(row,col) = A(col,row) 2 1 2 4 6 4 3 A = B=AT =At =A’= 1 3 5 6 5 2x3 3x2 If A=A’, then A is symmetric (must be square)

  34. Trace Sum of diagonal Elements Trace = S M(i,i) 2 4 6 1 3 5 M = 7 8 9 Trace = 2 + 3 + 9 = 14

  35. Determinant • Property of a square matrix • Complicated in general • Simple for a 2x2 c11 c12 D = c11*c22 - c12* c21 C = c21 c22 2x2

  36. 1 D Matrix Inverse • C*A = I, then A=C-1 andC=A-1 • Must be square • Complicated in general • Simple for a 2x2 c11 c12 c22 -c12 C = C-1 = c21 c22 -c21 c11 2x2 2x2 D = c11*c22 - c12* c21

  37. 1.0 2.0 1.0 -2.0 C = 0.5 1.0 C-1 = -0.5 1.0 2x2 2x2 1 0 Invertibility IMPORTANT!!! • Not all matrices are invertible • D=0 • “Singular” D = 1.0*1.0 - 2.0*0.5 = 1-1 = 0

  38. Singularity and “Ill-Conditioned” 1.0 2.0 C = 0.5 1.0 2x2 • Column 2 = twice Column 1 • Linear Dependence Ill-Conditioned: D is “close” to 0 Relates to efficiency of a GLM. D = 1.0*1.0 - 2.0*0.5 = 1-1 = 0

  39. Pseudo-Inverse • Can be used for non-square • Same rules on invertibility apply • Least-mean-square (LMS) for over-determined system If X has more columns than rows X+ = (X’*X)-1*X’ Check: X+*X = (X’*X)-1*X’*X=I If X has more rows than columns X+ = X’ *(X*X’)-1* Check: X*X+ = X*X’ *(X*X’)-1=I

  40. Matrix Applications

  41. 1 1 x = y = 1 2 3 1 3x1 3x1 Sum/Length/Average of a List x’ = [1 1 1]1x3 “Ones vector” sum(y) = x’*y = 1*1 + 1*2 + 1*3 = 6 1x3 3x1 length(y) = x’*x = 1*1 + 1*1 + 1*1 = 3 Averagem = (x’*x)-1(x’*y) [Note: GLM] = (3-1)*6 = 6/3 = 2

  42. 1 0 x = y = 1 2 3 1 3x1 3x1 Selective Average x’ = [0 1 1]1x3 “Ones vector” sum(y) = x’*y = 0*1 + 1*2 + 1*3 = 5 1x3 3x1 length(y) = x’*x = 0*0+ 1*1 + 1*1 = 2 Averagem = (x’*x)-1(x’*y) = (2-1)*5 = 5/2 = 2.5

  43. 1 y = 2 3 3x1 Sum of the Squares of a List y’ = [1 2 3]1x3 SS =S(y2) SS(y) = y’*y = 1*1 + 2*2 + 3*3 = 14

  44. 1 1 y = x = 2 1 3 1 3x1 3x1 Variance of a List s2 =S(yi-m)2 (N-1) Residual e = y – m*x s2 = sqrt(e’*e)/DOF DOF = x’*x - 1 = #RowsX - #ColsX

  45. 2 3 .2 Orthogonality • Vector product is 0 • Property of two vectors • Uncorrelated/Independence - statistical property z = x*y = S xi*yi 1 -1 5 x = 1x3 1 * 2 + -1 * 3 + 5 * .2 = [0] y = 1x1 3x1

  46. y1 y2 x1 x2 Fitting a Line Thickness Dependent Variable, Measurement Subject 1 Subject 2 HRF Amplitude IQ, Height, Weight Age Independent Variable

  47. m= (y2-y1) (x2-x1) Intercept: b y1 Slope: m y2 x1 x2 Age b= (x2*y1-x1*y2) (x2-x1) Linear Model System of Linear Equations y1 = 1*b + x1*m y2 = 1*b + x2*m Two Eqns, Two Unknowns • Solve 1st Eq for b • Substitute into 2nd Eq • Solve 2nd equation for m • Substitute m back into Eq for b “Intercept” = “Offset”

  48. Intercept: b y1 Slope: m b m y1 y2 1 x1 1 x2 y2 = * x1 x2 Age Matrix Formulation/GLM y1 = 1*b + x1*m y2 = 1*b + x2*m 2x1 2x1 2x2 y = X*b • - parameter vector X - design matrix

  49. Intercept: b y1 Slope: m b m y1 y2 1 x1 1 x2 y2 = * x1 x2 Age Matrix Formulation/GLM y = X*b 1 x1 • # of Cols of X = # Parameters • Each parameter has a column • Each column means something • Parameter meaning from column X = 1 x2

  50. Intercept: b y1 Slope: m b m y1 y2 1 x1 1 x2 y2 = * x1 x2 Age GLM Solution y = X*b Multiply both sides by X-1 b=X-1*y 1 x1 x2 -x1 1 D X-1 = X = 1 x2 -1 1 • = x2-x1 Non-invertible if x1=x2 Ill-conditioned if x1 near x2 -- Sensitive to noise

More Related