Applied Geometrical Matrix Computations

1. Applied GeometricalMatrix Computations Alan Edelman Dept of Mathematics: MIT MIT Laboratory for Computer Science Householder Symposium XV June 21, 2002

2. Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations

3. Geometrical Matrix Computations Working definition: • Concerns geometry of matrix space (n2 dimensions rather than n) • Involves numerical computation (probably MATLAB) • Relates to an NLA problem Some Other GMC People Absil, Demmel, Elmroth, Huhtanen, Kagstrom, Kahan, Lippert, Ma, Mahony, Malyshev, Sepulchre, Tisseur, Trefethen, Van Dooren

4. Vector Space Diagrams Points are vectors (not matrices!) Geometric relationships for vectors, subspaces, and linear transformations

5. Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations

6. Eigenland (in 2d) Isoeig surfaces are hyperbolas z x

7. The Eigenvalue Map Zero Matrix z z x x /2  0 0 

8. The Eigenvalue Map Zero Matrix z z x x /2 M M  0 0 

9. The Eigenvalue Map Zero Matrix z z x x /2 Uniformly M ? M  0 0 

10. Pseudospectra (Trefethen) Pseudoportraits Random Points z z “z is an eigenvalue of a matrix near A” Pseudoportraits = pictures of contours of z A = 1 -1 0 0 0 1 1 -1 0 0 1 1 1 -1 0 1 1 1 1 -1 0 1 1 1 1

11. pseudospectra & geometry matrix space eig (w/singularity) spectral portrait Project X L L

12. Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations

13. Circle/Hyperbola Tangency = High Density * * * * * * * * * Circles tangent to 2 hyperbolas… Circles tangent to 4 hyperbolas… Circles tangent to 3 hyperbolas… have eigenvalue distributions with 4 spikes. have eigenvalue distributions with 2 spikes. have eigenvalue distributions with 3spikes. frequency frequency frequency eigenvalue eigenvalue eigenvalue

14. Radius of Curvature = Highest Density This is even better than tangency, which means a higher spike frequency frequency eigenvalue eigenvalue Circles are tangent to 3 hyperbolas when two tangency points collide * The circle also shares a radius of curvature with the hyperbola at this point * * frequency eigenvalue

15. Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations

16. Where do Matrix Factorizations Come From? A=UV’ Classical Answer: Representation Theory of Semisimple Groups

17. Semisimple group recipe • Nicely links factorizations • Three Examples Nonsingulars Unitary Orthogonal SVD UeiV’ CS decomp SPD Eigen UeiU’ Essentially Sym Orth One more example Hyperbolic Svd as in last talk Group = SO(p,q) ( XJ=JX)

18. Matrix Factorizations Where can we look for new factorizations? • The Mathematics Literature • Lie Algebra: Cartan, Iwasawa, Bruhat • Representation Theory: Quivers • Nearness Problems • Applications • Engineering: A factorization is useful if someone can use it • Mathematics: The useful factorizations are characterized by an abstract criterion

19. Ideas to Generalize E = (antisymmetric) + (symmetric) M= Q * S [polar] expm expm expm Non-singular Pos Definite Orthogonal 1: Cartan Decomposition

20. Ideas to Generalize E = (antisymmetric) + (symmetric) M= Q * S [polar] expm expm expm Non-singular Pos Definite Orthogonal 1: Cartan Decomposition 2:KAK Decomposition Positive Diagonals = Maximal Group M=UV’ Conjugates give S=QQ’

21. Ideas to Generalize E = (antisymmetric) + (symmetric) M= Q * S [polar] expm expm expm Non-singular Pos Definite Orthogonal 1: Cartan Decomposition 2:KAK Decomposition Positive Diagonals = Maximal Group M=UV’ Conjugates give S=QQ’ 3:Iwasawa, Bruhat Above not unique at I. Gives M=LU, other permutations, totally positive, etc

22. Ideas to Generalize E = (antisymmetric) + (symmetric) M= Q * S [polar] expm expm expm Non-singular Pos Definite Orthogonal 1: Cartan Decomposition 2:KAK Decomposition Positive Diagonals = Maximal Group M=UV’ Conjugates give S=QQ’ 3:Iwasawa, Bruhat Above not unique at I. Gives M=LU, other permutations, totally positive, etc 4:Eigenvalue, Jordan Schur

23. Step 1:Cartan Decomposition • Group: non-singular matrices • Involution: ((M))=M (M1M2)= (M1)(M2) (M)=M-T • Fixed Points (M)=M are a group K K = orthogonal matrices • Near I M = (antisymmetric) + (symmetric) • Cartan: expm M = QS (S>0) (polar)

24. Step 1:Cartan Decomposition (U/O) • Group: unitary matrices • Near I M = (antisymmetric) + (i*symmetric) • Cartan: M= (real orth)(unitary symmetric)

25. Step 2:KAK Decomposition P = sym pos def • A = biggest group inside P (abelian) e.g.  diagonal > 0, or conjugates UU’ (fix U) • KAK M=UV’ • P = union of conjugates S=QQ’

26. Step 2:KAK Decomposition (U/Q) P = unitary symmetric • A = biggest group inside P (abelian) e.g. diagonals (ei) or conjugates • KAK M=UeiV’ (U, V real orthogonal) • P = union of conjugates S=QeiQ’ (Q real orthogonal)

27. Step 2:KAK Decomposition (On/Op XOq ) P = matrices orthogonally similar to ( ) • A = biggest group inside P (abelian) e.g. =( ) or conjugates • KAK The CS Decomposition

28. Missing • The constructible decompositions Tridiagonalization, Bidiagonalization • The NNMF (Lee, Seung 1999) • V  WH Input: Vij>0 Output: Wij>0 Hij>0 (low rank) Algorithm: H  H .* (W’V)./(W’WH) W  W .* (VH’)./(WHH’) Original Application: Eigenfaces Another Example: Color Science

29. Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations

30. Color Science: Light Spectra from film wavelength vs density Greens Blues Reds Grays

31. Film Recording and measurements Solid colors sent to film recorder, e.g. reds Negative sent through projector to spectrometer • Log ratio with no film (only bulb) film density = log(no film / with film) Negative is produced: film appears as cyans Energy data at each wavelength Reds

32. The Data Three significant singular values svd index • Inputs (r,g,b) for 1r,g,b 10 scaled (1000 frames) • Output Space: Densities at 400:3:700 nm’s • Data Structure: 101 x 1000 matrix “A” Compute SVD(A) Project onto best 3 space

33. SVD Basis = no physical meaning

34. The NNMF Basis = primary colors

35. Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations

36. Singular 2x2 Matrices (by svd) _ A=() cos  cos  cos  sin  -sin  cos  -sin  sin  Torus! Torus Cone? All isoeig surfaces are translates of the =0 surface! hyperpolas and hyperboloids are cross sections!

37. Bohemian Dome

38. Linear Algebra with movies Horizontal Vertical Villarceau A=QQT A=QQ A=QR Hopf Fibration

39. Challenges Incorporate 3d graphics tools directly into Matrix computations. Include geometry of matrix space. How should this look? Generalize everything and incorporate into software