Manifold Learning – from Brain Visualization to Advanced Image Processing

1. IMT IMT • Intrinsic and Extrinsic means Geodesics • Manifolds • Image Processing Manifold Learning– from Brain Visualization to Advanced Image Processing • Manifold Learning • Diffusion Weighted MRI Tensors • Visualization • Machine Learning Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center for Medical Imaging and Visualization (CMIV) Linköpings universitet, Sweden

2. Thesis Advisors • Hans Knutsson Prof. Medical Informatics Department of Biomedical Engineering Linköpings universitet • Carl-Fredrik Westin Associate Prof. of Radiology Laboratory of Mathematics in Imaging Harvard Medical School / Brigham and Women´s Hospital • Magnus Herberthson Assistant Prof. Department of Mathematics Linköpings universitet

3. Abstract • In this talk I discuss the need to standardize image processing on manifolds. Two tools are presented, a framework for storage of geometric tensor arrays and a method for manifold learning, related to the Riemannian exponential and logaritmic map. Finally some future perspectives are presented, in particular the fusion of manifold learning and image processing.

4. Processing Tensor fields on Manifolds • Storage, processing and visualization of functions • :M!V­V ... ­V­ V*­ V*­ ... ­V* • Intrinsic formulations, Charts, Atlases, Coordinate Free Methods ... • Extrinsic methods can be problematic and there are more complicated cases than 2D surfaces embedded in 3D! RP2 also SO(3), ...

5. The Similar Tensor Array Core The Similar Tensor Array Conventions A collaboration by: Anders Brun, Burak Acar, Emma Munos, Leila Cammoun, Andreas Sigfridsson, Dario Sosa, Marcos Martin Fernandez, Björn Svensson, Magnus Herberthson Hans Knutsson ”Simple conventions for storing tensor array data ...”

6. What problem are we solving? • ”The Ideal Customer” • A fresh MSc or PhD student starting to work with tensor data • Wanting to: • Apply new algorithms to old (known) test data • Apply old algorithms to new datsets • Exchange tensor data between algorithms and/or people

7. Core and Extensions • A lot of tensor processing can be done in the core • Extensions should include conventions for: • DTI • Elastography • Image Processing • Geoscience • ... • Existing standards • NRRD, VTK, Analyze, ... Worlds STAE Physics Applications STAC Mathematics Processing paradigms Data storage

8. Scalar data • array_dimensionality = 2 • array_size = [4, 3] • tensor_order = 0 • tensor_index_types = {} • data = [0.1 0.3 0.7 0.5 0.7 0.5 ... 0.5 0.6 0.8 0.1 0.5 0.3] (4,1) (4,2) (4,3) (3,1) (3,2) (3,3) (2,1) (2,2) (2,3) (4 * 3) (1,1) (1,2) (1,3)

9. Vector data • array_dimensionality = 2 • array_size = [4, 3] • tensor_order = 1 • tensor_index_types = {contravariant} • data = [1.2 1.4 1.2 0.4 0.5 0.4]... but in what basis??? (4,1) (4,2) (4,3) (3,1) (3,2) (3,3) (2,1) (2,2) (2,3) (4 * 3 * 2) (1,1) (1,2) (1,3)

10. The basis describing tensors • Indices are coordinates:(c1, c2) or generally ci • Basis vectors given by: or generally (4,1) (4,2) (4,3) (3,1) (3,2) (3,3) (2,1) (2,2) (2,3) (1,1) (1,2) (1,3)

11. Dual vectors • array_dimensionality = 2 • array_size = [4, 3] • tensor_order = 1 • tensor_index_types = {covariant} • data = [0.2 2.4 3.1 0.3 1.2 0.7] (4,1) (4,2) (4,3) (3,1) (3,2) (3,3) (2,1) (2,2) (2,3) (4 * 3 * 2) (1,1) (1,2) (1,3)

12. A metric instead of voxelsize • array_metric_tensor = gij (4,1) (4,2) (4,3) (4,1) (4,2) (4,3) (3,1) (3,2) (3,3) (3,1) (3,2) (3,3) (2,1) (2,2) (2,3) (2,1) (2,2) (2,3) (1,1) (1,2) (1,3) (1,1) (1,2) (1,3)

13. A metric instead of voxelsize • array_metric_tensor = gij (4,1) (4,2) (4,3) (4,1) (4,2) (4,3) (3,1) (3,2) (3,3) (3,1) (3,2) (3,3) (2,1) (2,2) (2,3) (2,1) (2,2) (2,3) (1,1) (1,2) (1,3) (1,1) (1,2) (1,3)

14. Additional core fields • [tensor_name] = D • [tensor_description] = ”Diffusion” • [tensor_index_names], e.g. {a, b} • [array_index_names], e.g. {i, j, k}

15. Image Processing is often invariant ... ... to how the image is displayed!

16. Tensor Array Processing should be invariant too. Remember, in some applications there is no extrinsic space!

17. Even without an extrinsic space you can do ... • Geodesic tracking (given a metric field gij)) • Distances (given a metric field gij) • Balls (given a metric field gij) • Fiber tracking (given a metric tensor) • Image processing on curved manifolds (given a metric field gij) • Tilted volumes with non-isotropic voxelsizes (given a metric tensor) • Correct visualizations, up to actions of the orthogonal group O(n) (with a metric) or general linear group GL(n) (without a metric) • Solve PDEs • Interpolate the field

18. Manifolds in Image Processing – It´s a Klein bottle!

19. Manifolds in Biology Optical recording from the visual cortex of a macaque monkey. “[...] reasoning based on topological arguments may be increasingly helpful in attempting to understand how the world, Klein bottles and all, is unfolded onto the surface of the cortex.” – N. Swindale Adapted from Visual Cortex: Looking into a Klein Bottle by Nicholas Swindale, Current Biology, Vol. 6 No. 7:776–779, 1996

20. Dimension reduction / Manifold Learning • Dimension Reduction • Principal Components Analysis, PCA (Pearson, 1901; Hotelling, 1933) • Multidimensional Scaling, MDS (Torgerson, 1952; cf. Young and Householder, 1938) • Projection Pursuit (Friedman and Tukey, 1974) • Independent Components Analysis, ICA (Herault and Jutten, 1986) • Non-linear Dimension Reduction • Self Organizing Maps, SOM (Kohonen, 1982) • Principal Curves ([Hastie and Stuetzle, 1989) • Kernel PCA (Schölkopf et al, 1998) • Isomap (Tenenbaum et al, 2000) • Locally Linear Embedding, LLE (Roweis and Saul, 2000) • Laplacian Eigenmaps, LE (Belkin & Niyogi, 2002) • Hessian Eigenmaps, H-LLE (Donoho & Grimes, 2003) • Diffusion Maps (Coifman, S. Lafon, 2004) • Local Tangent Space Alignment, LTSA (Zhang and Zha, 2004) • Sample Logmaps (Brun et al 2005)

21. Isomap • Isomap (Tenenbaum et al, Science, 2000)

22. Geodesics, Exponential maps and Logarithmic maps • Lagrangian style (Exponential maps) • Eulerian ...

23. Why are the Exp and Log functions Important in Signal Processing? Analogies of operations in vector spaces and in manifolds, reproduced from (Pennec et al.,2005).

24. An Eulerian approach to Logp(x) ... ... easy if we can calculate distances on M!

25. LOGMAP Example: The Swiss Roll No distortion for a “flat” metric!

26. LOGMAP – manifold hunting Data (high.dim.) Logmap – It´s a Klein bottle! From LOGMAP:A Method for estimating Riemannian normal coordinates for a set of datapoints sampled from a manifold by A. Brun, C.-F. Westin, M. Herberthson and H. Knutsson, submitted to ICIP 2005.

28. Example: 2D shape (S-Logmap)

29. Example: 2D shape (PCA)

30. Î L2(Rn) time M y x Manifolds of Images and Image Processing – What manifold is this? Find your favourite application for R3£ P2 ...

31. Problems in Manifold Learning • Calculation of distances! • What is the topology • What is a suitable metric? • Large samples sizes • Small sample sizes

32. Problems in Tensor Processing • How to move tensors on a manifold to perform filtering in a point? • The structure tensors in Hyper Image Manifolds? • Choosing and managing charts in an Atlas

41. Learning • Encyclopedia Britannica: “Process of acquiring modifications in existing knowledge, skills, habits, or tendencies through experience, practice, or exercise.” • Oxford Advanced Learner’s Dictionary: “gain knowledge or skill by study, experience or being taught.”