Topology in Manifold Learning

1. Topology in Manifold Learning Jonathan Huang Presented at misc-read, 11.22.06

2. Bibliography • *Simultaneous Inference of View and Body Pose Using Torus Manifolds*Chan-Su Lee and Ahmed Elgammal The 18th International Conference on Pattern Recognition (ICPR), Hong Kong, August 21-24, 2006 • *Finding the Homology of Submanifolds with High Confidence from Random Samples.*P. Niyogi, S. Smale, and S. Weinberger to appear, Discrete and Computational Geometry, 2006. • *On the Local Behavior of Spaces of Natural Images*G. Carlsson, T. Ishkhanov, V. de Silva, and A. Zomorodian, /preprint/, May 31, 2006. • *Computing Persistent Homology*A. Zomorodian and G. Carlsson, Discrete and Computational Geometry, *33* (2), pp. 247-274, 2005.

3. Outline • The Role of Topology in Manifold Learning • Constrained Topology Manifold Learning • Topology Basics • Learning a Topology from Noisy Data • Statistical Approach • Multi-scale Approach

4. The ISOMAP Algorithm • The ISOMAP algorithm: • Compute pairwise distances for some point cloud • Choose an embedding dimension, d (d=2 in this example) • Run the following code in matlab: >> options.dims = 2; >> [Y, R, E] = Isomap(DistanceMatrix, options); ISOMAP

5. Selecting the Dimensionality • Problem: How do we choose embedding dimension? • Several solutions (maybe some non-NIPS solutions too): • Brand (NIPS 2003) • Kegl (NIPS 2003) • Levina and Bickel (NIPS 2005) • Raginsky and Lazebnik (NIPS 2006) Dimensionality Estimation d=2

6. Why none of these are actually solutions Dimensionality Estimation d=2 ISOMAP (samples from a sphere)

7. Moral • Manifold Learning is hard if you don’t take topology into account!

8. Outline • The Role of Topology in Manifold Learning • Constrained Topology Manifold Learning • Topology Basics • Learning a Topology from Noisy Data • Statistical Approaches • Multi-scale Approaches

9. Manifold Learning with Known Topology • Images of a periodic gait from varying viewpoints • What is the intrinsic topology of this set of images? Body Pose View angle (0°-330°)

10. Learning a Mapping from a Torus • The product of two periodic spaces is a torus! • Use kernel methods to learn a map to/from a Torus

11. Some Results

12. Outline • The Role of Topology in Manifold Learning • Constrained Topology Manifold Learning • Topology Basics • Learning a Topology from Noisy Data • Statistical Approaches • Multiresolution Approaches

13. What is Topology? • Popular answer: It’s the branch of math that can’t tell the difference between a coffee cup and a donut

14. What is Topology? • Topology cares about how a space is connected • It does not care about distances • We will define a very general class of topological spaces – the simplicial complexes

15. Simplex • An n-simplex is the convex hull of (n+1) (independent) points 0-simplex 1-simplex 2-simplex

16. Simplicial Complex • A simplicial complex is a finite collection of simplices S such that • Any face of a simplex in S is also in S • The intersection of two simplices in S is either empty or a face for both simplices • The dimension of S is the maximum dimension over all simplices in S

17. Boundary Maps • Graphs are one-dimensional simplicial complexes • Define a boundary matrix M1 with rows corresponding to vertices and columns corresponding to edges • M1(v,e) = 0 if vertex v is not part of edge e • M1(v,e) = -1 if vertex v is the first vertex of edge e • M1(v,e) = 1 if vertex v is the second vertex of edge e a ab ac b bc c

18. Boundary Maps • Example: • The boundary of an edge is its two vertices

19. Boundary Maps • Example: • The boundary of the loop is empty:

20. Boundary Maps • In general, the dim(Nullspace(M)) is the number of “different” loops in the graph • And (#vertices)-Rank(M) is the number of connected components

21. Betti Numbers • For a simplicial complex S, we can define a boundary matrix Mk at each dimension k of S • Define the kth betti number to be • k = Dim(Nullspace(Mk-1))-Dim(Complement of column space(Mk))

22. Betti Numbers • For an object in 3d space • 0 is the number of connected components • 1 is the number of tunnels or handles • 2 is the number of ‘voids’ Point Circle Torus

23. Outline • The Role of Topology in Manifold Learning • Constrained Topology Manifold Learning • Topology Basics • Learning a Topology from Noisy Data • Statistical Approaches • Multi-scale Approaches

24. Problem Formulation • Given x1,x2,…,xn, i.i.d samples from a manifold. What are the betti numbers of the manifold?

25. An “Well Known” Algorithm • Two Steps: • First put an -ball around each point • Compute the betti numbers of the union of these balls using the Nerve Complex

26. The Nerve Complex • Given a collection of balls, U1,U2,… in Euclidean space, what is the topology of their union? • Construct the Nerve Complex: • For each ball, add a vertex • Add a k-simplex whenever k+1 balls have nonempty intersection

27. Nerve Complex • The Nerve Lemma states that the original space  and the Nerve Complex have the same Betti numbers! • Example A[ B A B A B A[ C B[ C C C

28. A PAC-bound • With high probability and enough samples, we can recover the true Betti numbers! • The number of samples depends on the volume of the manifold and its condition number (how close it gets to itself) • (Niyogi, Smale, Weinberger, 2006) • (but how does one choose ?)

29. Outline • The Role of Topology in Manifold Learning • Constrained Topology Manifold Learning • Topology Basics • Learning a Topology from Noisy Data • Statistical Approaches • Multi-scale Approaches

30. Another Example Spiral… or Torus???

31. Topological Persistence • Topological Persistence looks at topology from all scales at once • Surprisingly, it’s not much harder to compute the betti numbers at every scale!

32. Filtered Complex • A Filtered Complex is an increasing sequence of simplicial complexes (Ct Ct+1) • t is called the filtration index t=0 t=1 t=3 t=2

33. Barcodes - Example

34. Barcodes • Barcodes represent betti numbers as a function of filtration index • Intuitively, Barcodes measure the “lifetime” of topological features • The Persistence Algorithm provides a way to compute barcodes efficiently

35. Persistence Algorithm • Empirically, the algorithm works in linear time • Worst case complexity bound ~O(m3) • Where m is the # of simplices • Pros: • Persistence distinguishes local features from global features • Applies to learning manifold topology from noisy data • Cons: • No real probabilistic semantics

36. Conclusion • Learning a topology is not hopeless. So… • The next time you decide to learn a manifold, take a moment to contemplate the underlying topology!

37. Thank You!