1 / 23

Out-of-Sample Extension and Reconstruction on Manifolds

Out-of-Sample Extension and Reconstruction on Manifolds. Bhuwan Dhingra Final Year (Dual Degree) Dept of Electrical Engg. Introduction. An m - dimensional manifold is a topological space which is locally homeomorphic to the m -dimensional E uclidean space

hugh
Download Presentation

Out-of-Sample Extension and Reconstruction on Manifolds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Out-of-Sample Extension and Reconstruction on Manifolds BhuwanDhingraFinal Year (Dual Degree)Dept of Electrical Engg.

  2. Introduction • An m-dimensional manifold is a topological space which is locally homeomorphic to the m-dimensional Euclidean space • In this work we consider manifolds which are: • Differentiable • Embedded in a Euclidean space • Generated from a set of m latent variables via a smooth function f

  3. Introduction n >> m

  4. Non-Linear Dimensionality Reduction • In practice we only have a sampling on the manifold • Y is estimated using a Non-Linear Dimensionality Reduction (NLDR) method • Examples of NLDR methods –ISOMAP, LLE, KPCA etc. • However most non-linear methods only provide the embedding Y and not the mappings f and g

  5. Problem Statement g y* x* f

  6. Outline • p is the nearest neighbor of x* • Only the points in are used for extension and reconstruction

  7. Outline • The tangent plane is estimated from the k-nearest neighbors of p using PCA

  8. Out-of-Sample Extension • A linear transformation Aeis learnt s.t Y = AeZ • Embedding for new point y* = Aez* z* y* Ae

  9. Out-of-Sample Reconstruction z* y* Ar • A linear transformation Aris learnt s.t Z = ArY • Projection of reconstruction on tangent plane z* = Ary*

  10. Principal Components Analysis • Covariance matrix of neighborhood: • Let be the eigenvector and eigenvalue matrixes of Mk • Then • Denote then the projection of a point x onto the tangent plane is given by:

  11. Linear Transformation • Y and Z are both centered around and • Then Ae =BeRewhere Be and Re are scale and rotation matrices respectively • If is the singular value decomposition of ZTY, then

  12. Final Estimates

  13. Error Analysis • We don’t know the true form of f or g to compare our estimates against • Reconstruction Error: For a new point x* its reconstruction is computed as , and the reconstruction error is

  14. Sampling Density • To show: As the sampling density of points on the manifold increases, reconstruction error of a new point goes to 0 • In a k-NN framework, the sampling density can increase in two ways: • k remains fixed and the sampling width decereases • remains fixed and • We consider the second case

  15. Neighborhood Parameterization • Assume that the first m-canonical vectors of are along

  16. Reconstruction Error • But ArAe = I, hence

  17. Reconstruction Error • Tyagi, Vural and Frossard (2012) derive conditions on k s.t the angle between and is bounded • They show that as • Equivalently, where Rm is an aribitrarym-dimensional rotation matrix • and

  18. Reconstruction Error • Hence the reconstruction approaches the projection of x* onto

  19. Smoothness of Manifold • If the manifold is smooth then all will be smooth • Taylor series of : • As because x* will move closer to p

  20. Results - Extension • Out of sample extension on the Swiss-Roll dataset • Neighborhood size = 10

  21. Results - Extension • Out of sample extension on the Japanese flag dataset • Neighborhood size = 10

  22. Results - Reconstruction • Reconstructions of ISOMAP faces dataset (698 images) • n = 4096, m = 3 • Neighborhood size = 8

  23. Reconstruction error v Number of Points on Manifold • ISOMAP Faces dataset • Number of cross validation sets = 5 • Neighborhood size = [6, 7, 8, 9]

More Related