Geometric diffusions as a tool for harmonic analysis and structure definition of data

1. The second-round discussion* on Geometric diffusions as a tool for harmonic analysis and structure definition of data By R. R. Coifman et al. * The first-round discussion was led by Xuejun; * The third-round discussion is to be led by Nilanjan.

2. Diffusion Maps • Purpose • - finding meaningful structures and geometric descriptions of a • data set X. • - dimensionality reduction • Why? The high dimensional data is often subject to a large quantity of constraints (e.g. physical laws) that reduce the number of degrees of freedom.

3. Diffusion Maps • Markov Random Walk Many works propose to use first few eigenvectors of A as a low representation of data (without rigorous justification). • Symmetric Kernel • Relationship

4. Diffusion Maps • Spectral Decomposition of A where • Spectral Decomposition of Am • Diffusion maps

5. Diffusion Distance • Diffusion distance of m-step • Interpretation The diffusion distance measures the rate of connectivity between xi and xj by paths of length m in the data.

6. Diffusion vs. Geodesic Distance

7. Data Embedding • By mapping the original data into (often ) • The diffusion distance can be accurately approximated

8. Example: curves Umist face database: 36 pictures (92x112 pixels) of the same person being randomly permuted. Goal: recover the geometry of the data set.

9. Original ordering Re-ordering The natural parameter (angle of the head) is recovered, the data points are re-organized and the structure is identified as a curve with 2 endpoints.

10. Example: surface Original set: 1275 images (75x81 pixels) of the word “3D”.

11. Diffusion Wavelet • A function f defined on the data admits a multiscale representation of the form: • Need a method compute and efficiently represent the powers Am.

12. Diffusion Wavelet • Multi-scale analysis of diffusion Discretize the semi-group {At:t>0} of the powers of A at a logarithmic scale which satisfy

13. Diffusion Wavelet

14. The detail subspaces • Downsampling, orthogonalization, and operator compression A - diffusion operator, G – Gram-Schmidt ortho-normalization, M - AG • - diffusion maps: X is the data set

15. Diffusion multi-resolution analysis on the circle. Consider 256 points on the unit circle, starting with 0,k=kand with the standard diffusion. Plot several scaling functions in each approximation space Vj.

16. Diffusion multi-resolution analysis on the circle. We plot the compressed matrices representing powers of the diffusion operator. Notice the shrinking of the size of the matrices which are being compressed at the different scales.

17. Multiscale Analysis of MDPs [1] S. Mahadevan, “Proto-value Functions: Developmental Reinforcement Learning”, ICML05. [2] S. Mahadevan, M. Maggioni, “Value Function Approximation with Diffusion Wavelets and Laplacian Eigenfunctions”, NIPS05. [3] M. Maggioni, S. Mahadevan, “Fast Direct Policy Evaluation using Multiscale Analysis of Markov Diffusion Processes”, ICML06.

18. To be discussed a third-round led by Nilanjan Thanks!