Robust GPCA Algorithm with Applications in Video Segmentation Via Hybrid System Identification

1. MTNS 2004 Robust GPCA Algorithm with Applications in Video Segmentation Via Hybrid System Identification Kun Huang and Yi Ma Perception & Decision Laboratory Decision & Control Group, CSL University of Illinois at Urbana-Champaign http://black.csl.uiuc.edu/~kunh

2. INTRODUCTION A ROBUST RECURSIVE GPCA ALGORITHM HYBRID SYSTEM IDENTIFICATION VIA SUBSPACE SEGMENTATION VIDEO SEGMENTATION CONCLUSION

3. GPCA – Problem Formulation • Problem Statement [Multiple Linear Model Fitting] Given a set ofNnoisy data points sampled froms different subspaces with possibly different dimensions in aK-dimensional ambient space: • Estimate the number of subspacessand the dimensionki • (i=1,2,…,s) of each subspace; identify a basis for each subspace; • Segment the given data points into the subspaces. • Difficulties • Is the problem “chicken-and-egg” ? • Noise affects robustness. • Review of Generalized PCA (Vidal-Ma, CVPR’03, 04) • Solve the “chicken-and-egg” dilemma • Closed-form analytical solution • Propose a polynomial factorization approach to data segmentation • Number of groups = degree of a polynomial • Groups = polynomial factors • In the absence of noise • The exact solution can be computed using linear algebra

4. GPCA – Robust Algorithm • Contributions of the Robust GPCA Algorithm Research • Design a new model selection criterion for multiple linear subspaces. • Develop a Robust Recursive GPCA algorithm. • Recursively identify the correct number of subspaces and their dimensions and bases. • Robustly segment the data points based on specified maximum error tolerance.

5. GPCA – Effective Dimension • Model selection criteria • MML, MDL, AIC, G-AIC, Robust AIC Balance model complexity and data fidelity. • Effective dimension • Specifically developed for mixture of linear models (subspaces) Dimension of each subspace Number of subspaces Total number of points Number of points in each subspace

6. GPCA – Minimum Effective Dimension • Example Model selection criterion: Minimum Effective Dimension (MED)

7. GPCA - MED and Robust GPCA Algorithm MED of a data set is closely related to error tolerance t. • Extreme cases: If t is infinity, then MED=0; if t is 0, then MED=K. Robust approach for mixture linear model fitting: For a specified maximum error tolerance t, minimize the Effective Dimension (ED) of the data set. t t Robust recursive GPCA algorithm • Recursively segment each group; • Automatically search for the number and the dimensions of the subspaces; • Assign points to the subspace based on the specified error tolerance t; • Accommodate outliers.

8. GPCA - Algorithm details

9. GPCA - Algorithm details • The last Mn-reigenvectors of Lncorrespond to the coefficients of the desired polynomials. • The derivatives of the polynomials evaluated at different data points determine the subspaces. • Estimation of r is closely related to the segmentation of the data points: • If the estimation of r is too small, the estimation of subspaces are too small and not all the points can be assigned into the subspaces within the error tolerance t; • If the estimation of r is too large, the estimation of subspaces are too large and not all n subspaces can be found. • The search of r can be achieved via a binary search. Estimation of r Eigenvalues of Ln Index

10. ED=3 ED=2.0067 ED=1.6717 A ROBUST RECURSIVE ALGORITHM – A Simulation Example

11. Mixture of LTI Systems HYBRID SYSTEM IDENTIFICTION –Problem Formulation Single LTI System

12. Hybrid LTI System Identification HYBRID SYSTEM IDENTIFICTION –Problem Formulation Hybrid LTI System Switching function

13. HYBRID SYSTEM IDENTIFICTION –Embedding • Embed the input ut and output yt in a high-dimensional space. • The embedded data point resides on a subspace defined by • the system. • Mixture of LTI systems generate a mixture of linear subspaces. • The mixture of linear subspaces can be identified using the • GPCA algorithms. Two ways of embedding: • Embedding via the oblique projection (Overschee et. al. ’96); • Direct input/output embedding.

14. InputBlock Hankel Matrix HYBRID SYSTEM IDENTIFICTION –Notations Past Input Future Input Future Output Past Output OutputBlock Hankel Matrix

15. Embedding via the oblique projection (Overschee et. al. ’96) • Subject to • The covariance matrix of the input block Hankel matrix is 2i; • The intersection of the row spaces of future input and past state is trivial. HYBRID SYSTEM IDENTIFICTION –Oblique Projection Orthogonal Projection Oblique Projection

16. Embedded data point HYBRID SYSTEM IDENTIFICTION –Direction Embedding • Direct input/output embedding • The embedded data points are on a subspace. • If the system is observable and , .

17. HYBRID LTI SYSTEM IDENTIFICATION - Simulations 4th order 3rd order 1st order Data points around the switching points do NOT belong to any of the three subspaces. They cause outliers!

18. HYBRID LTI SYSTEM IDENTIFICATION -Simulations • Embedding via the oblique projection Average errors of the subspaces for 1000 trials Segmentation of the embedded data • Direct input/output embedding

19. APPLICATIONS –Video Segmentation PCA GPCA Stipulation: the segmentation is based on the “dynamics” in the image sequence.

20. APPLICATIONS –Video Segmentation • Video Sequence as Dynamical Systems • Perspective observability of image sequence (Ghosh-Loucks’94) • Dynamic textures (Dorreto et. al.’01,03) • Subspace mapping for image sequence (Brand’02) • Synthesizing Dynamic Texture with Closed-Loop Linear Dynamic System (Yuan et. al. ECCV’04) For image sequences, we only have the output but not the input. Look for subspace structure for each system.

21. APPLICATIONS –Video Segmentation Case 1: • Potential problems: • The subspaces are defined based on the observation matrix, not the system dynamics. • Possible over splitting based on local linear structure of xt.

22. APPLICATIONS –Video Segmentation Case 2: • Segmentation methods for multiple systems: • Direct segmentation using the low-dimensional output data yt. • Segmentation for embedded output data points.

23. APPLICATIONS – Video Segmentation Experiment Setup: Testing sequence (150 image frames) Image Size: 352X240 Error Tolerance for the GPCA Algorithm: 0.05 rad Outlier Tolerance for the GPCA Algorithm: 15% m=3 Image Frames Grayscale Images PCA m-dimensional space

24. Outliers – Cannot be assigned to lower-dimensional subspaces Over-splitting APPLICATIONS – Direct Segmentation Segmentation Results: m=3 m=2

25. Camera Zooming Out APPLICATIONS – Segmentation of Embedded Output m=3, k=2

26. CONCLUSION • Present a new model selection criterion for multiple linear subspaces. • Develop a robust GPCA algorithm to recursively segment the data points into multiple subspaces with different dimensions. • The hybrid LTI system identification problem can be converted into a GPCA problem. Two ways of embedding are provided. • Video sequences can be modeled using hybrid linear systems. The robust GPCA algorithm can segment the video sequence based on the hidden dynamics via the embedding of the system output.