Principal Component Analysis

1. Principal Component Analysis

2. Consider a collection of points

3. Suppose you want to fit a line

4. Project onto the Line Consider variance of distribution on the line

5. Different line . . . different variance

6. Maximum Variance

7. Minimum Variance

8. Given by eigenvectors of covariance matrix of coordinates of original points

9. PCA notes… • Input data set • Subtract the mean to get data set with 0-mean • Compute the covariance matrix • Compute the eigenvalues and eigenvectors of the covariance matrix • Choose components and form a feature vector. Order by eigenvalues – highest to lowest

10. PCA • To compress, ignore components of lesser significance • The feature vector F is a matrix is the matrix of ordered eigenvectors • Derive the data set in the new coordinates: • new_data = FT old_data

11. Covariance • C, of 2 random variables X and Y where

12. Example

16. OOBB Choose bounding box oriented this way

17. OOBB: Fitting Covariance matrix of point coordinates describes statistical spread of cloud. OBB is aligned with directions of greatest and least spread (which are guaranteed to be orthogonal).

18. OOBB Good Box

19. OOBB Add points: worse Box

20. OOBB More points: terrible box

21. OOBB