吳育德 陽明大學放射醫學科學研究所 台北榮總整合性腦功能研究室

1. Introduction To Linear Discriminant Analysis 吳育德 陽明大學放射醫學科學研究所 台北榮總整合性腦功能研究室

2. Linear Discriminant Analysis For a given training sample set, determine a set of optimal projection axes such that the set of projective feature vectors of the training samples has the maximumbetween-class scatter and minimum within-class scatter simultaneously.

3. Linear Discriminant Analysis Linear Discriminant Analysis seeks a projection that best separate the data . Sb : between-class scatter matrix Sw : within-class scatter matrix

4. LDA Fisher discriminant analysis Sol:

5. LDA Fisher discriminant analysis where , = k1+k2 and let

6. LDA Fisher discriminant analysis

7. LDA Generalized eigenvalue problem.....Theorem 2 Let M be a real symmetric matrix with largest eigenvalue then and the maximum occurs when , i.e. the unit eigenvector associated with . Proof :

8. LDA Generalized eigenvalue problem.....proof of Theorem 2

9. LDA Generalized eigenvalue problem.....proof of Theorem 2 Cor: If M is a real symmetric matrix with largest eigenvalue . And the maximum is achieved whenever ,where is the unit eigenvector associated with .

10. LDA Generalized eigenvalue problem…….. Theorem 1 Let Sw and Sb be n*n real symmetric matrices . If Sw is positive definite, then there exists an n*n matrix V which achieves The real numbers λ1….λn satisfy the generalized eiegenvalue equation : : generalized eigenvector : generalized eigenvalue

11. LDA Generalized eigenvalue problem.....proof of Theorem 1 Let and be the unit eigenvectors and eigenvalues of Sw, i.e Now define then where Since ri ﹥0 (Sw is positive definite) , exist

12. LDA Generalized eigenvalue problem.....proof of Theorem 1

13. LDA Generalized eigenvalue problem.....proof of Theorem 1 We need to claim : (applying a unitary matrix to a whitening process doesn’t affect it!) (VT)-1 exists since det(VTSwV) = det (I ) → det(VT) det(Sw) det(V) = det(I) Because det(VT)= det(V) → [det(VT)]2 det(Sw) = 1 > 0 → det(VT) 0

14. LDA Generalized eigenvalue problem.....proof of Theorem 1 Procedure for diagonalizing Sw (real symmetric and positive definite) and Sb (real symmetric) simultaneously is as follows : 1. Find λi by solving And then find normalized , i=1,2…..,n 2. normalized