Principal Component Analysis of Face Properties

1. Principal Component Analysis of Face Properties Samarasena Buchala1, Neil Davey1, Tim Gale1,2, Ray Frank1 1 School of Computer Science, University of Hertfordshire, College Lane, Hatfield, AL10 9AB, UK 2 Department of Psychiatry, QEII Hospital, Welwyn Garden City, AL7 4HQ, UK

2. Overview • Principal Component Analysis – Face Recognition • Principal Component Analysis • Analysis of components of PCA • Linear Discriminant Analysis of PCA components • Does PCA efficiently encodes information in face images • Analysis of gender, ethnicity, age, and identity • Do components encode information related to multiple properties

3. Face Properties • Human face is considered to be special in terms of biological and social roles • Has multiple properties from which they can be categorised at different levels of specificity – gender, ethnicity, age, identity, expression, degree of attractiveness, typicality, attractiveness, so on. • Widely researched in the fields of Computer Science and Psychology

4. Principal Component Analysis (PCA) on Face Images • Dimensionality reduction – Sirovich and Kirby (1987) • Face recognition – Turk and Pentland (1991) • Benchmark for face recognition algorithms - (Moon & Phillips, 2001). • Distinctiveness effects of faces - (Hancock, 1996) • “other-race effect” - (O'Toole et al., 1991b) • Dimensional-based model of facial expression - (Calder et al., 2001)

5. Research Questions • Does PCA encode information related to gender, ethnicity, age, and identity efficiently? • What information do PCA encode? • Are there components (features) of PCA that encode multiples properties?

6. w1 w2 x2 x1 PCA • The aim of the PCA is a linear reduction of D dimensional data to d dimensional data (d<D), while preserving as much information, in the data, as possible. • Linear functions y1= w1 X y2= w2 X * * * yd= wd X Y= WX • X – inputs; Y – outputs, components; W – eigenvectors, eigenfaces, basis vectors

7. W1 W2 x2 x1 How many components? • Usual choice consider the first d PC’s which account for some percentage, usually above 90 %, of the cumulative variance of the data. • This is disadvantageous if the last components are interesting

8. Dataset • A subset of FERET dataset • 2670 grey scale frontal face images • Rich in variety: face images vary in pose, background lighting, presence or absence of glasses, slight change in expression

9. Dataset • Each image is pre-processed to a 65 X 75 resolution. • Aligned based on eye locations • Cropped such that little or no hair information is available • Histogram equalisation is applied to reduce lighting effects

10. Does PCA efficiently represents information in face images? • Images of 65 × 75 resolution leads to a dimensionality of 4875. • The first 350 components accounted for 90% variance of the data. • Each face is thus represented using 350 components instead of 4875 dimensions • Classification employing 5-fold cross validation, with 80% of faces in each category for training and 20% of faces in each category for testing • for identity recognition leave-one-out method is used. • LDA is performed on the PCA data • Euclidean measure is used for classification

11. What information does PCA encode? – Gender • Gender encoding power estimated using the LDA • 3rd component carries highest gender encoding power followed by the 4th components • All important components are among the first 50 components

12. What information does PCA encode? – Gender -6 SD -4 SD -2 SD Mean +2 SD +4 SD +6 SD Reconstructed images from the altered components (a) third and (b) fourth components. The components are progressively added by quantities of -6 S.D (extreme left) to +6 S.D (extreme right) in steps of 2 S.D. • Third component encodes information related to the complexion, length of the nose, presence or absence of hair on the forehead, and texture around the mouth region. • Fourth component encodes information related to the eyebrow thickness, presence or absence of smiling expression

13. Gender (a) Face examples with the first two being female and the next two being male faces. (b) Reconstructed faces of (a) using the top 20 gender important components. (c) Reconstructed faces of (a) using all components, except the top 20 gender important components.

14. What information does PCA encode? – Ethnicity • 6th component carries highest ethnicity encoding power followed by the 15th components • All ethnicity important components are among the first 50 components

15. Ethnicity -6 SD -4 SD -2 SD Mean +2 SD +4 SD +6 SD Reconstructed images from the altered components (a) 6th and (b) 4th components. The components are progressively added by quantities of -6 S.D (extreme left) to +6 S.D (extreme right) in steps of 2 S.D. • 6th component encodes information related to complexion, broadness and length of the nose • 15th component encodes information related to length of the nose, complexion, and presence or absence of smiling expression

16. What information does PCA encode? – Age • Age – 20-39 and 50-60+ age groups termed as young and old) • 10th component is found to be the most important for age -6 SD -4 SD -2 SD Mean +2 SD +4 SD +6 SD Reconstructed images from the altered tenth component. The component is progressively added by quantities of -6 S.D (extreme left) to +6 S.D (extreme right) in steps of 2 S.D

17. What information does PCA encode? – Identity • Many components are found to be important for identity. However, their importance magnitude is small. • These components are widely distributed and not restricted to the first 50 components

18. Can a single component encode multiple properties? • A grey beard informs that the person is a male and also, most probably, old. • As all important components of gender, ethnicity, and age are among the first 50 components there are overlapping components. • One example is the 3rd component which is found to be the most important for gender and second most important for age

19. Can a single component encode multiple properties? Normal distribution plots of the (a) third (b) and fourth components for male and female classes of young and old age groups.

20. Summary • PCA encodes face image properties such as gender, ethnicity, age, and identity efficiently. • Very few components are required to encode properties such as gender, ethnicity and age and these components are amongst the first few components which capture large part of the variance of the data. Large number of components are required to encode identity and these components are widely distributed. • There may be components which encode multiple properties.