Automatic Face Feature Localization for Face Recognition

1. Automatic Face Feature Localization for Face Recognition Christopher I. Fallin honors thesis defense: May 1, 2009 advisor: Dr. Patrick J. Flynn Chris Fallin - thesis defense

2. Outline • Face recognition • Methods of evaluation • Elastic Bunch Graph Matching • Gabor Jets • Bunch Graphs and Feature Localization • My Contributions: Automatic Fiducial Points • Information Content model • Fiducial point placement • results

3. Subfield of biometrics: life (bio) measure (metric) Extract identifying information from measures of human traits Face recognition: digital images of face 2D, 3D, infrared, multimodal, … Face Recognition http://www-users.cs.york.ac.uk/~nep/research/3Dface/tomh/3DFaces.jpg Chris Fallin - thesis defense

4. Face Recognition Evaluation Image Set Chris Fallin - thesis defense

5. ROC curves System Decision Y N Actual Y N http://en.wikipedia.org/wiki/File:Roc-general.png – used under terms of GNU FDL Chris Fallin - thesis defense

6. EER = 11.1% Chris Fallin - thesis defense

7. Rank-one Score Chris Fallin - thesis defense

8. Elastic Bunch Graph Matching (EBGM) • Wiskott et al., USC/Bochum, mid-90s • Basis of ZN-Face, successful commercial system • We use Face Identity Evaluation System, from Colorado State • Face features represented by Gabor filter responses • Features are localized • Fit an elastic graph onto the features by localization: local optimization problems Chris Fallin - thesis defense

9. A Face Graph [Wiskott99] Chris Fallin - thesis defense

10. Gabor Jets • Vector of filter responses to 40 Gabor kernels • 5 wavelengths • 8 orientations • Each is complex-valued • Gabor jets capture information well: Gokberk et al. get 91% rank-one with fixed grid • On FERET: 78.5% max, with 12 grid points Chris Fallin - thesis defense

11. Bunch Graphs • Each feature has a bunch of canonical jets • Represents typical features • Best-match at each feature point for novel images [Wiskott99] Chris Fallin - thesis defense

12. Feature Localization • Initial alignment: eye locations known a-priori • Overlay bunch graph with average edge lengths • Take Gabor jets; pick best match in each bunch • Localize based on displacement estimation (local optimization problem) Chris Fallin - thesis defense

13. The Idea: Automatic Fiducial Point Placement • Bunch graph training requires manual fiducial point placement • 70 images, 25 points • Why not statistically determine optimal features to match on? • We can align/normalize all faces and take some statistical measure at each point in “face space” to determine goodness • Replaces training step; back-end algorithm is identical Chris Fallin - thesis defense

14. Related Work • Gokberk et al.: choosing fiducial points with genetic algorithms • But their chosen points are global • Same goal as our system, excluding prelocalization • Salient Points • Wavelet-based approach to image retrieval • Choras et al., 2006: similar approach with goodness function, but no EBGM Chris Fallin - thesis defense

15. Information Content: Variance Model • Compute goodness function over face-space • Inter-subject variance over intra-subject variance • Self-normalizing unitless measure • Requires multiple images per subject Chris Fallin - thesis defense

16. Computing the goodness function • FRGC: 5404 images – 700 MB, 128x128 grayscale (7 GB before normalization) • Each pixel: 12 seconds, on fast Athlon 64 • Split into 128 Condor jobs • Each pixel is independent: easy • Pre-normalize image set, dump to fast-loading binary format (single file) • Run Condor jobs: three hours • Post-processing to reassemble results Chris Fallin - thesis defense

17. Fiducial Point Placement • Random placement with probability density • Compute gradient of goodness function • Probability is product of gradient and goodness • Place points sequentially, decay probability around points exponentially • Mirror-point constraint: mirror placements across centerline, or snap to center Chris Fallin - thesis defense

18. Prelocalization: Pseudo-Bunches • Displacement estimation requires canonical feature jet from bunch • We can’t provide this if we have no knowledge of feature • Solution: fake a jet bunch • Make educated guess with K-means clustering on jets from all images at given point • Then, run displacement estimation to prelocalize points on each image Chris Fallin - thesis defense

19. Results FERET • Competitive with original, manual points • In both cases, automatic training points yield only ~1% performance drop • With no human training! • Prelocalization did not work as intended • Success without this suggested by Gokberk’s results FRGC Chris Fallin - thesis defense

20. ROC curves FERET FRGC (EER = 11.4%) (orig: 11.1%) (EER = 31.4%) (orig: 34.8%) Chris Fallin - thesis defense

21. Prelocalization: causes for failure • Poor pseudo-bunch clustering: K-means often found optimal clustering at self-imposed cap of N/10 clusters • Likely because initial jets are too far off • Naïve localization: single-step • Bolme thesis compares several optimization algorithms • Average displacement of 2.628 pixels: larger than 2.021 pixels in manual points Chris Fallin - thesis defense

22. Future Work • More sophisticated prelocalization • Look at pseudo-bunch statistics to determine failure mode in more detail • Look at per-fiducial point statistics to determine where performance is weak • Investigate: are manual pts a theoretical limit, or can we exceed them? • Try new image classes – test claim of genericism Chris Fallin - thesis defense

23. Questions? • Email cfallin@c1f.net • Full thesis and source code will be posted online: http://c1f.net/research/mark5/ Chris Fallin - thesis defense

24. Distance Metrics on Jets • Phase-insensitive: magnitude only • Selects best jet in bunch • Phase-sensitive • Can solve for displacement vector: basis of localization • Displacement estimation Chris Fallin - thesis defense