Active Appearance Models

1. Active Appearance Models Dhruv Batra ECE CMU

2. Active Appearance Models T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc. European Conference on Computer Vision 1998 (H.Burkhardt & B. Neumann Ed.s). Vol. 2, pp. 484-498, Springer, 1998 T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", IEEE PAMI, Vol.23, No.6, pp.681-685, 2001 G.J. Edwards, A. Lanitis, C.J. Taylor, T. F. Cootes. “Statistical Models of Face Images Improving Specificity”, BMVC (1996)

3. Essence of the Idea • “Interpretation through synthesis” • Form a model of the object/image (Learnt from the training dataset) I. Matthews and S. Baker, "Active Appearance Models Revisited," International Journal of Computer Vision, Vol. 60, No. 2, November, 2004, pp. 135 - 164.

4. Essence of the Idea (cont.) • Explain a new example in terms of the model parameters

5. So what’s a model Model “texture” “Shape”

6. Active Shape Models training set

7. Texture Models warp to mean shape

8. Random Aside • Shape Vector provides alignment = 43 Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

9. Random Aside • Alignment is the key 1. Warp to mean shape 2. Average pixels Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

10. Random Aside • Enhancing Gender more same original androgynous more opposite D. Rowland, D. Perrett. “Manipulating Facial Appearance through Shape and Color”, IEEE Computer Graphics and Applications, Vol. 15, No. 5: September 1995, pp. 70-76

11. Random Aside (can’t escape structure!) Antonio Torralba & Aude Oliva (2002) Averages: Hundreds of images containing a person are averaged to reveal regularities in the intensity patterns across all the images. Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

12. Random Aside (can’t escape structure!) Tomasz Malisiewicz, http://www.cs.cmu.edu/~tmalisie/pascal/trainval_mean_large.png

13. Random Aside (can’t escape structure!) “100 Special Moments” by Jason Salavon Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml

14. Random Aside (can’t escape structure!) “Every Playboy Centerfold, The Decades (normalized)” by Jason Salavon 1960s 1970s 1980s Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml

15. Back (sadly) to Texture Models raster scan Normalizations

16. PCA Galore Reduce Dimensions of shape vector Reduce Dimension of “texture” vector They are still correlated; repeat..

17. Object/Image to Parameters modeling ~80

18. Playing with the Parameters First two modes of shape variation First two modes of gray-level variation First four modes of appearance variation

19. Active Appearance Model Search • Given: Full training model set, new image to be interpreted, “reasonable” starting approximation • Goal: Find model with least approximation error • High Dimensional Search: Curse of the dimensions strikes again

20. Active Appearance Model Search • Trick: Each optimization is a similar problem, can be learnt • Assumption: Linearity • Perturb model parameters with known amount • Generate perturbed image and sample error • Learn multivariate regression for many such perterbuations

21. Active Appearance Model Search • Algorithm: • current estimate of model parameters: • normalized image sample at current estimate

22. Active Appearance Model Search • Slightly different modeling: • Error term: • Taylor expansion (with linear assumption) • Min (RMS sense) error: • Systematically perturb and estimate by numerical differentiation

23. Active Appearance Model Search (Results)