310 likes | 742 Views
Eigenfaces for Face Recognition ECE 533 Final Project, Fall ‘03. Min Luo, Dept. of Biomedical Engineering Yuwapat Panitchob, Dept. of Electrical & Computer Engineering. OUTLINE. Overview Eigenfaces for Recognition-- Algorithm Eigenfaces for Recognition-- Implementation
E N D
Eigenfaces for Face Recognition ECE 533 Final Project, Fall ‘03 Min Luo, Dept. of Biomedical Engineering Yuwapat Panitchob, Dept. of Electrical & Computer Engineering
OUTLINE • Overview • Eigenfaces for Recognition--Algorithm • Eigenfaces for Recognition--Implementation • Experiment Result • Conclusion
Overview • Face Representation -- Template-based approaches -- Feature-based approaches -- Appearance-based approaches • Face Detection -- Utilization of elliptical shape of human head (applicable only to front views ) [5] -- Manipulation of images in “face space” [1] • Face Identification -- Performance affected by scale, pose, illumination, facial expression, and disguise, etc.
OUTLINE • Overview • Eigenfaces for Recognition--Algorithm • Eigenfaces for Recognition--Implementation • Experiment Result • Conclusion
Eigenfaces Approach • In the language of information theory…… -- Efficient encoding followed by comparing one face encoding with a database of models encoded similarly • In mathematical terms……. -- Find the principal components of the face distribution, or the eigenvectors of the covariance matrix of the set of face images, called eigenfaces -- Eigenfaces are a set of features that characterize the variation between face images -- Each training face image can be represented in terms of a linear combination of the eigenfaces, so can the new input image -- Compare the feature weights of the new input image with those of the known individuals
Eigenfaces Approach(Contd.) • Major Steps 1. Initialization: acquire the training set of face images and calculate the eigenfaces, which define the face space 2. Given an image to be recognized, calculate a set of weights of the M eigenfaces by projecting it onto each of the eigenfaces 3. Determine if the image is a face at all by checking if the image is sufficiently close to the face space 4. If it is a face, classify the weight pattern as either a known person of as unknown 5. (Optional) If the same unknown face is seen several times, update the eigenfaces / weight patterns, calculate its characteristic weight pattern and incorporate into the known faces
Calculating Eigenfaces • Set of training images ( is a column vector of size ) Average face of the training set: Each training image differs from the average face by: • A total number of pairs of eigenvectors and eigenvalues of the covariance matrix (C: matrix) Eq. (1) where (A: matrix) Computationally Intractable!
Calculating Eigenfaces (Contd.) • For Computational Feasibility -- Only M-1 eigenvectors are meaningful ( ) Eigenvectors and associated eigenvalues of : Eq. (2) Therefore, are the eigenvalues of , are the associated eigenvalues Eq. (3) The associated eigenvalues allow us to rank the eigenvectors according to their usefulness in characterizing the variation among the images
Using Eigenfaces for Identification • Construction of Known Individuals’ Face Classes -- Images of known individuals are projected onto “face space” by a simple operation , wherei=1, 2, ……, M represents the ith individual, and k=1, 2, ……, M’ represents the weight coefficient of eigenvector . The pattern vector of the ith individual -- If an individual has more than one image, take the average of the pattern vectors of this person • Given a new image -- Project onto face space, and get its pattern vector -- Determine whether is a face image: Eq. (4) If < a predefined threshold , it is a face image; otherwise, not -- Classify either as a known individual or as unknown: Eq. (5) If < a predefined threshold , is identified as the k’th face class; otherwise, it is identified as unknown
Summary of Eigenfaces Recognition Procedure • Collect a set of training images of known individuals. The set may have a number of images for each person, with some variation in expression and in the lighting (say four images of ten people each, so M=40) • Calculate the (40 x 40) matrix L, find its eigenvectors and eigenvalues, and choose the M’ eigenvectors with the highest associated eigenvalues. (Let M’=10 in this example.) • Combine the normalized training set of images according to Eq. (3) to produce the (M’=10) eigenfaces • For each known individual, calculate the class vector by averaging the eigenface pattern vectors [from Eq. (5)] calculated from the original (four) images of the individual. Choose a threshold that defines the maximum allowable distance from any face class, and a threshold that defines the maximum allowable distance from face space [according to Eq. (4)].
Summary of Eigenfaces Recognition Procedure (Contd.) • For each new face image to be identified, calculate its pattern vector , the distance to each known class, and the distance to face space. If the minimum distance , and the distance , classify the input face as the individual associated with class vector ; if the minimum distance , but , then the image may be classified as “unknown”, and optionally used to begin a new face class 6. If the new image is classified as a known individual, this image may be added to the original set of familiar face images, and the eigenfaces may be recalculated (steps 1-4). This gives the opportunity to modify the face spaces as the system encounters more instances of known faces.
OUTLINE • Overview • Eigenfaces for Recognition--Algorithm • Eigenfaces for Recognition--Implementation • Experiment Result • Conclusion
System Structure LoadImages trainingimages.mat eigenfaces.mat ConstructEigenfaces Arrow pointing to a file: file being created/modified Arrow pointing from a file: file being loaded ClassifyNewface faceclasses.mat undoUpdateEigenfaces note_eigenfaces.mat Functional block File containing intermediate data: for the ease of communication between functional blocks
OUTLINE • Overview • Eigenfaces for Recognition--Algorithm • Eigenfaces for Recognition--Implementation • Experiment Results • Conclusion
Training Images Training images: (1) full head scale (2) head-on lighting (3) upright head tilt
Recognition with Different Head Tilts • Total number of test images: 24 -- each person in the training set has two test images -- one test image with head left-oriented, the other with head right-oriented • Recognition results -- true-positive’s: 11 -- false-positive’s: 13 -- false-negative’s (identified as unknown): 0 training image test image 1 test image 2
d a Recognition with Different Head Tilts(True-positive examples) david irfan b • training image • test image 1 • test image 2 e • training image • test image 1 c
a c Recognition with Different Head Tilts(False-positive examples) foof WRONG! b d b & d: test images a: b’s corresponding training image c: d’s corresponding training image
Recognition with Varying Illuminance • Total number of test images: 24 -- each person in the training set has two test images -- one test image with 45 degree lighting, the other with 90 degree lighting • Identification results -- true-positive’s: 21 -- false-positive’s: 3 -- false-negative’s (identified as unknown): 0 training image test image 1 test image 2
a b c Recognition with Varying Illuminance (Contd.) True-positive example: stan False-positive example: ming • training image, head-on lighting • test image, light moved by 45 degrees d c. training image, head-on lighting d. test image, light moved by 90 degrees
Recognition with Varying Head Scale • Total number of test images: 24 -- each person in the training set has two test images -- one test image has a medium head scale, the other has a small one • Recognition results -- true-positive’s: 7 -- false-positive’s: 17 -- false-negative’s (identified as unknown): 0 training image test image 1 test image 2
a d Recognition with Varying Head Scale (Contd.) True-positive example: stan b c • training image, full head scale • test image 1, medium head scale • test image 2, small head scale False-positive example: ming d. training image, full head scale e. test image, medium head scale e
Experiment Result Summary • From the experiments performed -- fairly good recognition rate with varying illuminance: 21/24 -- acceptable recognition rate with different head tilts: 11/24 -- poor identification with varying head scale: 7/24 • Large case study needs carrying out before a general conclusion can be drawn -- large training set: large group of people + a number of face images for each individual -- numerous testing: face images of people who are or aren’t in the training set -- combination of condition changes: e.g. head scale change + lighting change • Thresholding issue -- no false-negative identification in the experiment, because is set to be large -- tradeoff between false-negative and false-positive identification rates: they cannot be reduced simultaneously; optimal choice of could well balance them
OUTLINE • Overview • Eigenfaces for Recognition--Algorithm • Eigenfaces for Recognition--Implementation • Experiment Result • Conclusion
Advantages & Limitations of Eigenfaces Recognition Algorithm • Advantages 1. Ease of implementation 2. No knowledge of geometry or specific feature of the face required 3. Little preprocessing work • Limitations 1. Sensitive to head scale 2. Applicable only to front views 3. Good performance only under controlled background (not including natural scenes)
Further Improvement • To reduce false-positive rate, return a number of candidates from existing face classes instead of a single face class • Regarding the pattern vector representing a face class, make each face class consist of several pattern vectors, each constructed from a face image of the same individual taken under a certain condition, instead of taking the average of these pattern vectors to represent the face class • To be discovered…
References • “Eigenfaces for recognition”, M. Turk and A. Pentland, Journal of Cognitive Neuroscience, vol.3, No.1, 1991 • “Face recognition using eigenfaces”, M. Turk and A. Pentland, Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pages 586-591, 1991 • “Face recognition for smart environments”, A. Pentland and T. Choudhury, Computer, Vol.33 Iss.2, Feb. 2000 • “Face recognition: Features versus templates”, R. Brunelli and T. Poggio, IEEE Trans. Pattern Analysis and Machine Intelligence, 15(10): 1042-1052, 1993 5. “Human face segmentation and identification”, S. A. Sirohey, Technical Report CAR-TR-695, Center for Automation Research, University of Maryland, College Park, MD, 1993
References (Contd.) 6. “Human and machine recognition of faces: A survey”, R. Chellappa, C. L. Wilson, and S. Sirohey, Proc. of IEEE, volume 83, pages 705-740, 1995 7. “Eigenfaces vs. fisherfaces: Recognition using class specific linear projection”, P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, IEEE Trans. Pattern Analysis and Machine Intelligence, 19(7):711-720, 1997 8. “Illumination cones for recognition under variable lighting: Faces”, A. S. Georghiades, D. J. Kriegman, and P. N. Belhumeur, Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pages 52-59, 1998 • “Automatic recognition and analysis of human faces and facial expressions: A survey”, A. Samal and P. A. Iyengar, Pattern Recognition, 25(1): 65-77, 1992 • ftp://whitechapel.media.mit.edu/pub/images/
& Min Luo Yuwapat Panitchob Thank you!