Human Recognition Based On Facial Profile and ears

1. Human Recognition Based On Facial Profile and ears CmpE58Z Term Project Fırat Onur Alsaran, Neşe Alyüz, Melike Esma İlter, Mürsel Taşgın, Ahmet Burak Yoldemir

2. EAR RECOGNITION USING LOCAL BINARY PATTERNS Ahmet Burak Yoldemir

3. Motivation • Ear biometrics has several advantages over complete face • Facial biometrics may fail due to: • Expressions • Cosmetics • Hair styles • Growth of facial hair • Ears are affected very little from such changes

4. Ear database • 448 ear images are manually cropped from profile images of CMU Multi-PIE Database • Only left ears are used • There are 4 ear images of 112 people • Illumination conditions of these 4 images are all different

5. Samples from the database • Person 1: • Person 2: • High illumination variance!

6. First attempts • Filter bank approaches are applied first

7. First attempts • Filter bank approaches are applied first

8. First attempts • Filter bank approaches are applied first

9. First attempts • Filter bank approaches are applied first

10. Illumination tolerance • None of the filter bank approaches is able to tolerate illumination changes, as they have fixed bases • A grayscale invariant texture measure: Local Binary Patterns

11. Local binary patterns - Advantages • Tolerance against illumination changes • Computational simplicity • A compact description of the image

12. Local binary patterns - Example

13. Local binary patterns • After obtaining LBP codes, a histogram of these codes is obtained using 256 bins • This histogram is actually a histogram of micro-patterns • The result is a 256 dimensional feature vector of an ear image

14. Local binary patterns • LBP method is very sensitive to high frequency components • A slight noise can change the ordering of the pixel values in a neighborhood, which results in a different micro-pattern • To prevent this, images are filtered with a Gaussian kernel of 5x5 before finding micro-patterns

15. Recognition step • Euclidean distance between these feature vectors is used as the (dis)similarity measure • A similarity matrix is formed using these distances

16. Multi-presentation approach • To increase recognition performance, multi-presentation approach is adopted • Each ear is represented using 2 images, verification is accomplished by taking 2 ear images of the user • Mean and max rules are applied to fuse the scores

17. Results – Without Gaussian filtering

18. Results – With Gaussian filtering

19. Face Profile matching Mürsel Taşgın

20. Facial Profile recognition Motivation • Facial profile images can be collected from side cameras • Computation complexity is lower • Complementary solution for face recognition

21. Profile Database • 448 profile photos from Multi-PIE database • 112 subjects, each having 4 photos • Facial profiles are extracted manually in the first place

22. Facial Profile Registration 1 Extract profile 2 Rotate 90º CW 3 Edge detection 4 Chin & nose detection using gradient of image Nose at the center and touching top gradient Histogram representation (image to function) 6 Scale and move to top (nose at the center) 5

23. Facial Profile Registration (cont.) • Edge detection(Sobel) is used to convert black-white profile image to a histogram function • Profile line is decreased to a single pixel white line • Nose is the highest point in the histogram • Chin point is detected using gradient of histogram and image-filling function of Matlab: • If gradient of the image changes sharply at chin area, it is marked as chin point • If image-fill function fills in the chin area then the end point is marked as chin lips Image-filling detects lips, so use gradient to find chin

24. Facial Profile Matching (Histogram Matching) • Facial profiles are represented as histogram functions. • After registration, pointwise distance is measured: • Difference between points are summed over all points • Other metrics are available as well: Bhattacharyya distance • White line is profile-1 • Red line is profile-2 • Green vertical lines are distances

25. Multi-biometric fusion of facial profile and ear Neşe Alyüz

26. Motivation • Multiple biometric sources can provide better performance • Ear and Facial Profile biometrics can be acquired simultaneously • Instead of using a single modality of ear or profile, apply fusion • Most common fusion level: score level • Heterogeneous Scores –> score normalization is important

27. Score Normalization Techniques • Min-max normalization • Z-Score normalization • Median Absolute Deviation (MAD) normalization • Tanh normalization

28. Min-max Normalization • Best suited for the case where bounds are known • Shift scores into range [0 1] • Given a set of matching scores: {sk} • Normalized scores: • Original distribution is kept, only scaling When bounds are estimated, not robust to outliers

29. Z-score Normalization • Performs well if prior knowledge is available • Mean and standard deviation are used • Given a set of matching scores: {sk} • Normalized scores: Original distribution is not retained Does not guarantee a common numerical range When mean and std are estimated, very sensitive to outliers

30. Median Absolute Deviation (MAD) Normalization • Median and MAD are insensitive to outliers and to points in the extreme tails of the distribution • MAD normalization benefits from this fact • Normalized scores: where MAD = median(|sk - median|) Median and MAD have low efficiencies When score distribution is not Gaussian, poor estimates Input distribution is not retained Normalized scores are not in a common range

31. Tanh Normalization • Robust to outliers • Highly efficient • Normalized scores: • Tanh distribution: normalized genuine scores has a mean of 0.05 and std of ~o.o1. Determines the spread of genuine scores

32. Score Fusion Techniques • MAX rule • MEAN rule • SUM rule • PRODUCT rule Evaluated on scores that are normalized with different approaches

33. Experimental Results • Initial Results on Similarity matrices of Assignment #3: Face and Fingerprint biometrics • 40 subjects with 8 sample/subject • SMs: 320x320 similarity matrices • Enrollment: 1 sample/subject for each bimetric

34. Experimental Results - EERs

35. Experimental Results - TODO