ITEC4310 Applied Artificial Intelligence

1. ITEC4310Applied Artificial Intelligence Lecture 6 Computer Vision and Autonomous Vehicles

2. Attribution • The following slides are taken from Charles R. DyerEmeritus Professor of Computer Sciences and Biostatistics and Medical InformaticsUniversity of Wisconsin, Madisonhttp://pages.cs.wisc.edu/~dyer/

3. Face Detection and Recognition Reading: Chapter 18.10

4. Face Classifier Window Non-face Face Detection Problem • Scan window over image • Classify window as either: • Face • Non-face

5. Face Detection: Motivation Automatic camera focus http://cdn.conversations.nokia.com.s3.amazonaws.com/wp-content/uploads/2013/09/Nokia-Pro-Camera-auto-focus_half-press.jpg http://cdn.conversations.nokia.com.s3.amazonaws.com/wp-content/uploads/2013/09/Nokia-Pro-Camera-auto-focus_half-press.jpg

6. Face Detection: Motivation Automatic camera focus Easier photo tagging First step in any face recognition algorithm http://images.fastcompany.com/upload/camo1.jpg

7. Face Detection: Challenges • Large face shape and appearance variation • Scale and pose (yaw, roll, pitch) variation • Background clutter • Occlusion • hair • glasses • hat • Lighting • Expression • Makeup

8. The Viola-Jones Real-Time Face Detector Challenges: Each image contains 10,000 – 50,000 locations and scales where a face may occur Faces are rare: 0 - 50 per image > 1,000 times as many non-faces as faces Want a very small # of false positives: <10-6 P. Viola and M. Jones, 2004

9. Training Data (grayscale) 5,000 faces (frontal) 108 non-faces Faces are normalized Scale, translation Many variations Across individuals Illumination Pose (rotation both in plane and out) Use Machine Learning to Create a 2-Class Classifier

10. Use Classifier at AllLocations and Scales

11. Building a Classifier Compute lots of very simple features Efficiently choose the best features Each feature is used to define a “weak classifier” Combine weak classifiers into an ensemble classifier based on boosting Learn multiple ensemble classifiers and “cascade” them together to improve classification accuracy and speed

12. Computing Features At each position and scale, use a sub-image (“window”) of size 24 x 24 Compute multiple candidate features for each window Want to rapidly compute these features

13. Local Features What are local features trying to capture? The local appearance in a region of the image David G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2 (2004)

14. What Types of Features? • Use domain knowledge • The eye region is darker than the forehead or the upper cheeks • The nose bridge region is brighter than the eyes • The mouth is darker than the chin • Encoding • Location and size: eyes, nose bridge, mouth, etc. • Value: darker vs. lighter

15. Features 4 feature types (similar to “Haar wavelets”): Two-rectangle Three-rectangle Value = ∑ (pixels in white area) - ∑ (pixels in black area) Four-rectangle

16. Huge Number of Features 160,000 features for each window!

17. Computing Features Efficiently:The Integral Image • Intermediate representation of the image • Sum of all pixels above and to left of (x, y) in image i: • Computed in one pass over the image: ii(x, y) = i(x, y) + ii(x-1, y) + ii(x, y-1) − ii(x-1, y-1)

18. Using the Integral Image x (0,0) s(x, y) = s(x, y-1) + i(x, y) ii(x, y) = ii(x-1, y) + s(x, y) (x,y) y • With the integral image representation, we can compute the value of any rectangular sum in constant time • For example, the integral sum in rectangle D is computed as: • ii(4) + ii(1) – ii(2) – ii(3)

20. Results

21. Profile Detection

22. Profile Features

23. Face Alignment and Landmark Localization Goal of face alignment: automatically align a face (usually non-rigidly) to a canonical reference Goal of face landmark localization: automatically locate face landmarks of interests http://www.mathworks.com/matlabcentral/fx_files/32704/4/icaam.jpg http://homes.cs.washington.edu/~neeraj/projects/face-parts/images/teaser.png

24. Face Image Parsing Given an input face image, automatically segment the face into its constituent parts Smith, Zhang, Brandt, Lin, and Yang, Exemplar-Based Face Parsing, CVPR 2013

25. Face Image Parsing: Results + Input Soft segments Hard segments Ground truth

26. Face Image Parsing: Results + Input Soft segments Hard segments Ground truth

27. Face Detection and Recognition Reading: Chapter 18.10 and, optionally, “Face Recognition using Eigenfaces” by M. Turk and A. Pentland

28. Face Recognition Problem database query image Query face

29. Face Verification Problem • Face Verification (1:1 matching) • Face Recognition (1:N matching)

30. Application: Access Control www.viisage.com www.visionics.com

31. Biometric Authentication – Breaches!

32. Pay by Selfie Amazon, Mastercard, Alibaba developing methods

33. Application: Video Surveillance Face Scan at Airports www.facesnap.de

35. iPhoto Can be trained to recognize pets! http://www.maclife.com/article/news/iphotos_faces_recognizes_cats

36. iPhoto Things iPhoto thinks are faces

37. Why is Face Recognition Hard? The many faces of Madonna

38. Recognition should be Invariant to • Lighting variation • Head pose variation • Different expressions • Beards, disguises • Glasses, occlusion • Aging, weight gain • …

39. Intra-class Variability • Faces with intra-subject variations in pose, illumination, expression, accessories, color, occlusions, and brightness

40. Inter-class Similarity • Different people may have very similar appearance www.marykateandashley.com news.bbc.co.uk/hi/english/in_depth/americas/2000/us_elections Twins Father and son

41. Blurred Faces are Recognizable

42. Blurred Faces are Recognizable Michael Jordan, Woody Allen, Goldie Hawn, Bill Clinton, Tom Hanks, Saddam Hussein, Elvis Presley, Jay Leno, Dustin Hoffman, Prince Charles, Cher, and Richard Nixon. The average recognition rate at this resolution is one-half.

43. Upside-Down Faces are Recognizable The “Margaret Thatcher Illusion”, by Peter Thompson

44. Face Recognition Architecture Face Detection Feature Extraction Classification Face Identity Feature Vector Image Window

45. Image as a Feature Vector Consider an n-pixel image to be a point in an n-dimensional “image space,” x∈𝓡n Each pixel value is a coordinate of x Preprocess images so faces are cropped and (roughly) aligned (position, orientation, and scale) x 1 x x 3 2

46. A Rapid Survey II (Lecture 1) • Edge Detection • https://www.slideshare.net/simrangori/basics-of-edge-detection • Computer Vision • https://www.slideshare.net/stevencharlesmitchell/introduction-to-computer-vision • Autonomous Cars • https://www.slideshare.net/ShantanuVashishtha1/autonomous-vehicles-70049669

47. Readings and Assignments