REAL TIME FACE DETECTION

1. REAL TIME FACE DETECTION Justin Rilling Pooja Mhapsekar Moinuddin Sayed Ogom J Obinor

2. Introduction • The idea is to be able to detect faces appearing in an image. • The faces can be of different sizes and orientations. • The motivation behind implementing this on an FPGA is that image processing is inherently parallel in nature and lends itself well to an FPGA.

3. Viola and Jones’ Detector • Key Contributions • 1. Integral Image – fast computation of features used by the detector. • 2. Combines complex classifiers in “cascade” – focus on facial features.

4. Integral Image • Major contribution of the Viola and Jones detector. • Summation of pixel values of the original image. • Value at location (x,y) = sum of values of pixels above and to the left of (x,y). • Whole image converted to integral image and a window buffer used to scan the entire image.

5. Haar Classifiers • Classifiers are composed of 2-3 rectangles. • Weights and sizes associated with features – obtained through AdaBoost. • Classifier sum = ∑ (area * weights) • Integral image makes area calculation easy. Area = L4–L3–L2+L1. • Several classifiers compose a stage.

6. Stage • Sum of all Haar feature classifiers compared with the stage threshold. • Multiple stages, each stage has different number of classifiers. • Threshold obtained from AdaBoost algorithm.

7. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

8. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 426 542 Right Value 0.8378106 0 Stage Threshold Stage Sum + < 0.8226894 0.0 Left Value 0.0337941 1

9. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 426 542 Right Value 0.8378106 0 Stage Threshold Stage Sum + < 0.8226894 0.0337941 Left Value 0.0337941 1

10. Stage Number 0 Classifier Number 1 Face Detection Example Classifier Sum Classifier Threshold < 887 2045 Right Value 0.7488812 0 Stage Threshold Stage Sum + < 0.8226894 0.0337941 Left Value 0.1514132 1

11. Stage Number 0 Classifier Number 1 Face Detection Example Classifier Sum Classifier Threshold < 887 2045 Right Value 0.7488812 0 Stage Threshold Stage Sum + < 0.8226894 0.1852073 Left Value 0.1514132 1

12. Stage Number 0 Classifier Number 2 Face Detection Example Classifier Sum Classifier Threshold < 349 568 Right Value 0.6374819 0 Stage Threshold Stage Sum + < 0.8226894 0.1852073 Left Value 0.0900493 1

13. Stage Number 0 Classifier Number 2 Face Detection Example Classifier Sum Classifier Threshold < 349 568 Right Value 0.6374819 0 Stage Threshold Stage Sum + < 0.8226894 0.2752566 Left Value 0.0900493 1

14. Stage Number 0 Classifier Number 2 Face Detection Example Classifier Sum Classifier Threshold < 349 568 Right Value 0.6374819 0 Stage Threshold Stage Sum + < 0.8226894 0.2752566 Left Value 0.0900493 1 Window Buffer Failed Stage 0

15. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

16. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

17. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

18. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

19. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

20. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

21. Stage Number 0 Classifier Number 0 Face Detection Example Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

22. Top Level Design Get Image Get Scaled Image Get Integral Image Window Buffer (21x21) Draw Rectangle If face is det. End of Image? Face Detection Pipeline YES Monitor NO

23. Face Detection Pipeline

24. Original End Product USB Webcam FPGA Display USB DVI

25. Updated End Product FPGA Display DVI Digital Interface • 1/3 Color Camera Mod C3188A-6018 • Supports several standard image data formats including YCrCb 4:2:2 • 640 x 480 resolution • 5 V signaling • 30 fps

26. Updated End Product XC95144XL CPLD FPGA 5 V Signaling 3.3 V Signaling DVI • 1/3 Color Camera Mod C3188A-6018 • Supports several standard image data formats including YCrCb 4:2:2 • 640 x 480 resolution • 5 V signaling • 30 fps Display

27. DVI Controller

28. Questions ??? • References: • P. Viola and M. Jones, “Robust real-time object detection,” International Journal of Computer Vision, 57(2), 137-154, 2004. • Junguk Cho, ShahnamMirzaei, Jason Oberg, Ryan Kastner, “FPGA-Based Face Detection System Using Haar Classifiers,” Proceeding of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 103-112, 2009.

29. OpenCV Program – Lena.jpg

30. OpenCV Program – ER.jpg

31. Window Buffer (2, 152, 122) OpenCV Program – ER.jpg

32. Face Detection Pipeline

33. Window Buffer (3, 165, 20) OpenCV Program – ER.jpg

34. Face Detection Pipeline

35. Questions ???