1 / 7

Thermal Imaging and Facial Expression Detection

Thermal Imaging and Facial Expression Detection. A brief foray into thermographic and visual imaging approaches. Robert S. Pienta. Detecting Expressions. Locate the face within the image or video. Extract features and information from the face.

tolla
Download Presentation

Thermal Imaging and Facial Expression Detection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Thermal Imaging and Facial Expression Detection A brief foray into thermographic and visual imaging approaches. Robert S. Pienta

  2. Detecting Expressions • Locate the face within the image or video. • Extract features and information from the face. • Use these features in a classifier to determine which expression is formed.

  3. Standards and Commonalities • Most facial recognition papers utilize a system to categorize human emotional and facial expressions based on facial musculature • Although the implementation varies significantly, but the most common system is the Facial Action Coding System (FACS) developed by Paul Ekman and Wallace Friesen in 1976 [1]. • FACS provides a qualitative classification of facial action units (FAUs or AUs) • Combinations of AUs form different facial expression • Often papers reduce the number of base expression types to fewer than 10 different categories [2, 3, 4, 6, 9]. • The Candide facial grid is used a means by which to geometrically represent a face. The grid is a 100 polygon abstraction of the human face. http://www.icg.isy.liu.se/candide/

  4. Thermal Approaches • Similarly to the visual approach, the taxonomies of the FACS are still utilized [8,9]. • Again, similarly to before, rotation and intra-sequence alignment are necessary [8,9]. • Neural Network Approaches • Yoshitomi, Miyawaki, Tomita, and Kimura used NNs to classify faces into either neutral, happy, surprised, or sad expressions. They were among the very first to use thermal imaging to detect facial expression and had a success rate of 90% [9]. • Spatial pattern detection and analysis (SP) and region of interest (ROI) were both investigated by [8]. • SP utilizes PCA to determine the vectors of greatest variability among a set of vectorized thermal images. These regions are then mapped to a muscle-map in the face, and under the assumption that muscular contraction causes a heat fluctuation (e. g. [10]), they were able to classify individual AUs. • The system was then used to measure speeds of contraction in expressions in the face.

  5. Why use thermal? Visual Approach Drawbacks Overcome Via Thermal Imaging • Variance in color, amount, and position of lighting makes expression recognition more difficult for visual data • Some classification systems have ethnic or gender bias • The OpenCV face detector, the basis of comparison for many research applications, has a 16% disparity between the hit rates on white and black faces (87% versus 71% respectively) when tested using the GENKI dataset [4].

  6. Challenges • Camera position and the angle at which the image was captured further deepen the complexity of both types of approaches • There are also some major challenges that are unique to and follow from the use of thermal imagining. • Physiological response to the test environment may obscure accurate measurement • Atmospheric conditions within the test environment may adversely affect the measurements

  7. Questions? Works Cited [1] Ekman, P. and Friesen, W. ; Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, 1978. [2] Kotsia, I.; Pitas, I.; , "Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines," Image Processing, IEEE Transactions on , vol.16, no.1, pp.172-187, Jan. 2007 [3] Zeng, Z.; Fu, Y.; Roisman, G. ; Wen, Z.; Hu, Y;. Huang, T. ;, “Spontaneous Emotional Facial Expression Detection” Journal of Multimedia, vol. 1, pp.1-8, 2006 [4] Whitehill, J.; Littlewort, G.; Fasel, I.; Bartlett, M.; Movellan, J.; , "Toward Practical Smile Detection," Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.31, no.11, pp.2106-2111, Nov. 2009 [5] T. Sim, S. Baker, and M. Bsat, “The CMU Pose, Illumination, and Expression Database,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003. [6] Fasel, B.; Luettin, J.; Automatic facial expression analysis: a survey, Pattern Recognition, Volume 36, Issue 1, January 2003, Pages 259-275, ISSN 0031-3203, DOI: 10.1016/S0031-3203(02)00052-3. [7] Black, M. ; Yacoob, Y. ; “Recognizing facial expressions in image sequences using local parameterized models of image motion”,International Journal of Computer Vision 25 (1) (1997) 23–48. [8] Jarlier, S.; Grandjean, D.; Delplanque, S.; N’Diaye, K.; Cayeux, I.; Velazco, M.; Sander, D.; Vuilleumier, P.; Scherer K.; “Automatic Thermal Analysis of Facial Expressions” IEEE Transactions on Affective Computing. Mar. 2010 [9] Yoshitomi, Y.; Miyawaki, N.; Tomita, S.; Kimura, S.; , "Facial expression recognition using thermal image processing and neural network," Robot and Human Communication, 1997. RO-MAN '97. Proceedings., 6th IEEE International Workshop on , vol., no., pp.380-385, 29 Sep-1 Oct 1997

More Related