1 / 27

Signals relevant to counter-terror.

Explore cutting-edge computer vision techniques for identifying individuals and potential malicious behavior. Learn about gait recognition, stress detection, and facial action analysis. Uncover how lies can be betrayed through cognitive and emotional clues. Discover the challenges and solutions in utilizing computer vision for behavioral science.

lansing
Download Presentation

Signals relevant to counter-terror.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer vision approaches to identifying people and possible malfeasant behavior. Dimitris N. Metaxas Mark G. FrankDirector, Computational Biomedicine School of Communication, Imaging & Modeling Center Information & Library Studieswith special thanks to:Paul Ekman, UCSF;Sinuk Kang, Amy Marie Keller, Anastacia Kurylo, Maggie Herbasz, Belida Uckun, Rutgers;Jeff Cohn, Pitt; Takeo Kanade, CMU; Javier Movellan & Marni Bartlett, UCSD.David Dinges, UPENNAlso thanks to: Office of Naval Research, National Science Foundation (ITR program), AFOSR, DARPA M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  2. M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  3. Signals relevant to counter-terror. • Identification of bad guys • Changes in gait with loads as small as 1 kg • Anger prior to imminent attack • Fear/distress when lying M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  4. Take a closer look… Kim Philby, 1960’s (Frank & Ekman, 2003) M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  5. How lies are betrayed.Lie Cognitive clues Emotional clues Lying about feelings Feelings about lying • - Contradictory statements • Hesitations • Speech errors • Reduced illustrators • Contradictory emblems • Reduced detail • Etc. -Look for reliable signs of emotion • -Duping delight • Guilt • Detection • apprehension M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  6. The Facial Action Coding System (FACS) Ekman & Friesen, 1978 46 Action Units Action code: 1, 2, 4, 5, 7, 20, 26 1 Inner brow raise 2 Outer brow raise 4 Brow lower 5 Upper lid raise 7 Lid tighten 20 Lip stretch 26 Jaw drop M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  7. Challenges facing behavioral science: • Advantages: - reliably identify people & behaviors - non-obtrusive - non-inferential, allows for discovery • Disadvantage: - laborious - mistaken identity via cognitive capacity, disguise, etc • Solution - automatic computer vision techniques M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  8. Gait recognition • Identify people from the way they walk • Important for surveillance and intrusion detection • What are good features for identifying a person? • i.e., what features are person-specific? M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  9. Background Sagittal plane - divides body into left and right halves Limb segment - a vector between two sites on a particular limb M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  10. Elevation Angles M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  11. The trajectories of the sagittal elevation angles are invariant across different subjects. As a consequence, person-independent gait recognition will require less training data. (Borghese et al., 1996) M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  12. The cyclogram • Elevation angles trace curve in a 4D space • Curve is called “cyclogram” • Cyclogram lies in a 2D plane • Well, almost • Hypothesis: deviation of cyclogram from plane is person-specific M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  13. Cyclogram example Curve is cyclogram projected into best-fit plane Green points are real points of cyclogram Red lines trace the deviation of points from plane (exaggerated scale) M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  14. Cyclogram sequence • Deviation from cyclogram plane can be represented as a sequence • e.g., CCCGTTTTATATTTTTAAAAGCCGGTAAATTAGGGG • Compare sequences between people via longest common subsequence (LCS) matching • Well-known dynamic programming algorithm, used in computational biology M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  15. Examples of People Detection M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  16. Examples of Gait Analysis M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  17. Face: Tracking: Stress Recognition • Identify which Facial Features (space and time) are important to recognize stress • Assymetries and Movements around the mouth and eyebrows M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  18. Slope = Asymmetry A horizontal line would indicate no asymmetry. This facial expression, however, is generally slanted upward and to the left. M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  19. asymmetry time Expression of high stress in form of asymmetrical facial expression Plots of high and low stress M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  20. asymmetry time In contrast, low stress M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  21. Some more high stress from different subjects M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  22. Face: Tracking: Stress Recognition M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  23. Face: Tracking: Stress Recognition M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  24. M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  25. Subtle brow changes important. M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  26. Technical Challenges. • Pose • Head motion • Occlusion from glasses, facial hair, rotation, hands • Talking • Video quality • Frame rate (blinks) M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

  27. Conclusions. • There are reliable means to identify people as well as behaviors associated with deception and hostile intent. • We can detect these behaviors. • We can represent them digitally. • Can this make us more secure? M. Frank & D. Metaxas - Rutgers HMS Symposium 2003

More Related