1 / 33

Where, Who and What? @AIT Intelligent Affective Interaction ICANN, Sept. 14, Athens, Greece

Where, Who and What? @AIT Intelligent Affective Interaction ICANN, Sept. 14, Athens, Greece. Aristodemos Pnevmatikakis, John Soldatos and Fotios Talantzis Athens Information Technology, Autonomic & Grid Computing. Overview. CHIL AIT SmartLab Signal Processing for perceptual components

nysa
Download Presentation

Where, Who and What? @AIT Intelligent Affective Interaction ICANN, Sept. 14, Athens, Greece

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Where, Who and What?@AITIntelligent Affective InteractionICANN, Sept. 14, Athens, Greece Aristodemos Pnevmatikakis, John Soldatos and Fotios Talantzis Athens Information Technology, Autonomic & Grid Computing

  2. Overview • CHIL • AIT SmartLab • Signal Processing for perceptual components • Video Processing • Audio Processing • Services • Middleware • Easing application assembly

  3. Computers in the Human Interaction Loop • EU FP6 Integrated Project (IP 506909) • Coordinators: Universität Karlsruhe (TH) Fraunhofer Institute IITB • Duration: 36 months • Total Project costs: Over 24M€ • Goal: Create environments in which computers serve humans who focus on interacting with other humans as opposed to having to attend to and being preoccupied with the machines themselves • Key Research Areas: • Perceptual Technologies • Software Infrastructure • Human-Centric Pervasive Services

  4. AIT SmartLab Equipment • Five fixed cameras (one with fish-eye lens) • PTZ camera • NIST 64-channel array • 4 clusters of 4 inverted T-shaped SHURE microphone clusters • 4 tabletop microphones • 6 dual Xeon 3 GHz, 2 Gb PCs • Firewire cables & repeaters

  5. AIT SmartLab

  6. Perceptual Components

  7. Detection and Identification System Recognizer Frontality confidence Frontal verifier Detector Face normalizer Head detector Eye detector Tracker Confidence estimator Classifier confidence Weighted voting Face recognizer ID

  8. Unconstrained Video Difficulties

  9. Where and Who are the World Cup Finalists? • and European Champions?

  10. Tracking

  11. Tracking – Smart Spaces

  12. Tracking – 3D from Synchronized Cameras

  13. Tracking – Outdoors Surveillance • AIT system 2nd in the VACE / NIST surveillance evaluations

  14. Head Detection Frontal verifier Face normalizer Head detector Eye detector • Detection of head by processing the outline of the foreground belonging to the body Confidence estimator Tracker Weighted voting Face recognizer

  15. Eye Detection Frontal verifier Face normalizer Head detector Eye detector • Vector quantization of colors in head region • Detect candidate eye regions • Based on resemblance to skin, brightness, shape and size • Selection amongst candidates based on face geometry Confidence estimator Tracker Weighted voting Face recognizer

  16. Face Recognition from Video

  17. Effect of Eye Misalignment: LDA

  18. Effect of Eye Misalignment

  19. Classifier Fusion Illumination variationsPose variations • Classifier fusion addresses the fact that different classifiers are optimum for different recognition impairments

  20. Fusion Across Time, Classifiers and Modalities

  21. Face Recognition @ CLEAR2006

  22. Speaker ID @ CLEAR2006

  23. Audiovisual ID @ CLEAR2006

  24. Audiovisual Tracker • Information-theoretic speaker localization from mic. array • Accurate azimuth, approximate depth, no elevation • Moderate targeting of speaker’s face using a PTZ camera • Refine targeting by visual face detection

  25. Services

  26. Memory Jog • Memory Jog: • Context-Aware Human-Centric Assistant for meetings, lectures, presentations • Proactive, Reactive Assistance and Information Retrieval • Features-Functionalities • Sophisticated Situation Modeling / Tracking • Essentially Non-obtrusive Operation • Intelligent Meeting Recording Functionality • GUI runs also on PDA • Full Compliance to CHIL Architecture • Integration actuating devices (Targeted Audio, Projectors)

  27. Context as Network of Situations

  28. What Happened While I was Away?

  29. Middleware

  30. Virtualized Sensor Access

  31. CHIL Compliant Perceptual Components • Several sites develop site, room, configuration specific Perceptual Components for CHIL • Provide common abstractions in the input and output of the PC (black box) • Facilitate Component Exchange Across Sites & Vendors • Standardization commenced for Body Trackers • Continues to Face ID Components

  32. Architecture for Body Tracker Exchange Services complying to current API Common control API (CHILiX) Information retrieval Non-CHIL Compliant Body Tracker Transparent connection to sensor output Sensor abstraction

  33. Thank you!Questions?

More Related