1 / 29

NTT Visit: Image Database Retrieval Variable Viewpoint Reality

Join Professor Paul Viola and his collaborators for a day of exploring the latest advancements in image database retrieval and variable viewpoint reality. Topics include face detection and recognition, 3D reconstruction of people, and automatic camera calibration.

spataro
Download Presentation

NTT Visit: Image Database Retrieval Variable Viewpoint Reality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NTT Visit: Image Database Retrieval Variable Viewpoint Reality Professor Paul Viola Collaborators: Professor Eric Grimson, Jeremy De Bonet, John Winn, Owen Ozier, Chris Stauffer, John Fisher, Kinh Tieu, Dan Snow, Tom Rikert, Lily Lee, Raquel Romano, Janey Hshieh, Mike Ross, Nick Matsakis, Jeff Norris, Todd Atkins Mark Pipes

  2. Overview of Visit • Morning: Image Database Retrieval • Gatekeeper: Face detection and recognition • Complex Feature Image Database Retrieval (Tieu) • Flexible Template Retrieval (Yu) • Interlude • Video/Audio Source Separation (Fisher) • Mathematical Expression Recognition (Matsakis) • Lunch • Visit Prof. Brooks lab

  3. Overview of Visit - 2 • Afternoon: Variable Viewpoint Reality • Real-time 3D reconstruction of people (Snow) • Automatic camera calibration (Snow + Lee) • Tracking of articulate human models (Lee + Winn) • Modeling of human dynamics (Viola + Fisher)

  4. Gatekeeper:Receptionist & Security • Greet guests • Direct people to their destinations • Recognize employees • Turn back unauthorized visitors

  5. Gatekeeper in action … Gatekeeper Movie

  6. Gatekeeper is a constant observer… Professor Paul Viola

  7. Detecting faces is very difficult

  8. Detecting and Recognizing Faces • Key Difficulty: Variation in Pose • State of the art: generalized templates • Neural Networks / Deformable Templates / etc. • Templates have difficulty with pose variation… • Rotation, scale, complex deformation • Must reduce the dependence on relative pose. • Approach: Detecting people as a statistical distribution of multi-scale features

  9. Statistical Distribution of Multi-scale Features The distribution of multi-scale features determines appearance Wavelet Pyramid

  10. A multi-scale feature associates many values with each pixel in the image Multi-scale Wavelet Features

  11. Discrimination via Cross Entropy IMODEL Cross Entropy ITEST

  12. Motivation: Finding vehicles in clutter BTR70-C71 T72-132 Supported by Darpa: IU/ATR initially MSTAR Extension

  13. Can also be used for segmentation…

  14. Original Texture The multi-scale statistical model can be used to generate new example textures Synthesis Results

  15. Synthesis Procedure Step 1: Build analysis pyramid 2x2 64x64 Input Image Note: We are using only the Gaussian pyramid here! Normally we use an oriented pyramid...

  16. Synthesis Procedure Step 2: Build synthesis pyramid

  17. Synthesis Procedure Step 2a: Fill in the top... Pixels are generated by sampling from the analysis pyramid.

  18. Synthesis Procedure Step 2b: Fill in subsequent levels Pixels are generated by conditional sampling (dependent on the parent).

  19. Synthesis Procedure Finish the pyramid Decisions made at low resolutions generate discrete features in the final image.

  20. Detection Results Non-face test images Web face test images

  21. 1000 bins or less! Pruning the density estimator Reduce the number of bins through clustering Result: Detection/Classification is faster than template correlation

  22. Key facial features - determined automatically - located automatically Multi-scale features which are come from the face model can be automatically detected for many individuals

  23. Another key feature

  24. New Face Recognition Algorithm • Measure the occurrence and location of “key” facial features. • Facial identity depends both on the types of features and their location. • Relation to Active Search… • Match measure is a histogram of multiscale features • Like color histogram, Active Search can be used...

  25. Presentation on Image Database Technology

More Related