1 / 16

Recovering Human Body Configurations: Combining Segmentation and Recognition

Recovering Human Body Configurations: Combining Segmentation and Recognition. Greg Mori, Xiaofeng Ren, Alyosha Efros and Jitendra Malik. Problem. Input image. Stick figure. Support masks. Single image: No background subtraction. Human Figures in Still Images.

abraham
Download Presentation

Recovering Human Body Configurations: Combining Segmentation and Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Recovering Human Body Configurations: Combining Segmentation and Recognition Greg Mori, Xiaofeng Ren, Alyosha Efros and Jitendra Malik

  2. Problem Input image Stick figure Support masks Single image: No background subtraction

  3. Human Figures in Still Images • Detection of humans is possible for stereotypical poses • Standing • Walking (Mohan, Papageorgiou, and Poggio PAMI 01; Viola, Jones and Snow ICCV 03) • But we want to do more • Wider variety of poses • Localize joint positions

  4. Why Is This Hard? • Variety of poses • Clothing • Missing parts • Small support for parts • Background clutter

  5. Previous Work • Exemplar approaches • Shape Contexts (Mori and Malik ECCV 2002) • Order Structure (Sullivan and Carlson ECCV 2002) • LSH (Shakhnarovich et al. ICCV 2003) • Part-based approaches • Rectangles (Ioffe and Forsyth, IJCV 2001) • Corners (Song et al., PAMI 2003) • Dynamic programming (Felzenszwalb and Huttenlocher, CVPR 2000)

  6. Approach:Unifying Segmentation and Recognition • Bottom-up • Detect half-limbs and torsos • Top-down • Assemble parts into human figure

  7. SEGMENTS • Segmentation • How many? SUPERPIXELS Why Segmentation for Recognition? • Window-scanning (e.g. face detection)

  8. Limb/Torso Detectors • Learn limb and torso detectors from hand-labeled data • Cues: • Contour • Average edge strength on boundary • Shape • Similarity to rectangle • Shading • x,y gradients, blurred • Focus • Ratio of high to low frequency energies

  9. Islands of Saliency • “Partial configurations” • 3 half-limbs plus a torso • Combinatorial search over sets of limbs and torsos configurations

  10. Pruning Partial Configurations • Many partial configurations are physically impossible • Prune using global constraints (not a tree) • Proximity • Relative widths • Maximum lengths • Symmetry in clothing colour • Results in 1000 partial configurations

  11. Completing Configurations • Use superpixels to complete half-limbs • 2 or 3-limbed people • Sort partial configurations • Use limb, torso, and segmentation scores • Extend missing limb(s) of best configurations

  12. LU ARM RL ARM LL LEG RL LEG LL LEG RL LEG

  13. Results I

  14. Results II Rank 3 Rank 3

  15. Addressing the Challenges • Variety of poses • Parts-based approach, no restrictive priors • Clothing • Shape, shading, edge cues • Missing parts • Start with “islands of saliency” • Small support for parts • Start with “islands of saliency” • Background clutter • Global constraints on relative sizes of parts, colour

  16. All segmentations are wrong, but some segmentations are useful!

More Related