1 / 33

3D LayoutCRF

3D LayoutCRF. Derek Hoiem Carsten Rother John Winn. Goal 1: Object Description. Object Description: Bounding Box Viewpoint Color Pose Subclass. Goal 2: Object Segmentation. Key Idea. Combine object-level and pixel-level reasoning. Recognition Requires Object-Level Reasoning.

sandra-townsend
Download Presentation

3D LayoutCRF

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D LayoutCRF Derek Hoiem Carsten Rother John Winn

  2. Goal 1: Object Description • Object Description: • Bounding Box • Viewpoint • Color • Pose • Subclass

  3. Goal 2: Object Segmentation

  4. Key Idea • Combine object-level and pixel-level reasoning

  5. Recognition Requires Object-Level Reasoning • Position • Shape/Size • Viewpoint/Pose • Style/Color

  6. Recognition Requires Object-Level Reasoning

  7. Solution: Window Detector? • 45 degree range of viewpoints • Minor scale/position variation

  8. What if we have a really good model?

  9. Recognition Requires Part-Level Reasoning • Propose good global model

  10. Recognition Requires Part-Level Reasoning • Propose good global model • Occlusions

  11. Context Requires Both Object and Part-Level Info • Size relationships require object model

  12. Context Requires Both Object and Part-Level Info • Surface relationships require occlusion info Not visibly sitting on ground Visibly sitting on ground

  13. Our Object/Part Model … T1 Tm Ti = { bounding box, viewpoint, color model, instance cost } h1 h2 h3 h4 hj object parts h5 h6 h7 h8 part consistency occlusions … … h9 h10 h11 hn x Extension from [Winn Shotton 2006]

  14. Modeling Viewpoint Parameterized by Bounding Box and Corner

  15. L F Training Annotation 3D Parts Model Assigning Parts from Model Training Image Assigned Parts

  16. Part Assignment Consistency

  17. Relabeling • Allowing slight deformations, relabel training data Training Image Original Labels New Labels

  18. Height Range Eight Viewpoint/Scale Ranges • Appearance (but not location) constant within each range

  19. Modeling Part Appearance • Template patches (normalized xcorr) • Intensity / Color Image Edges (DT)

  20. Modeling Part Appearance • Randomized decision trees • 25 trees, 250 leaf nodes • Once: • Learn structure on 50,000 object / 50,000 background pixels • For each appearance model: • Learn parameters on all pixels (850 LabelMe images)

  21. Inference Input Image

  22. Inference Input Image • Proposals • One per appearance model • Objects proposed by connected components

  23. Proposal Stage Model • CRF Inference (TRW-BP) h1 h2 h3 h4 hi object parts h5 h6 h7 h8 part consistency occlusions … … h9 h10 h11 hn x

  24. Inference Input Image Proposals • Refinement • One per proposal • Incorporate viewpoint, size information

  25. Refinement Stage Model T1 Ti = { bounding box, viewpoint } h1 h2 h3 h4 hi object parts h5 h6 h7 h8 part consistency occlusions … … h9 h10 h11 hn x

  26. Inference Input Image Proposals Refinement • Arbitration • Includes color model, instance penalty (graph cuts)

  27. Preliminary Results on UIUC • Trained on 20, tested on rest • Quantitatively comparable to best

  28. T1 h1 h2 h3 h4 h5 h6 h7 h8 … … h9 h10 h11 hn x Preliminary Results on UIUC Without Instance Cost With Instance Cost

  29. Preliminary Results on PASCAL’06 • 25 images • One proposal (viewpoint within 45 degrees, scale of 26-38 pixels)

  30. Preliminary Results on PASCAL’06

  31. Preliminary Results on PASCAL’06

  32. Preliminary Results on PASCAL’06 Without Color Model With Color Model

  33. Conclusion • Combined object-level and pixel-level reasoning • Object-level: Position/Size, Viewpoint, Color • Pixel-level: Part appearance, Occlusion reasoning • Good preliminary results

More Related