1 / 22

A Critical View of context

A Critical View of context. Biologically Inspired Models of Vision course. Alexandru Rusu , Guillaume Lemaître , Isabel Rodes and Oscar Ramos. Introduction. Extraction Low Level Image Features Extraction Semantic Image Features Building the Context Features

draco
Download Presentation

A Critical View of context

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Critical View of context Biologically Inspired Models of Vision course AlexandruRusu, Guillaume Lemaître, Isabel Rodesand Oscar Ramos

  2. Introduction • Extraction LowLevel Image Features • Extraction Semantic Image Features • Building the ContextFeatures • Experiments and Results • Improvements [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  3. Introduction [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  4. Low Level Image features ExtractionOverview • Downsize the images to 60 × 80 pixels • Extractcolor information • Extract texture information • Extract global position information [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  5. Low Level Image features ExtractionColor Features extraction L* Component RGB to CIE L*a*b* a* Component RGB image b* Component [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  6. Low Level Image features ExtractionColor Features extraction L* Component L* Component RGB image a* Component a* Component b* Component b* Component [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  7. Low Level Image features Extraction ColorFeatures [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  8. Low Level Image features ExtractionTexture features extraction Polarity Anisotropy RGB image Contrast [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  9. Low Level Image features Extraction ColorFeatures Texture Features [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  10. Low Level Image features ExtractionPosition features extraction RGB image [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  11. Low Level Image features Extraction ColorFeatures Texture Features Position Features [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. [2] - Carson, C., Belongie, S., Greenspan, H., and Mali, J. 1998. Colorand texture-based image segmentation using EM and its application to context-based image retrieval, ICCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  12. Semantics features Extraction - Building • - Tree - Road • - Sky [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. • Semantic Layers used: • Example (for building): 1=building, 0=no building Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  13. Semantics features Extraction True Semantic Label Learned Semantic Label [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. In Test images: no ground truth → Use 4 SVM binary classifiers (input: low-level feature image) Training set: 10 000 samples per category Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  14. Semantics features Extraction [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. ROC curve for the SVM classifiers Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  15. Building the context features • 40 samples: • (40)(20) = 800 dimensional context feature per pixel Green: size of car Red: size of pedestrian [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. • Image has been converted to 20 layers: - 4 binary semantic features - 3 color features - 3 texture features - 10 global position features • Data sampled at 8 orientations and radii of 3,5,10,15,20 pixels Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  16. Experiments and results Fidelity of semantic information Empirical semantic features: Four SVMs Four classes: building, tree, road, sky Three features: colour, texture, position Training and testing Cross-validation and ROC [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  17. Experiments and results Performance of the context detector Comparison with ROC curves of true high-level context detector and appearance detector Appearance detector outperforms the context-based [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  18. Experiments and results Relative importance of context features Comparison of four context classifiers Low-level feature-based detection only marginally improved by addition of semantic features [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  19. Experiments and results Relative importance of context features Testing of possible overlap of context with target object Low-level and high-level classifiers at d ϵ{3,5,10,15,20} Semantic features only important at farther distances [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  20. Improving Object Detection with Context [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. Dataflow of the a rejection cascade Tune the thresholds THC and THA. Different ROCs Validation set of 200 images Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  21. Improving Object Detection with Context • Three different objects • Horizontal lines indicate the performance of the system with no context • The marks the selected parameters for the system [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. Tuning the context threshold The ROCs of full system performance Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

  22. Conclusions [1] - Wolf, L., and Bileschi, S. 2006. A criticalview of context, IJCV. An effective context detection system Rejection cascade architecture Importance of contextual cues Good performance when the appearance information is weak (critically low resolution and very noisy images) Ways of extracting context information Critical View of Context – Alexandru Rusu, Guillaume Lemaître, Isabel Rodes and Oscar Ramos

More Related