Semantic Contours from Inverse detectors

1. Semantic Contours from Inverse detectors

2. Outline • Introduction • Inverse detector • Localizing semantic contours • Experiments • Conclusion

3. Localizing and classifying category-specific object contours in real world images Low-level contours (No-class specific) Problem [ J.Malik ,IEEE Trans. on PAMI 2011 ] Class specific contours

4. Localizing and classifying category-specific object contours in real world images Naive Solution • Using detector outputs will result is contours from surrounding context • To avoid this problem they propose the inverse detector

5. Outline • Introduction • Inverse detector • Localizing semantic contours • Experiments • Conclusion

6. Given localized contours I and object detector , the Inverse Detector produces the object contour image The Inverse Detector Inverse detector • I – image • G – output of contour detector • Gij – scores the likelihood of a pixel (i,j) lying on a contour • R1, ..., Rl – l activation windows of the detector • sk – score corresponding to each activation window Rk • - Feature vector for pixel (i, j)

7. Each detector window divided into S spatial bins • Contours are binned into O orientation bins • For a pixel (i, j), for an activation window RK, assigned into one of bins (from SO) • Feature Vector at a location (i, j), and detector RK: Feature Vector • en: an SO-dimensional vector with 1 in the nth position and 0 otherwise • index of the bin into which the pixel (i, j) falls • Feature vector for pixel (i, j): • weighted sum of across all the activation windows

8. Inverse detectors is of the following form: • where, learn weight vector using a linear SVM with these features Inverse detectors Inverse detector • Complete system: use of inverse detectors for localizing semantic contours • Using poselet types object detectors[1] • bottom-up contour detector[2] [1]-Detecting people using mutually consistent poselet activation. L. Bourdev et.al., ECCV-2010 [2] - Contour detection and hierarchical image segmentation. P. Arbelaez et.al, PAMI-2011

9. Outline • Introduction • Inverse detector • Localizing semantic contours • Experiments • Conclusion

10. System has two stages • traininverse detectors for each poselet types • let Pposelets corresponding to category C be • combine output of these inverse detectors to produce category-specific contours • Stage 1: train inverse detectors (of the following form) for each poselet (as discussed previously) Localizing semantic contours using inverse detectors • Stage 2: combining the outputs of each of these inverse detectors • Train a linear SVM (with classifying each pixel belonging to object contour or not) • Features: concatenate the outputs of the inverse detectors corresponding to each of the poselet type

11. Previous model: considers each category independently. • In this model: combine information from across categories • Propose two methods Method 1 • First level: Train contour detector for each category separately • Second level: Train on the outputs of these contour detectors Combining information across categories • Feature vector at the second level: Method 2 • Only One level: Train on the features which are the outputs of the inverse detectors corresponding to the poselets of all categories • Feature vector this level:

12. Outline • Introduction • Inverse detector • Localizing semantic contours • Experiments • Conclusion

13. Experiments

14. PASCAL VOC2011, 20categories , 2223images • 8498 training images and 2820 test images Semantic Boundaries Dataset (SBD)

15. Show precision-recall curve for a detector producing soft output, parameterized by the detection score • Report two summary statistics: • Average precision (AP) • maximal F-measure (MF) = (F = 2PR/(P+R) • Precision: fraction of true contours among detections • Recall: fraction of ground-truth contours detected Benchmark precision and recall are practically zero

16. Experiments

17. Outline • Introduction • Inverse detector • Localizing semantic contours • Experiments • Conclusion

18. Conclusion • Three distinct contributions • A new task • A new annotated dataset • A semantic contour detector

19. Thank you 