Object Recognition as Ranking Holistic Figure-Ground Hypotheses

1. Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and CristianSminchisescu

2. Outline Introduction Method Overview Segment Categorization Segment Post-Processing Experiment Conclusion

3. Introduction Object detector : Top-down approaches

4. Introduction Semantic segmentation results produced by our algorithm

5. Introduction Ideally segment Can model entire object At least sufficiently distinct parts of them

7. Introduction Constrained Parametric Min Cuts algorithm (CPMC) [6]

8. Method Overview CPMC Number of segment This paper focus

9. Method Overview Recognition framework Segment categorization Segment post-processing

10. Segment categorization Scoring function Sort Combine high-rank segment

12. Segment post-processing COW

13. Segment Categorization Multiple Segment and Features Learning Scoring Functions with Regression Learning the Kernel Hyperparameters Compare with Structural SVM

14. Multiple Segment and Features Model object appearance： Extracted four bag of words of SIFT Two on foreground Two on Background, aim to improve recognition

15. Multiple Segment and Features Encode shape information： A bag of word of local sharp contexts [2] : measure similarity between shapes Three pyramid HOGs [5] : classifying images by the object categories they contain

16. Multiple Segment and Features Chi-square kernel ： Computed from each histogram feature and use a weighted sum of such kernel for regression

17. Learning Scoring Functions with Regression : Image I with ground truth segments : Segmentation algorithm provides a set of segment : Denote the K object categories : Indicator function

18. Learning Scoring Functions with Regression Quality function : Measure overlap with all denote the value for and is the maximal overlap with ground truth segments belonging to , and do not appear

19. Learning Scoring Functions with Regression Learn the function for each : use nonlinear SVR(Support Vector Regression) to regress against ,the features extracted from

20. Learning Scoring Functions with Regression Use kernel trick : : support vector from training set : obtained by the SVR optimizer : maximal score of the segment : final class of the segment

21. Learning the Kernel Hyperparameters Fundamental equation (3) is infeasible to estimate all kernel hyperparameter via grid search Use subset of data comprised segments that best overlap each ground truth segment

22. Compare with Structural SVM The structural SVM(in [3]) formulation for sliding window prediction is :

23. Connections with Structural SVM Our algorithm VS Structural SVM Structural SVM score the bounding box and Our algorithm score the segment Important advantage Guarantee the highest rank for the ground truth Correct ranking for all segment

24. Segment Post-Processing Simple decision rule : avoid the post-processing and direct choose the segment , cannot detect multiple objects Our methodology : weighted consolidation of segment and sequential interpretation strategy

25. Segment Post-Processing

26. Segment Post-Processing To decide which segments to combine Consider segment with intersection > 0.75 for combination

27. Segment Post-Processing Highest-scoring segment as seed Group segments that intersect it Generated a final mask Proceed with the next higher rank segment Choose segment that are not overlapping with 3

28. Segment Post-Processing Generate the score for the pixels in the mask by (9), only pixels with score > 0.65 are displayed in the mask.

29. Experiments Classification : Caltech-101 Detection : ETHZ Shape classes Segmentation : VOC 2009

30. Classification

31. Classification

32. Detection

33. Detection

34. Detection

35. OWT-UCM Masks

36. Segmentation Bounding box

37. Conclusion CPMC Categorization Post-processing Achieve good performance Future work : improve the scalability to be able to process hundreds of thousands of image