Quantifying and Transferring Contextual Information in Object Detection

1. Quantifying and Transferring Contextual Information in Object Detection Professor:S. J. Wang Student : Y. S. Wang

2. Outline • Background • Goal • Difficulties in Usage of Contextual Information • Provided solutions • Another method: TAS • Experimental Results and Discussion • Conclusion and Future Direction

3. Background (I) • Only the properties of target object used in the detection task in the past. • Problem: Intolerable number of false positive

4. Background (I) • Only the properties of target object used in the detection task in the past. • Problem: Intolerable number of false positive

5. Background (II) • What else??? Contextual information!

6. Goal • Establish a model to efficiently utilize the contextual information to boost the performance of detection accuracy.

7. Difficulties(I) • Diversity of Contextual Information • There are may different types of context often co-existing with different degrees of relevance to the detection for the target object(s) in different images. • Terminology: • Things (e.g. cars and people) • Stuffs (e.g. roads and sky) • Scene (e.g. what happen in the image) • Thing-Thing, Thing-Stuff,Stuff-Stuffand Scene-Thing

8. Difficulties(II) • Ambiguity of Contextual Information • Contextual information can be ambiguous and unreliable, thus may not always have a positive effect on object detection. • Ex: Crowded Scene with constant movement and occlusion among multiple objects.

9. Difficulties (III) • Lack of Data for Context Learning • Not enough training data : • Over-fitting problem • Wrong degree of relevance • Ex: The contextual information of people on top of sofa can be more useful than people on top of grass.

10. Training Data Preparation & Notation Representation Training Image Base Detector (HOG) Candidate windows Positive sample: Red Negative sample: Green

11. Provided Solutions • A polar geometric descriptor for contextual representation. • A maximum margin context model (MMC) for quantifying context. • A context transfer learning model for context learning with limited data.

12. Polar Geometric Descriptor • Instead of traditional annotation based descriptor, here we use polar geometric descriptor to describe two kind of contextual information (Thing-Thing, Thing-Stuff). r :orientation b+1 :radial bins r*b+1 :patches 0.5σ, σ and 2σ :bin length Feature :HOG Patch representation: Bag of Words method using K-means with K = 100

13. Provided Solutions • A polar geometric descriptor for contextual representation. • A maximum margin context model (MMC) for quantifying context. • A context transfer learning model for context learning with limited data.

14. Quantifying Context(I) Risk function:

15. Quantifying Context(II) • Goal = Minimize the Risk function Minimize L equal to fulfill the following constraint Hard to be solved, could be replaced by

16. Quantifying Context(III)Maximum Margin Context Model Add some extra variables and constraints

17. Provided Solutions • A polar geometric descriptor for contextual representation. • A maximum margin context model (MMC) for quantifying context. • A context transfer learning model for context learning with limited data.

18. Context Transfer Learning • Two Cases: • Similar contextual information • Ex: Cars and motorbikes • Little in common in both appearance and context, but similar level of assistance provided by contextual information. • Ex: People and bikes

19. TMMC-I: Transferring Discriminant Contextual Information • Similar context provide the assistance on the learning of w.

20. TMMC-I: Transferring Discriminant Contextual Information • New Constraint: • Modified optimization function:

21. TMMC-II: Transferring the Weight of Prior Detection Score • Similar level of assistance, same weight

22. TMMC-II: Transferring the Weight of Prior Detection Score • New Constraint: • Modified optimization function:

23. Another Method: TAS

24. Another Method: TAS (I) Steps: Segmenting image into regions. Use base-detector to get the candidate patches. Establish the relationships between candidate patches and regions. Use the relationships to judge there is a target object in the patch or not.

25. Another Method: TAS (II) • Region clusters:

26. Another Method: TAS (III) • Examples of experiment:

27. Experimental Result and Discussion • Use four data sets for testing: • VOC 2005 • VOC 2007 • I-LIDS • FORECOURT

28. Experimental Result and Discussion

29. Experimental Result and Discussion

30. Experimental Result and Discussion • Context Transfer Learning Models:

31. Experimental Result and Discussion • Context Transfer Learning Models:

32. Conclusion and Future Direction • In this paper, the author proposes a contextual information model to quantify and select useful context information to boost the detection performance. • What can we do next? • HOG feature not suits for stuff (e.g. sky, road) • Automatic selection between TMMC-I, TMMC-II • Automatic selection between target object category and source category

33. Reference • Wei-Shi Zheng, Member, IEEE, Shaogang Gong, and Tao Xiang, ”Quantifying and Transferring Contextual Information in Object Detection ”, PAMI accepted. • GeremyHeitz, Daphne Koller, “ Learning Spatial Context: Using Stuff to Find Things”, ECCV 2008. • Youtube Search “Hard-Margin SVM”