1 / 22

Classifying Visual Objects Regardless of Depictive Style

Classifying Visual Objects Regardless of Depictive Style. Qi Wu, Peter Hall Department of Computer Science University of Bath. Summary. Conventional Comp.Vis . classifiers do not generalise well across depictive styles .

varsha
Download Presentation

Classifying Visual Objects Regardless of Depictive Style

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath

  2. Summary • Conventional Comp.Vis. classifiers do not generalise well across depictive styles. • We propose new visual class model,one invariant to depictive style. • Experiments validate our model.

  3. People can see objects in a wide variety of depictive styles. Photos Artwork

  4. Literature Gap: BoW does not generalise across depictive styles 47% (Dense SIFT) Photos Artwork 51% (Dense SIFT) Photos Artwork

  5. Our solution: A new Visual Class Model that does generalise across styles. 47% (Dense SIFT) Photos Artwork 64% 51% (Dense SIFT) Photos Artwork 67%

  6. A New Visual Class Model • We assume an object class is characterised by: • the qualitative shape of object parts, • the structural arrangement of those parts. • A hierarchical graph model per image: • coarse-to-fine representation (layered), • nodes labelled by primitive shapes, • abstracting region shape brings greater robustness. • arcs labelled with displacement vectors • Median graph models: • aggregates models from several instances, • single class model.

  7. Making a VCM (a): An input collection. (b): Probability maps for each input image, and graph models for each map. (c): The median graph model for the whole class. (d): The refined median graph as the final class model

  8. A schematic VCM • A hierarchical description • Berkeley segmentation • Filtering process using cLge • A graph • Arcs at same level denote touching neighbours. • Arcs between layers link parent – children. • Nodes label • A 6-elements probability vector. • The probability that a region belongs to a given prime shape class.

  9. Prime Shapes,BMVC 2012

  10. Prime Shapes • Does a set of elementary planar shapes appear commonly in the world ? • Art provides strong anecdotal evidence “yes” • 20th century Western Art --- Cubism

  11. Determine Prime Shapes • A fully unsupervised framework

  12. Determine Prime Shapes

  13. Back to our model…

  14. Build graphs, one for each image Left: graph model. Right: Object broken in primitive shapes

  15. Compute an initial Visual Class Model • Median Graph • First compute the graph distance between each pair. • Using the distance matrix to embed graph into a vector space • Compute the Euclidean Median of all the data points. • Transfer the median vector back to graph using a state-of-art method proposed in [Ferrer and Valveny, 2008]

  16. Refine the Visual Class Model The initial model contains nodes and arcs that derive from visual clutter in back ground of images in the training set • Refine the model • Match the median back into each training image. • Count the number of times a given node in the model appears in the training data. • Delete all nodes below a frequency threshold., which is computed via maximising matching score.

  17. Some Examples

  18. Experiments • Compare with other two methods • PHOW features (Dense SIFT) [Bosch and Zisserman, ICCV 2007] • Local PAS features [Ferrari and Jurie, IJCV 2010] • Structure Only [Bai and Song, CVIU 2011]

  19. Results

  20. Conclusions • It’s possible to learn models of objects classes that generalise across depictive styles. • Many applications are promised. • Just a first step • Simplify the model, still too much nodes and arcs. • Time consuming. • Additional labelling • Move to object localisation.

  21. Questions?

  22. One application of Prime shapes

More Related