1 / 15

Discovering Collocation Patterns: from Visual Words to Visual Phrases

Discovering Collocation Patterns: from Visual Words to Visual Phrases. Junsong Yuan, Ying Wu and Ming Yang CVPR’07. Discovering Visual Collocation. An exciting idea: detour.

orenda
Download Presentation

Discovering Collocation Patterns: from Visual Words to Visual Phrases

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discovering Collocation Patterns:from Visual Words to Visual Phrases Junsong Yuan, Ying Wu and Ming Yang CVPR’07

  2. Discovering Visual Collocation

  3. An exciting idea: detour • Related Work: J. Sivic et al. CVPR04, B. C. Russellet al. CVPR06, G. Wang et al. CVPR06, T. Quack et al. CIVR06, S. C. Zhu et al. IJCV05, …

  4. Confrontation • Spatial characteristics of images • over-counting co-occurrence frequency • Uncertainty in visual patterns • Continuous visual feature quantized word • Visual synonym and polysemy

  5. Our Approach

  6. Selecting visual phrases • Visual collocations may occur by chance • Selecting phrases by a likelihood ratio test: • H0: occurrence of phrase P is randomly generated • H1: phrase P is generated by a hidden pattern • Prior: • Likelihood: • Check if words are co-located together by chance or statistically meaningful

  7. Discovery of visual phrases Closed FIM Frequent Word-sets ( |P|>=2 ) AB AE CD CE DE AF BE BF CDE ABF ABE pair-wise student t-test ranked by L(P) Group Database likelihood ratio AB 15.7 14.3 12.2 10.9 9.7 AF Visual Phrase Lexicon (VPL) ABF BF CD

  8. Frequent Itemset Mining (FIM) • If an itemset is frequent  then all of its subsets must also be frequent

  9. Phrase Summarization • Measuring the similarity between visual phrases by KL-divergence Yan et al., SIGKDD 05 • Clustering visual phrases by Normalized-cut

  10. Pattern Summarization Results Face database: summarizing top-10 phrases into 6 semantic phrase patterns Car database: summarizing top-10 phrases into 2 semantic phrase patterns

  11. Partition of visual word lexicon • Metric learning method: • Neighborhood component analysis (NCA). Goldberger, et al., NIPS05 • improve the leave-one-out performance of the nearest neighbor classifier

  12. Evaluation • K-NN spatial group: K=5 • Two image category database: car (123 images) and face (435 images) • Precision of visual phrase lexicon • the percentage of visual phrases Pi ∈ Ψ that are located in the foreground object • Precision of background word lexicon • the percentage of background words Wi ∈ Ω−that are located in the background • Percentage of images that are retrieved:

  13. Results: visual phrases from car category Visual phrase pattern 1: wheels different colors represent different semantic meanings Visual phrase pattern 2: car bodies

  14. Results: visual phrases from face category

  15. Comparison

More Related