1 / 23

Giving Meanings to WWW Images

Giving Meanings to WWW Images. Heng Tao Shen Beng Chin Ooi Kian Lee Tan. Outline. Image Representation Model Semantic Measure Model Relevance Feedback Experiments. Background. Image: indispensable component in WWW 1 image = 1000 words WWW: rich resource of images Some 100 billions?

karly-dale
Download Presentation

Giving Meanings to WWW Images

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Giving Meanings to WWW Images Heng Tao Shen Beng Chin Ooi Kian Lee Tan ACM MM 2000, LA, USA

  2. Outline • Image Representation Model • Semantic Measure Model • Relevance Feedback • Experiments

  3. Background • Image: indispensable component in WWW • 1 image = 1000 words • WWW: rich resource of images • Some 100 billions? • Tradition: poor performance • Keywords • Content_based: no enough semantic • Like object, event, and relationship Not effective for images from WWW

  4. Cont • Semantics of embedded images in HTML • Image Title, ALT, Page Title, Image Caption -> ChainNet model • Similarity between query and image • List space model • Relevance feedback: • Improve precision further

  5. Weight ChainNet model • Lexical chain(LC) • A sentence that carries certain semantics by its words • 6 types of LC • TLC: Title Lexical Chain • PLC: Page Lexical Chain • ALC: Alt Lexical Chain • SLC: Sentence Lexical Chain • RSLC: Reconstructed Sentence Lexical Chain • CLC: Caption Lexical Chain

  6. Page Title Title ALT Caption 4 1 7 2 SLC: 1->2->3->4->5 RSLC: 1->2->8->9 CLC: 1->2->…->14 8 3 9 14 4 5

  7. Semantic measure model • Computing similarity between two LCs • List space model Where ei and ej are matched terms in list 1 and list 2 respectively.

  8. Semantic measure model • Match scale: closeness in view of match order Here v1 and v2 represent the children of first and second original lists respectively. Where v2j is the matched word in v2 for v1i in v1 Inspired from the angle between two vectors

  9. Semantic measure model LC Match Level(LC1, LC2): the number of distinct matched words by two LCs • Match level threshold: The minimum match level for LC to keep its original semantic • LC Semantic similarity: similarity(list1, list2) in its LC Match Level

  10. Semantic measure model Image Match Level(image, query) = MAX ( TLC.weight * LCMatchLevel( TLC, QLC), ALC.weight * LCMatchLevel( ALC, QLC), PLC.weight * LCMatchLevel( PLC, QLC), SLC.weight * LCMatchLevel( SLC, QLC), RSLC.weight *LCMatchLevel( RSLC, QLC), CLC.weight * LCMatchLevel( CLC, QLC) )

  11. Relevance Feedback • Semantic Accumulation • Choose one best image as feedback • Accumulate the previous feedback images’ semantics to construct a new QLC • Results are more close to the specific image selected • More noise

  12. New query QLC Image Title Page Title Image Caption Image ALT Last feedback image Semantic accumulation Weight F/Q ChainNet

  13. Semantic Integration and Differentiation • Semantic Integration and Differentiation • Choose several Good and Bad images as feedback • Integrate Good semantics to construct new query • Differentiate irrelevant images by Bad images • Results are more diverse and less noise

  14. Most related LC New query QLC LC1 LC2 LC3 LCi Good feedback images Image 1 Image 2 Image 3 Image i Semantic integration and differentiation Similar weight F/Q ChainNet

  15. Experiments • Set up • Web crawler to collect images • 5232 images from over 2000 URLs • 12 general queries

  16. Tuning the LC Weights

  17. Tune the match level MatchLevel Threshold= coef * query.length()+ constant

  18. Impact of match scale • explore the importance of match order

  19. Feedback Mechanisms

  20. Feedback Mechanisms One-step feedback of Accu and I&D for Q1.

  21. Conclusion • Inner semantic structure of surrounding text is explored well for good precision achievement • ChainNet model and list space model work well • RF techniques help to return more accurate results

  22. Future work • Explore LC meanings by AI technique • Extract semantics from visual content, then integrate with our system to construct a more advanced semantic retrieval system • Object-oriented detection

  23. DEMO ON THURSDAY SEE YOU THEN… http://efoto.geofoto.com

More Related