1 / 9

Experiments on Using Semantic Distances Between Words in Image Caption Retrieval

Experiments on Using Semantic Distances Between Words in Image Caption Retrieval. Alan F. Smeaton and Ian Quigley School of Computer Applications Dublin City University. Presenter: Cosmin Adrian Bejan. IR implementation - traditional approach. Represent:

jace
Download Presentation

Experiments on Using Semantic Distances Between Words in Image Caption Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experiments on Using Semantic Distances Between Words in Image Caption Retrieval Alan F. Smeaton and Ian Quigley School of Computer Applications Dublin City University Presenter: Cosmin Adrian Bejan

  2. IR implementation - traditional approach • Represent: • a user query = a bag of query terms • document = a bag of index terms • Compute: • a degree of similarity between a document and a query based on the overlap or number of query terms in common between them.

  3. Problems in IR implementation • caused by • same words describing different things (“bar”, “bank”) • different words describing same thing (“stomach pain” – “belly ache”) • natural language is fraught with ambiguities at all levels leading to multiple interpretations of words, phrases, etc. • Common way to address these problems: query expansion • The approach in this paper: when computing the degree of similarity between query and document instead of basing similarity on the terms in common between the two incorporate a quantitative measure of the semantic similarity between index terms into the measure.

  4. Measuring semantic distance between words • knowledge base – hierarchical concept graphs (HCGs) automatically constructed from WordNet • The similarity of two classes or synsets: • Computing the similarity between two word senses (nouns) can only be done if both are in the same HCG, otherwise they are regarded as being dissimilar. information content of the class ci P(ci) the class probability of class ci

  5. Experimental Set-up • Hand-caption 2714 images • Manually disambiguate polysemous words in caption • Manually built a collection of 60 queries • Compute various query-caption similarity measure using word-word semantic distances.

  6. Retrieval Strategies [1-2] • Notation • query Q={q1, q1, … qm}. • caption C={c1, c1 … cn} where a qi or a cj is the original term used only as a representation for its synset. • Sim(ti, tj) is the similarity between the sense-disambiguated form of two terms ti and tj. • Run1 • Run2 straightforward statistically-based tf*IDF match between the word forms or strings, i.e. not using word sense disambiguated captions or queries. where terms in caption in query are both expanded to include other word strings from their sense disambi-guated sysnsets (query expansion).

  7. Retrieval Strategies [3-5] • Run3 • Run4 • Run5 when considering different threshold values for each HCG, given that there is a concentration of usage of concepts from some HCGs (like entity) and hardly any use of others (like shape).

  8. Retrieval Strategies [6-8] • Run6 • Run7 • Run8

  9. Experimental Results

More Related