1 / 10

Multimedia Information Retrieval

Multimedia Information Retrieval. Unlike alphanumeric data, multimedia data do not have any semantic structure Achieving symmetry between annotation and query is difficult Retrieval is based on similarity between query and stored information instead of exact match

tteresa
Download Presentation

Multimedia Information Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Multimedia Information Retrieval • Unlike alphanumeric data, multimedia data do not have any semantic structure • Achieving symmetry between annotation and query is difficult • Retrieval is based on similarity between query and stored information instead of exact match • Stored information is represented using indexing

  2. IR Model • Information is preprocessed to extract features and semantic contents • Indexed based on these features and semantics • User’s query is processed and main features are extracted • Query’s features are then compared with features or index of each information item in the database • Information item whose features are similar to those of the query are retrieved and presented to the user

  3. Design Issues • Indexing • a mechanism that reduces the search space of an operator without losing any relevant information • Similarity Computation • easy to compute and should conform to human judgement

  4. Performance Measures • Retrieval speed, recall, precision • Recall measures the ability of retrieving relevant information items from the database • defined as the ratio between the number of retrieved relevant items and the total number of relevant items in the database • Precision measures retrieval accuracy • defined as the ratio between the number of retrieved relevant items and the number of total retrieved items • Recall and precision are usually considered together • high recall and low precision • high precision and low recall

  5. Text Retrieval • Text may be used to annotate other media such as audio, images and video and conventional IR techniques used to retrieve multimedia information • Boolean IR systems or text-pattern search systems • Substantial effort is spent in analyzing the contents of the documents and in generating keywords and indices • Boolean queries are keywords connected with logical operators (AND, OR, NOT)

  6. File Structures • Flat files • Inverted files • for each term a separate index is constructed that stores the document identifiers for all documents containing the term • each term and the document IDs containing the term are organized into one row • searching and retrieval is fast because only rows containing the query terms need to be retrieved and there is no need to search the whole database

  7. Extensions • Nearness parameters used in query specification help define the topic more precisely and therefore increase probable relevance of the retrieved item • Within Sentence and Adjacency specification in queries • Term location information is included in the inverted file • Term i : document id, paragraph no., sentence no., word no. • For example, if an inverted file has the following entries: information: R99, 10, 8, 3; R155, 15, 3, 6; R166, 2, 3,1 retrieval: R77, 9, 7, 2; R99, 10, 8, 4; R166, 10, 2, 5

  8. Indexing • Stop words -- grammatical functional words, such as “of,” “the,” and “a.” • Stemming -- reducing words to a common root form • Thesaurus -- list of synonyms • Weighting -- term significance derived from occurrence frequency within a document and among different documents

  9. Relevance Feedback • Query modification • terms occurring in documents previously identified as relevant are added to the original query or the weight of such terms is increased • terms occurring in documents previously identified as irrelevant are deleted from the query or the weight of such terms is reduced • Document modification • terms in the query, but not in the user-judged relevant documents, are added to the document index list with an initial weight • weights of index terms in the query and also in relevant documents are increased by a certain amount • weights of index terms not in the query but in the relevant documents are decreased by a certain amount

  10. Audio Search and Retrieval • Keywords can be highly subjective because of a different perspective or even a different taxonomy • Hard to browse directly since it must be heard in real-time (unlike video which can be keyframed) • Two categories : Speech and Non-speech • with speech, indexing and retrieval is based on obtaining spoken words either manually or by speech recognition technique • with non-speech, indexing and retrieval may be based on text annotation (but will it help a query like “find the first occurrence of the note G-sharp.”)

More Related