1 / 12

Matthew Simpson, Md Mahmudur Rahman, Dina Demner-Fushman, Sameer Antani, George R. Thoma

Text- and Content-based Approaches to Image Retrieval for the ImageCLEF 2009 Medical Retrieval Track. Matthew Simpson, Md Mahmudur Rahman, Dina Demner-Fushman, Sameer Antani, George R. Thoma Lister Hill National Center for Biomedical Communications,

ugo
Download Presentation

Matthew Simpson, Md Mahmudur Rahman, Dina Demner-Fushman, Sameer Antani, George R. Thoma

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Text- and Content-based Approaches to Image Retrieval for the ImageCLEF2009 Medical Retrieval Track Matthew Simpson, Md Mahmudur Rahman, Dina Demner-Fushman, Sameer Antani, George R. Thoma Lister Hill National Center for Biomedical Communications, National Library of Medicine, NIH, Bethesda, MD, USA CLEF 2009

  2. Retrieval tasks and approaches • ITI project long term goal • Find a way to combine image and text features so that the whole is greater than the sum of its parts • Ad-hoc image retrieval • Text-based • Image content-based • Automatic mixed • Relevance feedback mixed • Case-based document retrieval • Text-based

  3. Text-based approach • Indexing: • Create image documents for ad-hoc image retrieval • Create surrogate documents for case-based retrieval • Index using Essie • term normalization using the SPECIALIST Lexicon • query expansion based on UMLS synonymy • term weighting based on location in the document • Phrase-based search

  4. Text documents • Image document • Title and caption provided by organizers • Mention extracted from paper • MEDLINE citation (abstract +MeSH) • PICO frame of the caption + image modality (structured caption summary) • Surrogate document • MEDLINE citation • caption, mention, and structured caption summary of each image contained in the article

  5. Text retrieval • PICO-based structured query and case representation • <topicID>19</topicID> <description>Crohn's disease CT</description> • <modality essieExp="false">ct</modality> <modSyn>c.a.t.</modSyn><modSyn>cat</modSyn><modSyn>computerised axial tomography</modSyn>…. • <cond essieExp="true">Crohn's disease</cond><condPN>crohn disease</condPN><condSyn>Regional enteritis</condSyn> <condSyn>eleocolitis</condSyn><condSyn>Cicatrizing enterocolitis</condSyn><condSyn>granulomatous enteritis</condSyn><condSyn>INFLAMMATORY BOWEL DISEASE</condSyn><condSyn>regional enterocolitis</condSyn> …

  6. CBIR - Image feature representation • Concepts - color and texture patches from local image regions • Low-level global features • Color (Color Layout Descriptor, MPEG-7) • Edge (histogram of local edge distribution and direction) • Texture (grey level co-occurrence matrix) • Average grey level (256-dimensional vector of blocks in image normalized to gray-level 64x64) • Lucene (LIRE)-based Color Edge Direction Descriptor and Fuzzy Color Texture Histogram

  7. Image similarity computation • Category-specific • Determine image category (training set of 5000 images manually assigned to 32 mutually exclusive categories) • Use category-specific weights in linear similarity matching • Relevance feedback • Feature weights updated using images judged relevant

  8. Combining text and image • Based on text search results, • Compute mean vector of top 5 retrieved images, use as input to category-specific retrieval • Select 3-5 relevant images manually, use as input to category-specific retrieval • Re-rank text retrieval results using visual retrieval scores • Provide feedback using all retrieval results, • expand query using image documents • Pad selected relevant images with new retrieval results

  9. Relevance Feedback

  10. Results category- specific RF text re- ranked BRF RF RF+QE case-based visual mixed

  11. Image-text search engine

  12. Thank you! Questions?

More Related