1 / 19

CLEF 2005: Multilingual Retrieval by Combining Multiple Multilingual Ranked Lists

CLEF 2005: Multilingual Retrieval by Combining Multiple Multilingual Ranked Lists. Luo Si and Jamie Callan Language Technology Institute, School of Computer Science Carnegie Mellon University CLEF 2005. Task Definition. Multi-8 Two Years On: multilingual information retrieval

silver
Download Presentation

CLEF 2005: Multilingual Retrieval by Combining Multiple Multilingual Ranked Lists

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CLEF 2005: Multilingual Retrieval by Combining MultipleMultilingual Ranked Lists Luo Si and Jamie CallanLanguage Technology Institute, School of Computer ScienceCarnegie Mellon University CLEF 2005

  2. Task Definition • Multi-8 Two Years On: • multilingual information retrieval • Multi-8 Merging Only: • participants merge provided bilingual ranked lists

  3. Task 1:Multilingual Retrieval System • Method Overview

  4. Task 1:Multilingual Retrieval System • Text Preprocessing • Stop words • Stemming • Decompounding • Word translation

  5. Task 1:Multilingual Retrieval System • Method Overview

  6. Task 1: Multilingual Retrieval System • Method 1: • Multilingual Retrieval via Query Translation no query feedback Raw score merge and Okapi system • Method 2: • Multilingual Retrieval via Query Translation with query feedback Raw score merge and Okapi system • Method 3: • Multilingual Retrieval via Document Translation no query feedback Raw score merge and Okapi system • Method 4: • Multilingual Retrieval via Document Translation with query feedback Raw score merge and Okapi system • Method 5: UniNe System

  7. Task 1:Multilingual Retrieval System • Method Overview

  8. Task 1:Multilingual Retrieval System • Normalization • drsk_mj denote the raw document score for the jth document retrieved from the mth ranked list for kth query,

  9. Task 1:Multilingual Retrieval System • Method Overview

  10. Task 1:Multilingual Retrieval System • Combine Multilingual Ranked Lists • (wm , rm) represents the weight of the vote and the exponential normalization factor for the mth ranked list

  11. Task 1:Experimental Results: Multilingual Retrieval • Qry/Doc: what was translated • fb/nofb: with/without pseudo relevance back • UniNe: UniNE system

  12. Task 1:Experimental Results: Multilingual Retrieval • MX: Combine models • W1/Trn: Equal or learned weights

  13. Task 2:Results Merge for Multilingual Retrieval • merge ranked lists of eight different languages (i.e., bilingual or monolingual) into a single final list • Logistic model (rank , doc score) language-specific methods  query-specific & language-specific

  14. Task 2:Results Merge for Multilingual Retrieval • Learn Query-Independent and Language-Specific Merging Model • estimated probability of relevance of document dk_ij • Model parameter • maximizing the log-likelihood (MLE) • maximizing MAP

  15. Task 2:Results Merge for Multilingual Retrieval • Learn Query-Specific and Language-Specific Merging Model • Calculate comparable scores for top ranked documents in each language (1) Combine scores of query-based and doc-based translation methods (2) Build language-specific query-specific logistic models to transform language-specific scores to comparable scores

  16. Task 2:Results Merge for Multilingual Retrieval (2) Build language-specific query-specific logistic models to transform language-specific scores to comparable scores • logistic model parameter estimate • minimize the mean squared error between exact normalized comparable scores and the estimated comparable scores • Estimate comparable scores for all retrieved documents in each language • Use comparable scores to create a merged multilingual result list

  17. Task 2:Experimental Results: Results Merge • Query-independent , language-specific • Mean average precision of merged multilingual lists of different methods on UniNE result lists • Mean average precision of merged multilingual lists of different methods on HummingBird result lists MAP is more accurate

  18. Task 2:Experimental Results: Results Merge • Query-specific , language-specific • Mean average precision of merged multilingual lists of different methods on UniNE result lists • C_X: top X docs from each list merged by exact comparable scores. • Top_X_0.5: top X docs from each list downloaded for logistic model to estimate comparable scores and combine them with exact scores by equal weight This means that the combination of estimated comparable scores and exact comparable scores can be more accurate than exact comparable scores in some cases

  19. Task 2:Experimental Results: Results Merge • Query-specific , language-specific • Mean average precision of merged multilingual lists of different methods on HummingBird result lists • Outperform query-independent and language-specific algorithm

More Related