1 / 5

TrebleCLEF and LRs

TrebleCLEF Evaluation package and language resources CLEF Evaluation Packages Nicolas Moreau / Khalid Choukri - ELDA. TrebleCLEF and LRs. CLEF evaluation campaigns produced large volumes of valuable data A major objective of TrebleCLEF: make CLEF collections publicly available

awhatley
Download Presentation

TrebleCLEF and LRs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TrebleCLEFEvaluation package and language resourcesCLEF Evaluation PackagesNicolas Moreau / Khalid Choukri - ELDA

  2. TrebleCLEF and LRs • CLEF evaluation campaigns produced large volumes of valuable data • A major objective of TrebleCLEF: make CLEF collections publicly available • ELDA responsible for: • Inventory of CLEF resources • IPR issues • Production of CLEF Evaluation Packages

  3. Evaluation Packages CLEF test suites consist of: • Multilingual document collections • Sets of topics • Sets of relevance assessments • Scoring tools • Participant results and related publications • Step-by-step documentation on how to perform a system evaluation Reusable test suites for system benchmarking

  4. CLEF Packages Already distributed (ELRA/ELDA catalogue): • CLEF 2000-2003 (AdHoc on news + domain specific)‏ Produced or being produced: • CLEF AdHoc/News 2004-2008 => Preliminary version • CLEF AdHoc/TEL 2008 • CLEF QA 2003-2008 + CLEF QAST 2007-2008 • CLEF Domain specific 2004-2008 • GeoCLEF 2005-2008 Still in discussion: • ImageCLEF, VideoCLEF • WebCLEF, CLEF-IP, and others ...‏

  5. Distribution and Pricing Policy Distribution • ELRA/ELDA catalogue: http://catalog.elra.info/ • Restricted to evaluation use Pricing policy • Low prices for academics • Cover the management of data (catalogue management, DVD shipping, etc.)‏

More Related