1 / 14

Winding up CLEF 2004 – towards CLEF 2005

Winding up CLEF 2004 – towards CLEF 2005. Finishing up with CLEF 2003. CLEF 2003 Proceedings coming soon ………. CLEF 2004 Proceedings. Again in Springer LNCS series Editors will be The Track Coordinators Date for submission of final, revised papers? Date for completion of editing?

didier
Download Presentation

Winding up CLEF 2004 – towards CLEF 2005

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Winding up CLEF 2004 – towards CLEF 2005

  2. Finishing up with CLEF 2003 • CLEF 2003 Proceedings coming soon ………

  3. CLEF 2004 Proceedings • Again in Springer LNCS series • Editors will be The Track Coordinators • Date for submission of final, revised papers? • Date for completion of editing? • Copies for authors?

  4. CLEF 2005 Working Notes • Published on Website - 1 week before workshop • CD-ROM instead of print • Me-bring-to-workshop vs You-download-your-own • Small printed Working Notes with Workshop programme plus long abstracts of each paper

  5. CLEF 2005 At the end of CLEF 2003 Workshop: • CLIR is at a crossroads (we need a vision for CLEF-2) • A vision that meets the needs of current research in MLIA/CLIR (and satisfies them cheaply!)

  6. The CLEF Model • Base Tracks • “Additional” Tracks • Must meet the current requirements of CLIR • What are the current requirements of CLIR? • Need to move towards more advanced CLIR tasks – information extraction, work with mixed media, understand user needs better

  7. The Funding Problem CLEF Model • Everyone supports CLEF • Overall Coordination (the CLEF Umbrella) + Base Tracks partially supported by DELOS NoE • All “additional tracks” and workshop are self-funding • Need to find additional support to ensure high-level continuity: European Commission, sponsors • Ideas please

  8. CLEF 2005: Tracks Main Tracks • Ad-hoc and GIRT • iCLEF • QA@CLEF • ImageCLEF • CL-SR Plus exciting-work-with-old-data task Pilot Experiments • WebCLEF • GIR-CLEF • More??? More information on web-site soon

  9. AD-HOC • Consolidate “new” collections • Add new data • Merging sub-task for veterans • Plus exciting-work-with-old-data task

  10. imageCLEF 2005 • Ad hoc retrieval task • St. Andrews or similar collection with good annotations • Tasks for textual and more visual retrieval • More examples, also negative examples • Medical retrieval task • More data, visually harder tasks, training data available • Text (cross language) with the images as query • Automatic annotation task through classification • Interactive retrieval task • Details to be defined • Web retrieval task • Collection, who will run it?

  11. Changes and open questions • Changes • Trainingdata • Less time between topic delivery and submission • Open questions • Databases to be used • St. Andrews, corbis, getty, … • How to extend medical image database • Will we get automated annotation data? • Restrictions of collected web data • Help with web page, ground truthing, evaluation • Oshu, Aachen, … (community effort) • Funding for the task

  12. iCLEF • Q&A interactive task: • Begin with 2004 experiment design • Discussion with Q&A people to adapt it to their research needs • Joint coordination (Alessandro Vallin, Julio Gonzalo) • Image CLEF interactive task: • Coordinated design of the task (Paul Clough, Henning Muller, Julio Gonzalo) • Strong interest from participants • Thanks Doug!

  13. CLEF 2005 Workshop • Implications of more tracks • More time / parallel paper sessions/ more breakout sessions /more poster sessions • Location ?

  14. CLEF 2005 TentativeSchedule • Calls for Participation – from November 2004 • Registration Open – 15 January 2005 • Data Release – end February 2005 • Topic Release – mid April 2005 • Submission of Runs by Participants – mid June 2005 • Release of Individual Results – 1 August 2005 • Submission of Working Notes Paper – 20 August 2005 • Workshop  – mid-September 2005 (ECDL2005, Vienna – 18-23/9/05)

More Related