1 / 14

Morpho Challenge competition 2005-2010 Evaluations and results Authors Mikko Kurimo

Morpho Challenge competition 2005-2010 Evaluations and results Authors Mikko Kurimo Sami Virpioja Ville Turunen Krista Lagus. Introduction. Started in 2005. Open to all. Organizers selected evaluation tasks, data and metric and performed all the evaluations .

mara-blankenship
Download Presentation

Morpho Challenge competition 2005-2010 Evaluations and results Authors Mikko Kurimo

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Morpho Challenge competition 2005-2010 Evaluations and results Authors MikkoKurimo Sami Virpioja Ville Turunen Krista Lagus

  2. Introduction • Started in 2005. • Open to all. • Organizers selected evaluation tasks, data and metric and performed all the evaluations. • Unsupervised and semi-supervised approach. • Semi-supervised approach was introduced in Morpho Challenge 2010.

  3. Aim • To develop Language – independent algorithms to discover morphemes from text material . • Morphemes : It is the smallest grammatical unit in a language. • To promote research in machine learning , NLP .

  4. Evaluation tasks & languages # From MikkoKurimo, Sami Virpioja, Ville Turunen, Krista Lagus. 2010. Morpho Challenge 2005-2010: Evaluations and Results.

  5. Word Segmentation • In 2005 : • Segment the text into morphemes . • In 2007 : • Locate the surface form (word segmentation). • Locate which surface form are the allomorph of the same underlying morpheme.

  6. Principles for segmentation The evaluation is based on a subset of the word forms given as training data. The frequency of the word form plays no role in evaluation. The evaluation score is balanced F-measure, the harmonic mean of precision and recall. If the linguistic gold standard has several alternative analysis for one word, for full precision, it is enough that one of the alternatives is equivalent to the proposed analysis

  7. Information retrieval • The algorithms were tested by using the morpheme segmentations for text retrieval. • A stemming algorithm is used to reduce inflected words to base words. • Problem : Language specific. • Challenges • Correct weighting method. • Number of queries were limited.

  8. Machine translation • Two stages • Alignment of parallel sentences in both languages. • Training a language model. • In 2009 Morph challenge the focus was on alignment problem.

  9. Some Algorithms • Bernhard (Bernhard, 2006) : • Best for Finnish , English and German linguistic evaluation. • First list of prefixes and suffixes is extracted. • Segmentations are generated using this list. • Best segmentation is selected on the basis of cost function.

  10. Some Algorithms • Morfessor algorithm : • To discover most basic & compact description of data. • Substrings occurring frequently in the training set are also considered as morphemes. • Ex. hand, hand+s , hand+ful , left+hand+ed. • Gives better result than other algorithms in Finnish & Turkish. • #From : Morfessor in the morpho challenge (2006)by Mathias Creutz , Krista Lagus

  11. Result Morpho Challenge : 2010 • S = semi-supervised algorithm • P = unsupervised algorithm with supervised parameter tuning • #From http://research.ics.aalto.fi/events/morphochallenge2010

  12. Open Challenges • What is the best analysis algorithm ? • What is the meaning of the morphemes ? • How to evaluate the alternative analyses ? • How to improve the analysis using context ? • How to effectively apply semi-supervised learning ?

  13. References • MikkoKurimo, Sami Virpioja, Ville Turunen, Krista Lagus. 2010. Morpho Challenge 2005-2010: Evaluations and Results. Proceedings of the 11th meeting of the ACL special interest group on Computational Morphology and Phonology . • Mathias Creutz and Krista Lagus. 2006 . Morfessor in the Morpho Challenge. Proceedings of the PASCAL Challenge Workshop on Unsupervised Segmentation of Words into Morphemes • Official site of Morpho Challenge : http://research.ics.aalto.fi/events/morphochallenge2010/ • Wikipedia : http://en.wikipedia.org/

  14. Thank You

More Related