Towards automatic enrichment and analysis of linguistic data for low-density languages

1. Towards automatic enrichment and analysis of linguistic data for low-density languages Fei Xia University of Washington Joint work with William Lewis and Dan Jinguji

2. Motivation: theoretical linguistics • For a particular language (e.g., Yaqui), find the answers for the following questions: • What is word order: SVO, SOV, VSO, ….? • Does it have double-object construction? • Can a coordinated phrase be discontinuous? (e.g., “NP1 Verb and NP2”) • …. • We want to know the answers for hundreds of languages.

3. Motivation: computational linguistics • For a particular language, we want to build • a Part-of-speech tagger and a parser • Common approach: create a treebank • a MT system • Common approach: • collect parallel data • test translation divergence (Dorr, 1994; Fox 2002; Hwa et al, 2002)

4. Main ideas • Projecting structures from a resource-rich language (e.g., English) to a low-density language. • Tapping the large body of Web-based linguistic data  using ODIN dataset

5. Structure projection • Previous work • (Yarowsky & Ngai, 2001): POS tags and NP boundaries • (Xi & Hwa, 2005): POS tags • (Hua et al., 2002): dependency structures • (Quirk et al., 2005): dependency structures • Our work: • Projecting both dependency structures and phrase structures • It does not require a large amount of parallel data or hand-aligned data. • It can be applied to hundreds of languages.

6. Outline • Background: IGT and ODIN • Data enrichment • Word alignment • Structure projection • Grammar extraction • Experiments • Conclusion and future work

7. Background: IGT and ODIN

8. Interlinear Glossed Text (IGT) Rhoddodd yr athro lyfr i’r bacjgem ddoe Gave-3sg the teacher book to-the boy yesterday The teacher gave a book to the boy yesterday (Bailyn, 2001)

9. ODIN • Online Database of Interlinear text • Storing and indexing IGT found in scholarly documents on the Web • Searchable by language name, language family, concept/gram, etc. • Current size • 36439 instances • 725 languages

11. Data Enrichment

12. The goal • Original IGT: three lines • Enriched IGT: • English phrase structure (PS), dependency structure (DS) • Source PS and DS • Word alignment between source and English translation

13. Three steps • Parse the English translation • Align the source sentence and its English translation • Project the English PS and DS onto the source side

14. Step 1: Parsing the English translation The teacher gave a book to the boy yesterday

15. Step 2: Word alignment

16. Source-gloss alignment

17. Gloss-translation alignment

18. Heuristic word aligner Gave-3sg the teacher book to-the boy yesterday The teacher gave a book to the boy yesterday The aligner aligns two words if they have the same root form.

19. Limitation of heuristic word aligner 1SG pig-NNOM.SG grasp-PST and cat-NNOM.SG I caught the pig and the cat

20. Statistical word aligner • GIZA++ package (Och and Ney, 2000) • It implemented IBM models (Brown et. al., 1993) • Widely used in statistical MT field • Parallel corpus formed by the gloss and translation lines of all the IGT examples in ODIN.

21. Improving word aligner • Train both directions (glosstrans, transgloss) and combine the results • Split words in the gloss line into morphemes 1SG pig-NNOM.SG grasp-PST and cat-NNOM.SG  1SG pig -NNOM –SG grasp -PST and cat –NNOM -SG

22. Improving word aligner (cont) Pedro-NOM Goyo-ACC yesterday horse-ACC steal-PRFV-SAY-PRES Pedro says Goyo has stolen the horse yesterday . Add (x,x) sentence pairs: (Pedro, Pedro) (Goyo, Goyo) …..

23. Step 3: Projecting structures • Projecting DS • Previous work: • (Hwa et. al, 2002) • (Quirk et. al, 2005) • Projecting PS

24. Projecting phrase structure

25. Projecting PS • Copy the English PS and remove all the unaligned English words • Replace English words with corresponding source words • Starting from the root, reorder children of each node. • Attach unaligned source words

26. Starting with English PS The teacher gave a book to the boy yesterday

27. Replacing English words

28. Reordering children

29. Calculating phrase spans

30. “Reordering” NP and VP

31. Removing VP

32. Removing a node in PS

33. After removing VP

34. Reordering VBD and NP

35. Removing NP

36. Merging IN and DT

37. Before “reordering”

38. After reordering 1 2 3 4 5 6 7

39. Reordering two children of x: y1 and y2 Let Si be the phrase span of yi: • S1 and S2 don’t overlap: reorder two nodes according to the spans. • S1½ S2: remove y2 • S1¾ S2: remove y1 • S1 and S2 overlap, and neither is a strict subset of the other: remove both nodes. If y1 and y2 are leaf nodes, merge them.

40. Attaching unaligned source words y yi yj yk

41. Information that can be extracted from enriched IGT • Grammars for source language • Transfer rules • Examples with interesting properties (e.g., crossing dependencies)

42. Grammars S  VBD NP NP PP NP NP  DT NN NP  NN PP  IN+DT NN

43. Examples of crossing dependencies Inepo kow-ta bwuise-k into mis-ta 1SG pig-NNOM.SG grasp-PST and cat-NNOM.SG I caught the pig and the cat (Martinez Fabian, 2006)

44. Examples of crossing dependencies

45. Examples of crossing dependencies

46. Outline • Background: IGT and ODIN • Data enrichment • Experiments • Conclusion and future work

47. Experiments • Test on a small set of IGT examples for seven languages: • SVO: German (GER) and Hausa (HUA) • SOV: Korean (KKN) and Yaqui (YAQ) • VSO: Irish (GLI) and Welsh (WLS) • VOS: Malagasy (MEX)

48. Test set Numbers in the last row come from the Ethnologue (Gorden, 2005) Human annotators checked system output and corrected - English DS - word alignment - source DS

49. Heuristic word aligner  High precision, low recall.

50. Statistical word aligner: training data