1 / 24

Japanese Dependency Analysis using Cascaded Chunking

Japanese Dependency Analysis using Cascaded Chunking. Taku Kudo 工藤　拓 Yuji Matsumoto 松本　裕治 Nara Institute Science and Technology, JAPAN. Motivation. Kudo, Matsumoto 2000 (VLC) Presented a state-of-the-art Japanese dependency parser using SVMs ( 89.09 % for standard dataset)

adelie
Download Presentation

Japanese Dependency Analysis using Cascaded Chunking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Japanese Dependency Analysis using Cascaded Chunking Taku Kudo 工藤　拓 Yuji Matsumoto 松本　裕治 Nara Institute Science and Technology, JAPAN

  2. Motivation • Kudo, Matsumoto 2000 (VLC) • Presented a state-of-the-art Japanese dependency parser using SVMs(89.09% for standard dataset) • Could show the high generalization performance and feature selection abilities of SVMs • Problems • Not scalable • 2 weeks training using 7,958 sentences • Hard to train with larger data • Slow in Parsing • 2 ～　3 sec./sentence • Too slow to use it for actual NL applications

  3. Goal • Improve the scalability and the parsing efficiency without loosing accuracy ! • How? • Apply Cascaded Chunking model to dependency parsing and the selection of training examples • Reduce the number of times SVMs are consulted in parsing • Reduce the number of negative examples learned

  4. Outline • Japanese dependency analysis • Two models • Probabilistic model (previous) • Cascaded Chunking model (new!) • Features used for training and classification • Experiments and results • Conclusion and future work

  5. Japanese Dependency Analysis (1/2) • Analysis of relationship between phrasal units called bunsetsu (segments), base phrases in English • Two Constraints • Each segment modifies one of the right-side segments (Japanese is head final language) • Dependencies do not cross each other

  6. 私は彼女と京都に行きます I go to Kyoto with her. Raw text Morphological analysis and Bunsetsu identification 私は / 彼女と / 京都に / 行きます I with herto Kyoto-loc go Dependency Analysis 私は / 彼女と / 京都に / 行きます Japanese Dependency Analysis (2/2)

  7. 1. Build a Dependency Matrix ME, DT or SVMs (How probable one segment modifies another) Modifiee 2. Search the optimal dependencies which maximize the sentence probabilities using CYK or Chart 2 3 4 1 0.1 0.2 0.7 Modifier 2 0.2 0.8 Output 3 1.0 Dependency Matrix 私は 1 / 彼女と 2 / 京都に 3 / 行きます 4 Probabilistic Model Input 私は 1 / 彼女と 2 / 京都に 3 / 行きます 4 I-top / with her / to Kyoto-loc / go

  8. Problems of Probabilistic model(1/2) • Selection of training examples: All candidates of two segments which have • Dependency relation→positive • No dependency relation→negative • This straightforward way of selection requires a total (where n is # of segments in a sentence) training examples per sentence • Difficult to combine probabilistic model with SVMs which require polynomial computational cost

  9. Problems of Probabilistic model(2/2) • parsing time is necessary with CYK or Chart • Even if beam-search is applied, 　　　　　parsing time is always necessary • The classification cost of SVMs is much more expensive than other ML algorithms such as ME and DT

  10. Cascaded Chunking Model • English parsing [Abney 1991] • Parses a sentence deterministically only deciding whether the current segment modifies the segment on its immediate right hand side • Training examples are extracted using this algorithm itself

  11. 彼は1　　彼女の2　 真心に4　 感動した。5 ? ? ? 彼は1　　彼女の2　 真心に4　 感動した。5 O DD O 彼は1　　 真心に4　 感動した。5 ? ? ? ? ? ? 彼は1　　 真心に4　 感動した。5 彼は1　　彼女の2　 温かい3　 真心に4　 感動した。5 O D O O O DD O 彼は1　　 　 感動した。5 彼は1　　 　 感動した。5 　　 　 感動した。5 ? D O finish Example: Training Phase Annotated sentence 彼は1　　彼女の2　 温かい3　 真心に4　 感動した。5 He her warm heart be moved (He was moved by her warm heart.) 彼は1　　彼女の2　 温かい3　 真心に4　 感動した。5 Tag is decided by annotated corpus Pairs of tag (D or O) and context(features) are stored as training data for SVMs Training Data SVMs

  12. 彼は1　　彼女の2　 真心に4　 感動した。5 ? ? ? 彼は1　　彼女の2　 真心に4　 感動した。5 O DD O 彼は1　　 真心に4　 感動した。5 ? ? ? ? ? ? 彼は1　　 真心に4　 感動した。5 彼は1　　彼女の2　 温かい3　 真心に4　 感動した。5 O D O O O DD O 彼は1　　 　 感動した。5 彼は1　　 　 感動した。5 　　 　 感動した。5 ? D O finish Example: Test Phase Test sentence 彼は1　　彼女の2　 温かい3　 真心に4　 感動した。5 He her warm heart be moved (He was moved by her warm heart.) 彼は1　　彼女の2　 温かい3　 真心に4　 感動した。5 Tag is decided by SVMs built in training phase SVMs

  13. Advantages of Cascaded Chunking model • Simple and Efficient • Prob.: v.s. cascaded chunking: • Lower than since most of segments modify segment on its immediate right-hand-side • Training examples is much smaller • Independent from ML algorithm • Can be combined with any ML algorithms which work as a binary classifier • Probabilities of dependency are not necessary

  14. B A C Features Modify or not? 彼の1 友人は2　 この本を3　 持っている4　女性を5　探している6 His friend-top this book-acc have lady-acc be looking for modifier modifiee His friend is looking for a lady who has this book. • Static Features • modifier/modifiee • Head/Functional Word: (surface,POS,POS-subcategory,inflection- 　　　　　　type,inflection-form), brackets, quotations, punctuations, position • Between segments: distance, case-particles, brackets, 　　　　　　　　　　　　　　　　　　　　　　　　　　quotations, punctuations • Dynamic Features [Kudo, Matsumoto 2000] • A,B : Static features of Functional word • C: Static features of Head word

  15. Experimental Setting • Kyoto University Corpus 2.0/3.0 • Standard Data Set • Training: 7,958 sentences / Test: 1,246 sentences • Same data as [Uchimoto et al. 98, Kudo, Matsumoto 00] • Large Data Set • 2-fold Cross-Validation using all 38,383 sentences • Kernel Function: 3rd polynomial • Evaluation method • Dependency accuracy • Sentence accuracy

  16. Results

  17. Effect of Dynamic Features(1/2)

  18. B A C Effect of Dynamic Features (2/2) Modify or not? 彼の1 友人は2　 この本を3　 持っている4　女性を5　探している6 His Friend-top this book-acc have lady-acc be looking for modifier modifiee

  19. Probabilistic v.s. Cascaded Chunking (1/2) Probabilistic Model uses all candidates of dependency relation as training data 彼は1 この本を2　　持っている3 女性を4　 探している5 He-top this book-acc have lady-acc be looking for modifier modifiee (He is looking for a lady who has this book.) Positive:この本を2 →　持っている3 Negative:この本を2 →　探している5 unnecessary Probabilistic models commit a number of unnecessary examples

  20. Probabilistic v.s. Cascaded Chunking (2/2)

  21. Conclusion • A new Japanese dependency parser using a cascaded chunking model • It outperforms the previous probabilistic model with respect to accuracy, efficiency and scalability • Dynamic features significantly contribute to improve the performance

  22. Future Work • Coordinate structure analysis • Coordinate structures frequently appear in Japanese long sentences and make analysis hard • Use posterior context • Hard to parse the following sentence only using cascaded chunking model 僕の　　母の　　　 ダイヤの　　 指輪 My mother’s diamond ring

  23. Comparison with Related Work

  24. Maximize the margin d Min.： s.t.： Support Vector Machines [Vapnik] • Soft Margin • Kernel Function

More Related