A Cross -Lingual ILP Solution to Zero Anaphora Resolution

1. A Cross-Lingual ILP Solution to Zero Anaphora Resolution Ryu Iida & Massimo Poesio (ACL-HLT 2011)

2. Zero-anaphora resolution • Anaphoric function in which phonetic realization of anaphors is not required in “pro-drop” languages • Based on speaker and hearer’s shared understanding • φ: zero-anaphor (non-realized argument) • Essential: 64.3% of anaphors in Japanese newspaper articles are zeros (Iida et al. 2007)

3. Research background • Zero-anaphora resolution has remained an active area for Japanese (Seki et al. 2002, Isozaki&Hirao 2003, Iida et al. 2007, Imamura et al. 2009, Sasano et al. 2009, Taira et al. 2010) • The availability of the annotated corpora such that provided by SemEVAL2010 task10 “Multi-lingual coreference (Recasens et al.2010) is leading to renewed interest (e.g. Italian) • Mediocre results obtained on zero anaphors by most systems in SemEVAL • e.g. I-BART’s recall on zeros < 10%

4. Resolving zero-anaphors requires • The simultaneous decisionof • Zero-anaphor detection: find phonetically unrealized arguments of predicates (e.g. verbs) • Antecedent identification: search for an antecedent of a zero-anaphor • Roughly correspond to anaphoricity determination and antecedent identification in coreference resolution • Denis&Baldridge(2007) proposed a solution to optimize the outputs from anaphoricity determination and antecedent identification by using Integer Linear Programming (ILP)

5. Main idea • Apply Denis&Baldridge (2007)’s ILP framework to zero-anaphora resolution • Extend the ILP framework into a two-way to make it more suitable for zero-anaphora resolution • Focus on Italian and Japanese zero-anaphora to investigate whether or not our approach is useful across languages • Study only subject zero-anaphors (only type in Italian)

6. Topic of contents • Research background • Denis&Baldridge (2007)’s ILP model • Proposal: extending the ILP model • Empirical evaluations • Summary & future directions

7. Denis&Baldrige (2007)’s ILP formulation of base model • object function • If , mentions i and j are coreferentand mention j is an anaphor : 1 if mentions i and j are coreferent; otherwise 0

8. Denis&Baldrige (2007)’s ILP formulation of joint model • object function • If , mentions i and j are coreferentand mention j is an anaphor; otherwise j is non-anaphoric : 1 if mention j is an anaphor; otherwise 0

9. 3 constraints in ILP model characteristics of coreference relations • Resolve only anaphors:if mention pair ij is coreferent,mention j must be anaphoric • Resolve anaphors:if mention j is anaphoric, it must be coreferent with at least one antecedent • Do not resolve non-anaphors:if mention j is non-anaphoric, it should be have no antecedents transitivity of coreference chains

10. Proposal: extending the ILP framework • Denis&Baldridge’s original ILP-based model is not suitable for zero-anaphora resolution  Two modifications • Applying best-first solution • Incorporating a subject detection model

11. 1. Best-first solution • Select at most one antecedent for an anaphor • “Do-not-resolve-anaphors” constraint is too weak • Allow the redundant choice of more than one candidate antecedent • Lead to decreasing precision on zero-anaphora resolution • “Do-not-resolve-anaphors” constraint is replaced with “Best First constraint (BF)” that blocks selection of more than one antecedent:

12. 2. Integrating subject detection model • Zero-anaphor detection • Difficulty in zero-anaphora resolution comparing to pronominal reference resolution • Simply relying on the parser is not enough • most dependency parsers are not very accurate at identifying grammatical roles • detecting subject is crucial for zero-anaphor detection

13. 2. Integrating subject detection model • Resolve only non-subjects:if a predicate j syntactically depends on a subject,the predicate j should have no antecedent of its zero anaphor : 1 if predicate j syntactically depends on a subject; otherwise 0

14. Experiment 1: zero-anaphors • Compare the baseline models with the extended ILP-based models • Use the Maximum Entropy model to create base classifiers in the ILP framework and baselines • Featuredefinitions basically follow the previous work (Iida et al. 2007) and (Poesio et al. 2010)

15. Two baseline models • PAIRWISE classification model (PAIRWISE) • Antecedent identification and anaphoricity determination are simultaneously executed by a single classifier (as in Soon et al. 2001) • Anaphoricity Determination-then-Search antecedent CASCADEd model (DS-CASCADE) • Filter out non-anaphoric candidate anaphors using an anaphoricity determination model • Select an antecedent from a set of candidate antecedents of anaphoric anaphors using an antecedent identification model

16. Data sets • Italian (Wikipedia articles) • LiveMemories text corpus 1.2 (Rodriguez et al. 2010) • Data set on the SemEval2010: Coreference Resolution in Multiple Languages • #zero-anaphors: train 1,160 / test 837 • Japanese (newspaper articles) • NAIST text corpus (Iida et al. 2007) ver.1.4ß • #zero-anaphors: train 29,544 / test 11,205

17. Creating subject detection models • Data sets • Italian: 80,878 tokens in TUT corpus (Bosco et al. 2010) • Japanese: 1753 articles (i.e. training dataset) in NAIST text corpus merged with Kyoto text corpus • dependency arc is judged as positive if its relation is subject; as negative otherwise • Induce a maximum entropy classifier based on the labeled arcs • Features • Italian: lemmas, PoS tags and morphological information automatically computed by TextPro (Pianta et al. 2008) • Japanese: similar features as Italian except gender and number information

18. Results for zeroanaphors +BF: use best first constraint, +SUBJ: use subject detection model

19. Experiment 2: all anaphors • Investigate performance of all anaphors (i.e. NP- coreference and zero-anaphors) • Use the same data set and same data separation • Italian: LiveMemories text corpus 1.2 • Japanese: NAIST text corpus 1.4ß • Performance of each model are compared in terms of MUC score • Different types of referring expressions display very different anaphoric behavior • Induce 2 different models for NP-coreference and zero-anaphora respectively

20. Results for all anaphors

21. Summary • Extended Denis&Baldridge (2007)’s ILP-based coreference resolution model by incorporating modified constraints & a subject detection model • Our results show the proposed model obtained improvement on both zero-anaphora resolution and overall coreference resolution

22. Future directions • Introduce more sophisticated antecedent identification model • Test our model for English constructions resembling zero-anaphora • Null instantiations in SEMEVAL 2010 ‘Linking Events and their Participants in Discourse’ task • Detect generic zero-anaphors • Have no antecedent in the preceding context • e.g. the Italian and Japanese translation of • I walked into the hotel and (they) said …

24. Data sets on English coreference • Use ACE-2002 data set • Data set is classified into the two subset • Pronouns and NPs

25. Details of experiment: English training data test data train: NPs train: zeros test: NPs test: zeros models: NP coreference models: zero anaphora outputs: NPs outputs: zeros outputs: all anaphors

26. Results: all anaphors (English)