Jointly Identifying Temporal Relations with Markov Logic

1. Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa†, Sebastian Riedel‡, Masayuki Asahara†, Yuji Matsumoto† †Nara Institute of Science and Technology, Japan ‡ University of Massachusetts, Amherst ACL-IJCNLP2-7 August, 2009 Suntec Singapore

2. Outline Background and Motivation Related work of temporal relation identification Proposed global approach with Markov Logic Experimental setup and highlighted data Summary and future work

3. Background and Motivation Temporal Relation Identification (temporal ordering) Identifying temporal orders of events and time expressions in a document With the introduction of the TimeBank corpus (Pustejovsky et al., 2003), machine learning approaches to temporal ordering became possible. Document Creation Time(August 2009) 2003 introduction became Past Present Future BEFORE Essential work for document understanding

4. Outline Background and Motivation Related work of temporal relation identification Proposed global approach with Markov Logic Experimental setup and highlighted data Summary and future work

5. We regard temporal ordering as a classification task With TimeML, the TimeBank corpus was created Allen‘s Temporal Logic [Allen 1983]TimeML and TimeBank [Pustejovsky et al. 2003] Allen’s (13 Labels)‏ TimeML (11 Labels)‏ EVENT / TIME before < BEFORE meets m IBEFORE overlaps o ENDED_BY finished-by fi INCLUDES contains c starts s BEGINS equal = SIMULTANEOUS started-by si BEGUN_BY during d DURING finishes f ENDS overlapped-by oi met-by mi IAFTER after AFTER >

6. TempEval (SemEval 2007 Task 15) Temporal Relation Identification in SemEval 2007 Shared Task (TempEval) Six temporal relation labels Main Label (BEFORE, AFTER，OVERLAP) Sub-Label (BEFORE-OR-OVERLAP, OVERLAP-OR-AFTER, VAGUE) TempEval includes three types of tasks (A, B, and C)

7. Task A of TempEval • Temporal relations between events and time expressions that occur within the same sentence With the introduction of the TimeBank corpus (Pustejovsky et al., 2003), machine learning approaches to temporal ordering became possible. 2003 DCT (August 2009) OVERLAP introduction became Past Present Future

8. Task B of TempEval • Temporal relations between events and the Document Creation Time (DCT) With the introduction of the TimeBank corpus (Pustejovsky et al., 2003), machine learning approaches to temporal ordering became possible. 2003 DCT (August 2009) BEFORE BEFORE introduction became Past Present Future

9. Task C of TempEval • Temporal relations between the main events of adjacent sentences The TimeBank corpus was created (Pustejovsky et al., 2003). As a result, machine learning approaches to temporal ordering became possible. 2003 DCT (August 2009) created became BEFORE Past Present Future

10. Issues of the TempEval Participants Local approaches with machine learning are employed by many participants in TempEval Considering only a single relation at a time Local approach cannot take into account the other relations DCT‏ DCT‏ BEFORE(Task B) AFTER(Task B) EVENT 2‏ EVENT 1 EVENT 1 EVENT 2‏ AFTER ?(Task C) BEFORE(Task C) A globalapproach can be useful in that case

11. Issues of the TempEval Participants Local approaches with machine learning are employed by many participants in TempEval Considering only a single relation at a time Local approach cannot take into account the other relations DCT‏ BEFORE(Task B) AFTER(Task B) EVENT 1 EVENT 2‏ BEFORE(Task C) A globalapproach can be useful in that case

12. Outline Background and Motivation Related work and task reviews of temporal relation identification Proposed global approach with Markov Logic Experimental setup and highlighted data Summary and future work

13. Overview of Our Global Approach Ensure consistency among the multiple relations with hard and soft constraints based on the transition rules Jointly identify the three types of relations in TempEval Learning one global model for the three tasks Global approach withMarkov Logic

14. Markov Logic[Richardson and Domingos, 2006] A Statistical Relational Learning framework An expressive template language of Markov Networks Not only hard but alsosoft constraints A Markov Logic Network (MLN) is a set of pairs (φ, w) where φ is a formula in first-order logic w is a real number weight Higher weight  stronger constraint

15. An Example of Markov Logic Networks hasPastTense(a) : indicates that an event a has past tense beforeDCT(a) : indicates that an event a happens before the DCT before(a,b) : indicates that an event a happens before another event b hasPastTense(e1) before (e1,e2) hasPastTense(e2) wa(e1) wa(e2) wb(e1,e2) grounding beforeDCT(e1) beforeDCT(e2) ※ e1 and e2 are events

16. Global Feature Representation (Predicate Definition) • relE2T(e, t, r) : the relation r between an event e and a time expression t • relDCT(e, r) : the relation r between an event a and the DCT • relE2E(e1, e2, r) : the relation r between two events e1 and e2 • relT2T(t1, t2, r) : the relation r between two time expressions t1 and t2 • dctOrder(t, r) : the relation r between a time expression t and the DCT DCT dctOrder dctOrder relT2T TIME (t1) TIME (t2) relDCT(B)‏ relDCT(B)‏ relE2T(A)‏ relE2T(A)‏ EVENT (e1) EVENT (e2) relE2E(C)‏

17. Global Feature Representation (Transition Rules) • We jointly solve the three tasks of TempEval • We use global features named Joint formulae • A joint formula is based on a transition rule DCT B→C DCT C→B BEFORE BEFORE AFTER BEFORE EVENT (e1)‏ EVENT(e2)‏ EVENT (e2)‏ EVENT(e1)‏ BEFORE AFTER BEFORE & AFTER ⇒ BEFORE BEFORE & AFTER ⇒ BEFORE If e1 happens before DCT and e2 happens after DCT => then e1 is before e2 If e1 happens before DCT and e1 happens after e2, => then e2 happens before DCT

18. Global Feature Representation (Templates of the all Joint Formulae) They are developed with events, time expressions and relations

19. Global Feature Representation (Templates of the all Joint Formulae) They are developed with events, time expressions and relations

20. Outline Background and Motivation Related work and task reviews of temporal relation identification Proposed global approach with Markov Logic Experimental setup and highlighted data Summary and future work

21. Experimental Setup Use a MLN Engine “Markov thebeast” Weight learning : MIRA Inference : Cutting Plane Inference (base solver: ILP) [Riedel, 2008] Employ the local features referred to the early work in TempEval [SemEval, 2007] Select joint formulae as global features Use the same data and evaluation schemes of TempEval

22. Comparison of Local and Global Over all tasks, Global is better than Local On Task A, Global model outperformed Local one. • Results with 10-fold cross validation on training data ※All scores denote F1-value ρ< 0.01 (McNemar’s test, 2-tailed)

23. Comparison to State-of-the-art Outperformed the others on Tasks A and C Always performed better than the best pure machine-learning based system (CU-TMP[Bethard and Martin, 2007]) • Results with the other systems on test data (F1-value) ※All scores denote F1-value

24. Outline Background and motivation Related work and task reviews of temporal relation identification Proposed global approach with Markov Logic Experimental setup and highlighted data Summary and future work

25. Summary We proposed a global framework with Markov Logic for Temporal Relation Identification Our global model with joint formulae successfully improved the performances of the identifications Our approach reported the competitive results among all participants in TempEval

26. Future Work Issues inherent to the task and the dataset Low inter annotator agreement Low transitive connectivity Small size • Numbers of labeled relations for all tasks and datasets • Semi-supervised approaches ease some issues