Completions and continuations in dialogue: a preliminary account

1. Completions and continuations in dialogue: a preliminary account Massimo Poesio (Uni Essex) Hannes Rieser (Uni Bielefeld)CATALOGBarcelona, July 2004

2. Sentence cooperations: an example Inst: So, jetzt nimmst DuWell, now you grasp Cnst: eine Schraubea screw. Inst: eine <-> orangene mit einem Schlitz.an <-> orange one with a slit Cnst: Ja. Yes.

3. Sentence cooperations: an informal definition (Clark, 1996) • SENTENCE COOPERATION: At least two dialogue participants contribute to a sentence production • COMPLETION: sub-sentential structure continued by obligatory constituents • CONTINUATION: material added to already `complete’ sentence

4. The significance of sentence cooperations • Clear evidence that dialogue requires coordination at the sub-sentential level (see also Pickering and Garrod, in press) • Provide insights into incrementality and compositionality issues • A tool to investigate competing claims about coordination in dialogue • Purely intentional models • Pickering and Garrod’s IAM based on simpler alignment mechanisms.

5. Outline of the talk • Sentence cooperations in the Bielefeld Toy Plane Corpus (BTPC) • An introduction to • PTT (Poesio and Traum 1997, 1998; Matheson et al, 2000; Poesio, to appear) • Intro to model of we-intentions we use, following Bratman (1992), Tuomela (2000) and Grosz and Kraus (1996) • A PTT implementation of an intentional analysis of completions • (May have time to sketch an IAM analysis)

6. The Bielefeld Toy Plane Corpus

7. The Bielefeld Toy Plane Corpus • 22 video-filmed, speech recorded and transcribed dialogues • two agents, Instructor and Constructor • constructing a “Baufix” airplane • different sight conditions: total screen, half-screen, face to face • 3675 contributions • 160 sentence cooperations (4.34 %) • in most of them cooperation other-initiated (95%)

8. Sentence cooperations in the BTPC (Skuplik, 1999) • 126 sentence cooperations from the BTPC • 54 completions (43%) • 72 continuations (57%) • Production of the completion / continuation: 79% Cnst, 21% Inst • 84% of compl. / contin. accepted by previous speaker (41% implicitly) • Release-turn signalled in 31% of cases

9. A few other observations • Completions become more frequent as dialogue procedes (routinization?)

10. RESULTING IN A SENTENCE WHEN MERGED CNST COMPLETION(70%) WITH AN OBLIGATORY NP SIGNALED BY LENGHTENING OF “DU”, LEVEL TONE The example, revisited Inst: So, jetzt nimmst DuWell, now you grasp Cnst: eine Schraubea screw. Inst: eine <-> orangene mit einem Schlitz.an <-> orange one with a slit Cnst: Ja. Yes.

11. Outline of the talk • The Bielefeld Toy Plane Corpus (BTPC) • An introduction to • PTT (Poesio, 1995; Poesio and Traum 1997, 1998; Matheson et al, 2000; Poesio, to appear) • Intro to model of we-intentions we use, following Bratman (1992), Tuomela (2000) and Grosz and Kraus (1996) • A PTT implementation of an intentional analysis of completions

12. PTT • A theory of semantics and interpretation in dialogue originally motivated by work on the TRAINS project • Key characteristics: • Building on (Compositional) DRT (Muskens, 1996) • Common ground as a record of the discourse situation (Barwise and Perry, 1983) • An account of incremental semantic interpretation • An account of GROUNDING • So far, primarily concerned with aspects of dialogue driven by obligations

13. Common ground: beyond assertion A They have at their disposal enormous assets // and their policy B //look can I just come in on that// last year A //YES IN A MINUTE IF YOU MAY AND WHEN I’M FINISHED // then you’ll know B // yes I’M SO SORRY (Coulthard 1977)

14. Common ground: beyond assertion B: Go to Elmhurst, pass the courthouse and go to Elmhurst and then to Elmhurst, uh north.A: mm hum.B: Towards Riverton, till you come to that Avila HallA: Oh yesB: Dju know where that//is?A: //uh huhA: Oh surelyB: Avilla Hall on the corner of Bor//donA: //uh huhB: Well there, on Bordon you turn back to town, left. (George Psathas, "Direction-giving in Interaction," in Boden and Zimmerman, ed.)

15. From DRT to PTT • a. A: There is an engine at Avon. • B: It is hooked to a boxcar • DRT: [ x,w,y,u,s,s’| engine(x), Avon(w), s: at(x,w), boxcar(y), s’:hooked-to(u,y), u=x]

16. Common ground and discourse situation in PTT [ce1,ce2,K1,K2| K1=[x,w,s| engine(x), Avon(w), s: at(x,w)], ce1: assert(A,B,K1) K2=[y,z,s’| boxcar(y), s’:hooked-to(z,y), z=x], ce2: assert(B,A,K2)]

17. Locutionary acts in the common ground “The fact that a speaker is speaking, saying the words he is saying in the way he is saying them, is a fact that is usually accessible to everyone present. Such observed facts can be expected to change the presumed common background knowledge of the speaker and his audience in the same way that any obviously observable change in the physical surroundings of the conversation will change the presumed common knowledge.” (Stalnaker, Assertion, p. 323)

18. The time-order of sentence processing • GARDEN-PATH phenomena shows that parsing is INCREMENTAL (Bever, 1974; Frazier, 1987) • Marslen-Wilson 1973, 1975: semantic information ALSO accessed immediately • Swinney, 1979: lexical access incremental • Just and Carpenter,1980: IMMEDIACY HYPOTHESIS (“Every word encountered should be processed to the deepest level possible before the eye moves on to the next word”) • Eye-tracking work (Tanenhaus et al, 1995, tomorrow): really fine-grained incrementality

19. Alignment at all levels Pickering & Garrod

20. Clarification questions (Ginzburg and Cooper, Purver and Ginzburg) A: Did Bo leave?B: BO? A: Bo Smith.B: Yes, half an hour ago. Matthew: It wasn’t all that bad. At least the pool was clean.Lara: MR POOL?Matthew: The pool.Lara: Oh. <laugh> (“What is the intended content of your utterance ‘Bo’?”) (“Did you utter the words ‘Mr. Pool’?”)

21. Micro conversational events (Poesio, 1995) boxcar  [u|u:utter(A,”boxcar”), Noun(u),sem(u)=x [|boxcar(x)], + SYN INFO (NEXT)] umm [u,ce| u: utter(A,”umm”), ce: keep-turn(A), generate(u,ce)]

22. MCEs in the example dialogue [mce1,ce1| mce1:utter(Inst,“so"), Adv(mce1), ce1:take-turn(Inst), generate(mce1,ce1)]; [mce2,ce2| mce2:utter(Inst,“jetzt"), Adv(mce2), ce2:keep-turn(Inst), generate(mce2,ce2)]; [mce3| mce3:utter(Inst,"nimmst"), Verb(mce3), sem(mce3)= Qx(Q(x’[e| e: grasp(x, x’)]))]; [mce4| mce4:utter(Inst,"Du"), Pro(mce4), sem(mce4)= P.P (you)]; [mce5| mce5:utter(Cnst,"eine"), Det(mce5), sem(mce5)= P’P([y| ]; P’(y); P(y))] [mce6| mce6:utter(Cnst,"Schraube"), Noun(mce6), sem(mce6)= v([ |screw(v)]];

23. MCE1 CE1 mce1:utter(Inst,“so"),ce1:take-turn(Inst), generate(mce1,ce1)]; U1:S U4:NP U2:NP MCE3:”nimmst”:V U3:NP MCE4:”Du”:Pro Syntactic interpretation with MCEs (Poesio, 1996) MCE2 CE2 MCE3 MCE4

24. MCE1 CE1 mce1:utter(Inst,“so"),ce1:take-turn(Inst), generate(mce1,ce1)]; MCE4:”Du”:Pro U1:S U2:NP MCE3:”nimmst”:V U3:NP Syntactic Interpretation with MCEs, II MCE2 CE2 MCE3 MCE4

25. U3:() U1: U2:  Semantic interpretation and compositionality BINARY SEMANTIC COMPOSITION

26. Intentions and obligations OBLIGATIONS: [o | o:OblCnst ([|address(Cnst, ce1)])] INTENTIONS: [i | i:IntInst&Cnst ([|join(Cnst, wing1,fuselage1)])] INTENTIONAL STRUCTURE: Grosz&Sidner-like sp(i1) = i2 dom(i1) = i2

27. Grounding • As in proposals such as Clark and Schaefer (1989) and Traum (1994), establishment of common ground (‘G’) modeled in terms of CONTRIBUTIONS, or DISCOURSE UNITS, that may be ACKNOWLEDGED or REPAIRED

28. DU1 …. DU17 MCE1 CE1 MCE2 CE2 MCE3 … ACK(DU17) mce1:utter(Inst,“so"),ce1:take-turn(Inst), generate(mce1,ce1)]; DU17 = MCE4:”Du”:Pro U1:S CONT(DU17) U2:NP MCE3:”nimmst”:V U3:NP REPAIR(DU17) Discourse Units and Grounding Acts

29. What prompts the completion? Two accounts • Intentional account • Need to explain why help • Alignment account • What representation is aligned? HR

30. Outline of the talk • The Bielefeld Toy Plane Corpus (BTPC) • An introduction to • PTT (Poesio and Traum 1997, 1998; Matheson et al, 2000; Poesio, to appear) • Intro to model of we-intentions we use, following Bratman (1992), Tuomela (2000) and Grosz and Kraus (1996) • A PTT implementation of an intentional analysis of completions

31. Public and private (partial) plans • Inst has a fully specified plan for building the toy-airplane (drawing or model) • The public plan among Inst and Cnst is usually underspecified, but gets more refined throughout the construction dialogue • Besides the shared partial plan, the agents have private plans which overlap to some extent with the shared plan. • The difference between the public plan and the private plans leads to discrepancies and negotiations

32. Assemble toy airplane Assemble fuselage Assemble wing get 5h bar get 3h bar join get 7h bar1 get 7h bar2 join alignwing&fuselage getbolt getnut putthrough getbolt getbolt The partial shared plan before the example Join wing and fuselage

33. We-intentions for shared cooperative activity • Tuomela’s (2000) modification of Bratman’s 1993 definition of SCA, adapted: • Inst and Cnst WE-INTEND that Cnst join wing and fuselage is equivalent to: • It is Inst’s and Cnst’s mutual knowledge that • Inst intends that Cnst join wing&fuselage because Cnst intends that Cnst join wing&fuselage • and • Cnst intends that Cnst join wing&fuselage because Inst intends that Cnst join wing&fuselage

34. Tuomela’s definition of we-intention, formal IntInst&Cnst(join(Cnst, W&F))  MK((IntInst join(Cnst, W&F)) /r IntCnst(join(Cnst, W&F))) and (IntCnstjoin(Cnst, W&F)) /r IntInst(join(Cnst, W&F))))(where /r is the reason relation, which is factual)

35. Outline of the talk • The Bielefeld Toy Plane Corpus (BTPC) • An introduction to • PTT (Poesio and Traum 1997, 1998; Matheson et al, 2000; Poesio, to appear) • Intro to model of we-intentions we use, following Bratman (1992), Tuomela (2000) and Grosz and Kraus (1996) • A PTT implementation of an intentional analysis of completions

36. The situation before example 1 MP

37. Assemble toy airplane Assemble fuselage Assemble wing get 5h bar get 3h bar join get 7h bar1 get 7h bar2 join alignwing&fuselage getbolt getnut putthrough getbolt getbolt The partial shared plan before the example Join wing and fuselage

38. Reaching the intention to perform a directive 1.: (partial) we-intention to join:[i | i:IntInst&Cnst ([|join(Cnst, wing1,fuselage1)])] • 2.: intertwined discourse / domain planjoin(Cnst,Obj1,Obj2) >Ash&Morr b & c & d & e & f, • (b) 1. direct(Inst, Cnst, grasp(Cnst, Bolt)),2. grasp(Cnst, Bolt),3. tell(Cnst, Inst, grasp(Cnst, Bolt )) • 1.direct(Inst, Cnst, grasp(Cnst, Nut)),2. grasp(Cnst, Nut),3. tell(Cnst, Inst, grasp(Cnst, Nut)) • …put through… (e) …fasten.. (f) ….feedback.

39. Deciding to perform a directive, II 3. Distributivity of we-intention  4. Achieve partial we-intention to perform directive:[i1 | i1:IntInst&Cnst ([K1,ce1|K1 = [e,x|bolt(x), e:grasp(Cnst,x)] ce1:direct(Inst, Cnst, K1)])] 5. Achieve partial Inst intention (& Cnst intention)[i2 | i2:IntInst ([K1,ce1|K1 = [e, x|bolt(x), e:grasp(Cnst,x)] ce1:direct(Inst, Cnst, K1)])]

40. Inst’s private plan Assemble toy airplane Assemble fuselage Assemble wing Join wing and fuselage get 5h bar get 3h bar join get 7h bar1 get 7h bar2 join alignwing&fuselage getbolt … getbolt getbolt orange-bolt-with-slit

41. Assemble toy airplane Assemble fuselage Assemble wing Join wing and fuselage get 5h bar get 3h bar join get 7h bar1 get 7h bar2 join alignwing&fuselage getbolt ….. getbolt getbolt Cnst’s private plan bolt

42. Planning the directive 6. arrive at more specific intention (evidence: subsequent repair) [i3 | i3:IntInst ([K2,ce2|K2 = [e, x|bolt(x),x=orange-slit-bolt, e:grasp(Const,x)] ce2:direct(Inst, Cnst, K2)])] 7. develop plan to perform utterance that generates directive: [i4 | i4:IntInst ([u1.1 | utterance(u1.1), sem(u1.1) = K2, generates(u1.1,ce2)])]

43. Micro-plan 8. plan to perform utterance in terms of MCEs [i5a | i5a:IntInst ([u1.2 | u1.2: „so“:take-turn])]; [i5b | i5b:IntInst ([u1.3 | u1.3:“jetzt“:keep-turn])]; [i5c | i5c:IntInst ([u1.4 .. u1.7| S(u1.1) , u1.4:“nimmst“:V, u1.5:“Du“:NP, u1.7:VP, NP(u1.6),u1.5 u1.1, u1.7 u1.1, u1.4  u1.7, u1.6  u1.7] 9. lengthening signals problem - Inst doesn’t necessarily know which bolts are unused  possibly does not know how to refer to bolt (NP type / content)

44. Possible motivations for the completion • Interpret lengthening as request to continue • obl(Cnst, cont(DU)) • Interpret lengthening as request for acknowledgment (also standard PTT) • obl(Cnst, ack(DU)) • Cooperativeness • `Blurting out’

45. A cooperativeness analysis 10. Cnst acquires intention to turn the directive in a joint action (cfr. Tuomela’s “unrequired contributory actions”) [i6 | i6:IntCnst ([K3,ce3|K3 = [e,x|bolt(x), e:grasp(Const,x)] ce3:direct(Inst&Cnst, Cnst, K3)])]Derivation of 10 not axiomatized by Tuomela, but we assume here is the result of an intention to help. 11. Cnst produces plan to perform action to generate directive; analogous to Inst’s, but content (partial) K3: [i7 | i7:IntCnst ([u1.1a | utterance(u1.1a), sem(u1.1a) = K3, generates(u1.1a,ce3)])]

46. Further specification not possible

47. Micro-plan for Cnst’s completion 12. most of actions in plan already performed by Inst; Cnst plans missing action [i8 | i8:IntCnst ([ K1.1d | u1.6:“eine Schraube“:NP,K1.1d=[x1|bolt(x1)]sem(u1.6) = K1.1d ])] (Same action would be planned to continue contribution and to acknowledge) 13. Instructor begins repair due to his private plan

48. An alternative analysis: the IAM model • Successful dialogue involves the development of aligned representations at all levels • Aligned representations the result of priming mechanisms at every level of linguistic representation • Mental state reasoning an option but not basic

49. Alignment at all levels

50. An IAM-analysis of the BTPC example • What leads Cnst to produce “eine Schraube”? • What is a situation model in this domain? • “the key dimensions encoded in situation models are SPACE, TIME, CAUSALITY, INTENTIONALITY, and REFERENCE to the MAIN INDIVIDUALS … “ (p. 7)