Accommodation, feedback and semantic plasticity

1. Accommodation, feedback and semantic plasticity Bielefeld Dialogue Colloquim November 18, 2005 Staffan Larsson Göteborg University

2. Background • Two important dialogue processes • Feedback • Signaling perception, understanding, acceptance • Signaling failure to percieve & understand; rejection • Accommodation: Adapting to the behaviour of other DPs • Adapting to presuppositions (adapting the situation model) • Adapting to language use (adapting the language / register) • Two kinds of coordination in dialogue • Information sharing • Grounding (updating common ground) • Adaptation of linguistic resources • Semantic plasticity (& linguistic plasticity in general) • Adaptation of linguistic resources to activity • Gradual changes in language over time

3. Overview • Introduction • Issue-based Dialogue Management • Accommodating questions • Accommodating meanings • An interpretation of Pickering & Garrod (2005)

4. Introduction

5. Accommodation • Lewis (1979): If someone says something at t which requires X to be in the conversational scoreboard, and X is not in the scoreboard at t, then (under certain conditions) X will become part of the scoreboard at t • Has been applied to referents and propositions, as parts of the conversational scoreboard / information state

6. Coordination and accommodation • “If someone says something at t which requires X to be in the conversational scoreboard, and X is not in the scoreboard at t...” • This seems to indicate a case of lack of coordination • but may also be used as a strategy for conveying implicit information • Accommodation can be used when there is miscoordination • Adjustment to presupposition of what is shared

7. Feedback in dialogue • Feedback: signals for achieving coordination (alignment, grounding) on several levels (Clark, Allwood) • Contact / attention: +/- • Perception: +/- • Understanding: +/-/? • Reaction: accepting and rejecting utterances • The hearer can react to some part of an utterance • A word, a phrase, a grammatical construct, or in general any linguistic construct • Example: Clarification Requests (Ginzburg)

8. Grounding • ”To ground a thing … is to establish it as part of common ground well enough for current purposes.” (Clark) • Common Ground includes • general facts about the world (commonsense knowledge) • More specific facts about the world (e.g. facts about history) • facts about words (dog can mean ”canine animal”) • and more • Henceforth, we will use grounding in a more limited sense • The process of adding information to the ”Dialogue Gameboard” (DGB) or Conversational Scoreboard

9. Two kinds of coordination in dialogue • Coordination of DGB • Adding new information (questions, propositions, actions) to DGB • Regulated by • feedback • question accommodation • existensial presupposition accommodation • ... • Coordination of semantic system, or “register” • Regulated by • feedback • meaning accommodation • ...? • Coordination of DGB presupposes a sufficiently coordinated conceptualisation of domain and situation, vocabulary, etc. (language system)

10. Issue-based Dialogue Management

11. Issue-based dialogue management • A computational theory for improving flexibility of linguistic human-computer interaction, based on general principles inspired by observations of human-human dialogue • Implemented in the GoDiS dialogue system • Background theory • Ginzburg’s Dialogue Gameboard (DGB) and • Related DGB update protocols • Feedback and grounding (Ginzburg, Clark, Allwood) • Accommodation (Lewis)

12. Dialogue moves: ask, answer, greet, quit • Dialogue is, basically, all about raising and addressing issues • incl. short answers. e.g.”yes”, ”no”, ”paris”, ”in april” • Dialogue plans drive dialogue in the absence of user intiative • Extension: • Handling multiple issues • Question accommodation • Feedback • ...

13. GoDiS control DME input inter- pret update select gene- rate output • TIS • DATABASE LEXICON DOMAIN data- base lexicon domain knowledge

14. Basic GoDiS information state record type • SHARED = DGB / scoreboard • QUD: Questions which have been raised but not resolved, and which are available for resolution of short answers AGENDA : stack( Action ) PLAN : stackset( Action ) PRIVATE : BEL : set( Prop ) COM : set( Prop ) QUD : stackset( Question ) SHARED : SPEAKER: Speaker MOVES: OpenQueue(Move ) LU:

15. Sample dialogue plan < findout(?x.dest-city(x)) findout(?x.depart-city(x)) findout(?x.transport(x)) findout(?x.dept-month(x)) findout(?x.dept-day(x)) raise({?class(economy), ?class(business)} consultDB(?x.price(x)) >

16. Answer integration RULE:integrateAnswer PRE: in( $/SHARED/LU/MOVES, answer(A) ) fst( $/SHARED/QUD, Q ) $DOMAIN :: relevant( A, Q ) EFF: ! $DOMAIN: combine( Q, A, P ) add( /SHARED/COM, P ) • Before an answer can be integrated by the system, it must be matched to a question on QUD

17. Basic dialogue with updates U: ”price information please”; raises price issue • if user asks Q, push Q on QUD and push respond(Q) on AGENDA • if respond(Q) on AGENDA and PLAN empty, find plan for Q and load to PLAN • if findout(Q) first on PLAN, ask Q S: ”where do you want to go?” U: ”Paris” • if LU/MOVES contains answer(A) and A relevant to Q, add P=Q[A] to SHARED.COM • if P in SHARED.COM and Q topmost on QUD and P resolves Q, pop QUD • if P in SHARED.COM and P resolves Q and findout(Q) on PLAN, pop PLAN

18. Basics cont’d • … S: ”Do you want economy class or business class?” U: ”economy class” • if consultDB(Q) on PLAN, consult database for answer to Q; store result in PRIVATE.BEL • if Q on QUD and P in PRIVATE.BEL s.t. P resolves Q, answer(P) S: ”The price is £123”

19. A problem with QUD A: Who won the world cup? B: Where? A: in Italy B: Are you sure it wasn’t in France? A: Yes B: Where did you say it was again? A: Italy B: Germany • If QUD=<q1,q2> and q1 is resolved, q2 is available for resolution of short answers • takes no account of how many turns since q2 was raised • but short answers a long distance away from the question are not as easily processed as an adjacent answer

20. ISSUES and QUD • We extend Ginzburg’s DGB by adding ISSUES of type Stack(Question) • ISSUES contains all raised but unresolved questions • ISSUES determines relevance of user answers • QUD used for resolving short answers • questions drop off QUD after N turns • a short answer to a question that’s on ISSUES but not QUD requires adjusting QUD by copying a question on ISSUES

21. Grounding and feedback • Grounding • More limited sense than Clark’s: the process of adding information to the ”Dialogue Gameboard” (DGB) • making sure that the participants are percieving, understanding, and accepting each other’s utterances • ICM • Interactive Communication Management • Feedback, Sequencing, Turntaking • Feedback moves • (short) utterances which signal grounding status of previous utterance • Sequencing moves • utterances which signal switching task (”returning to…”), reraising questions (”so…”) etc.

22. Grounding moves • We want a typology of feedback moves • Some parameters: • Action levels in dialogue (Allwood, Clark) • contact (con) • perception (per) • understanding • Word meanings (sem) • Contextual meaning (und) • acceptance (acc) • Polarity • Positive (pos) • Negative (neg) • Checking (chk / int)

23. Some grounding moves in GoDiS • Formal representation • icm:Level/Type{*Polarity}{:Content} • Feedback moves • icm:und*neg – ”I don’t understand” • icm:und*pos:P – ”To Paris.” • icm:acc*neg:Q – ”Sorry, I can’t …” • icm:acc*pos – ”Okay” • Feedback type selected depending on • Quality of recognised speech • Whether system can find a (relevant) interpretation • Whether system can accept what’s been said • Sequencing moves • icm:reraise:Q – ”Returning to the issue of Q” • icm:loadplan – ”Let’s see…”

24. Grounding on multiple levels S> Lets see. What can I do for you? U> (mumble mumble) S> Sorry, I didnt hear what you said. What can I do for you? U> siridus S> I heard you say siridus . Sorry, I dont understand. What can I do for you? U> channel one S> channel one. I dont quite understand. Do you want to add a program or change channel? (dialog3a_edit) Negative perception feeback Positive perception Negative understanding (sem) Positive understanding (word meaning) Negative understanding (und) Task clarification

25. Positive understanding Rejection (negative acceptance feedback) U> turn on the t v S> turn on the TV… Sorry, I cannot control the TV set. Do you want to add a program or change channel?” U> pardon S> Sorry, I cannot control the TV set. Do you want to add a program or change channel? U> change channel S> Okay. (dialog3b_edit) User negative perception System repetition Positive acceptance

26. Accommodating questions

27. Question accommodation • If questions are part of the information state, they too can be accommodated • If the latest move was an answer, and there is an action in the plan to ask a matching question, then • put that question on ISSUES • (and QUD if it is a short answer) • Requires that the number of possible matching questions is not too large • (or can be narrowed down by asking clarification question)

28. Issue accommodation PLANISSUES • If • LM=answer(A) • no Q in ISSUES s.t. relevant(A,Q) • then • find findout(Q) in PLAN s.t. relevant(A,Q) • push Q on ISSUES • used when prevously unraised question (available in plan) is answered using a short or full answer

29. QUD accommodation ISSUESQUD • If • LM=answer(A) • A is a short answer • no Q in QUD s.t. relevant(A,Q) • then • find Q in ISSUES s.t. relevant(A,Q) • push Q on QUD • raise Q in ISSUES (make Q topmost) • used when • previously raised question has dropped off QUD, but is answered using a short answer • previously unraised question is answered using short answer [needs PLANISSUES accommodation]

30. Dependent issue accommodationDOMAINISSUES (+PLAN) • If • LM=answer(A) • no Q in ISSUES s.t. relevant(A,Q) • no findout(Q) in PLAN s.t. relevant(A,Q) • then • find Plan for some Q’ in DOMAIN s.t. findout(Q) or raise(Q) in Plan and relevant(A, Q) • push Q’ on ISSUES • set PLAN to Plan • used when previously unraised question, unavailable in PLAN, is answered using full or short answer (AKA ”task accomodation”)

31. Sample dialogue: accommodation S: Welcome to the travel agency. U: From London to Paris in April • not relevant to any question that has been raised, or to any current plan • look in domain knowledge for a plan (for dealing with some question Q) with matching questions • load this plan, push Q on ISSUES • find in the plan the question(s) matching the user’s answer, and push them onto ISSUES • integrate answer (requres matching question on ISSUES) S: Alright, you want to know about price. (…) • proceed to next plan item S: How do you want to travel? • ISSUES=<?x.how(x), ?x.price(x)>

32. Task clarification (VCR application) S> What can I do for you? U> channel five S> channel five. I dont quite understand. Do you want to add a program or change channel? U> change channel S> Okay. change channel. S> Lets see. channel five? U> yes S> The channel was changed

33. Status of question accommodation mechanisms • QAMs are primarily a set of techniques (based on a common principle) for making dialogue systems more flexible in adapting to the user’s behaviour • Are QAMs cognitively plausible? • Remains to be investigated

34. Accommodating meanings

35. Structuralism • The sign relation, i.e. the connection between words (linguistic form) and concepts is arbitrary • The way that linguistic material is divided into words is arbitrary • The way that the world is divided into concepts is arbitrary • Focus on study of language as a structure (langue); the concrete use of language (parole) assumed too unruly for scientific study

36. Poststructuralism • Langue is continuously being affected by parole • Words change their meanings over time as a result of language use • If or concepts determine how we understand the world... • Concrete language use changes our understanding of the world • Communication is not (just) transmission of information

37. Phonetic plasticity • Cutler, McQueen, Norris (2005) • ACL paper [find it] • Experiment: • Ambiguous phoneme /?/ between /f/ and /s/ • Group A hears words where /?/ replaces /f/, e.g. ”carafe” • Group B hears words where /?/ replaces /s/

38. Result • For group A, the /f/ category became more inclusive (tested by phoneme categorisation) • For group B, the /s/ category became more inclusive • Exposure to /?/ in non-word context had no effect • Effect generalised to new words, and thus facilitates word recognition

39. Semantic plasticity • As with phonemes, semantic categories can gradually expand, contract, and shift

40. Kinds of semantic plasticity • Semantic systems exist on several levels • National • Regional • Domain, activity, language game • Personal (idiosyncratic) • Particular interactions • Semantic system can be adapted • to a new activity or domain • to a certain individual (who has an idiosyncractic way of using some concept) • to a certain interaction (ad-hoc) • Semantic systems can change • over time (diachronic semantics) • over space

41. Sketch of a formal general account of semantic plasticity • Meaning emerges from a multitude of interactions where the DPs of a linguistic community shape each other’s usage dispositions • Cf. simulation work in this direction by Steels et al. • A language-user A observes some linguistic construct c being used in a set of situations Sc • A generalises over Sc; this generalisation we call the usage disposition [c] • The way [c] is updated after a new use depends on the feedback given by other DPs

42. Indeterminacy • Upon hearing c in new situation s, A’sreaction (the kind of feedback A gives) partly depends on [c] • But A’s behaviour is not determined by [c] • This means that A can understand and accept uses of c that deviate from [c] • A’s own future uses of c are partly determined by [c] • Again, A’s own use of c is not determined by [c] • A can use c in ways that deviate from [c]

43. Disposition updates • If follows from the definition of meaning that whenever a construct c is used, Sc will be extended, and so the usage-disposition [c] may change • This is a disposition update • Disposition reinforcement • This use of c is consistent with usage disposition, i.e., c is appropriate in s • No drastic change; previous disposition is reinforced • Disposition revision • This use of c is non consistent with usage disposition • More or less drastic change of meaning; previous meaning is revised

44. Disposition reinforcement • Assume c was used in situation s • Disposition reinforcement • This use of c is consistent with usage disposition, i.e., c is appropriate in s • Appr(c, s)A = True • No drastic change; previous disposition is reinforced • Monotonic disposition update: [c]A := [c]A + s • Any uses previously deemed appropriate are still appropriate

45. Disposition revision • Disposition revision • This use of c is non consistent with usage disposition • Appr(c,s)A = false • More or less drastic change of meaning; previous meaning is revised • Nonmonotonic meaning update: [c]A := [c]A * s • some uses previously deemed appropriate would now be deemed non-appropriate • Exactly how the function is updated depends on the learning mechanism • This distinction does not make sense for memory-based learning models

46. The usage equation • Use(c, s)A = fuse( fappr(fdisp(ScA, s)), X ) • What does this mean? • Whether A uses c in s depends on • s: The current situation • ScA: Situational collocation for c - previous situations where c has been used, in A’s experience • fdisp: The way A generalises over these • fappr: The way A uses this generalisation do determine the appropriateness of c in s • Any additional factors X

47. Possible outcomes of using c in s

48. Meaning accommodation • For each construct used in an U, the addressee in a dialogue is (usually) expected to react if he thinks a construct in U was incomprehensible or inappropriately used • Clarification Ellipsis: negative understanding • If a breakdown occurs during interpretation of U by B, it may be due to a mismatch between • the situation in which c was being used by A, and • B’s usage disposition for c (or B’s reaction disposition, if it is different from the usage disposition)

49. The addressee B may now • either reject this use of c explicitly: negative feedback on understanding or acceptance level • or quietly alter B’s usage disposition for c so that c can be counted as appropriate after all. • The latter process we may call usage accommodation, or meaningaccommodation

50. Two variants of accommodated use • Accommodated creative use • Speaker non-appropriate, Hearer non-appropriate, successful • New use not “appropriate” according to speaker but speaker tries it anyway • revise [c]A and [c]B • Accommodated conservative use • Speaker appropriate, Hearer non-appropriate, successful • Not really creative since the speaker followed her appropriateness judgement, but hearer had not heard that use before • reinforce [c]A, revise [c]B • Example (of either of the above) • (in 1991 or so) • A: What are you doing? • B: I’m surfing the web • A: ... Ah, OK.