Miscommunication Workshop QMUL, London, January 26, 2006 Staffan Larsson Göteborg University

1. Semantic plasticity through feedback and accommodation in dialogue(Meaning through Miscommunication) Miscommunication Workshop QMUL, London, January 26, 2006 Staffan Larsson Göteborg University sl@ling.gu.se

2. Overview • Introduction • Negotiation, feedback, accommodation • Semantic plasticity • Meaning accommodation • Conclusions

3. Introduction

4. Questions • What goes on in dialogue? • Information exchange, but also: • Explicit and implicit negotiation of meaning (and other aspects of language) • Which mechanisms govern these processes? • How are these processes related? • What is meaning? • Is meaning private or social? In what sense? • How is meaning and use of language related? • What is the role of formal representations in a theory of meaning? • What is the role of subsymbolic cognition in a theory of meaning?

5. Two kinds of coordination in dialogue • Information sharing • Sharing of symbolically / linguistically represented information • Grounding (updating common ground) • Coordination of linguistic resources (language system) • Semantic change and adaptation (& linguistic change in general) • Adaptation of linguistic resources to activity or situation • Adaptation of linguistic resources to specific partner • Ad-hoc adaptation to a specific conversation • Long-term changes in language over time • Linguistic coordination is necessary for information sharing • Establishes shared meaning of linguistic constructs used to share information

6. Work on dynamics of information sharing • Starting point: language games (Wittgenstein 1953) • Formal descriptions of language games • Dialogue grammars (Sinclair & Coulthard) • Dialogue games as finite state automata (Lewin et al) • Game-theoretical accounts • Carlson 1983 • Jaeger et al (recent) • … and lots more… • Research on formal and computational accounts of dialogue games (SemDial workshop series 1997-); TRINDI and later projects • Dialogue games as Information State Updates • In terms of sequences of dialogue moves • Moves trigger updates to a Dialogue Gameboard (which is part of a more inclusive Information State) • Example: Issue-Based Dialogue Management (Larsson, Ginzburg)

7. Work on dynamics of language system • Halliday • Meaning potentials, adapting linguistic resources to specific activities • Clark • Creative language use: creating verbs from nouns • Brennan • Conceptual pacts, lexical entrainment • Healey • Emergence of shared vocabularies in groups vs. across groups • Cutler et al • Phonetic plasticity • Pickering & Garrod • Alignment of multiple levels in dialogue • Steels et al • Emergence of shared categories through social and embodied linguistic learning in robots

8. Common theme in work on dynamics of language • How new vocabulary emerges • How words acquire or change meaning through social linguistic interaction • Additional themes • Language is activity-specific (dialogue games) • Language and embodiment

9. Symbolic and nonsynbolic/subsymbolic • According to Dreyfus, Brooks and others, cognition is not based on symbolic representations • Dreyfus claims that human language understanding relies on a background which cannot be represented as a set of facts • Still, human language is the prototypical symbol system so linguistic cognition must involve symbolic representations • How does symbolic and nonsymbolic cognition interact in language understanding and dialogue in general?

10. Some theoretical issues • Give accounts and models of • Individual usage dispositions and how they are updated • How social meaning is related to usage dispositions • How meaning is negotiated, tacitly and explicitly, and how this relates to updates of usage dispositions • Dialogue games for meaning negotiation • Tacit negotiation through feedback and accommodation • Integrate account of semantic dynamics with existing accounts of dynamics of symbolic (linguistic) information updates, into a general account of alignment/coordination in dialogue

11. Negotiation, feedback, accommodation

12. Three important processes of coordination in dialogue • Explicit negotiation • “Negotiation” used here in weak sense of “reaching a joint decision” (may be antagonistic or cooperative) • E.g. ostensive language games (Steels & Belpaeme 2005: “the guessing game”), explicit verbal definitions • Feedback • Signalling perception, understanding, acceptance • Signalling failure to perceive or understand; clarification; rejection • Guides coordination of DGB (Grounding) • >Also: guides coordination of language use • Accommodation: Adapting to the behaviour of other DPs • Adapting to presuppositions (adapting the DGB) • >Also: adapting to language use (adapting linguistic resources)

13. Accommodation • Conversational scoreboard • Dialogue gameboard (DGB) • (Part of) Common Ground (CG) • (Shared part of) Information State (IS) • Lewis (1979): • Dialogue tends to evolve in a way that makes any move count as correct play • “If someone says something at t which requires X to be in the conversational scoreboard, and X is not in the scoreboard at t, then (under certain conditions) X will become part of the scoreboard at t” • Has been applied to referents and propositions, as parts of the conversational scoreboard • E.g. “Bo snores” or “My cat is hungry” presupposes referent in DGB • If no matching referent in DGB, hearer may ask clarification question • but only after failure to accommodate

14. Issue-based Dialogue Management • (Larsson, Ginzburg) • DGB includes a stack of Questions Under Discussion (QUD) • Answers require a matching question before they can be accepted and integrated RULE:integrateAnswer PRE: in( $/SHARED/LU/MOVES, answer(A) ) fst( $/SHARED/QUD, Q ) $DOMAIN :: relevant( A, Q ) EFF: ! $DOMAIN: combine( Q, A, P ) add( /SHARED/COM, P )

15. Typical human-human dialogue S(alesman), C(ustomer) S: hi C: flight to paris S: when do you want to travel? C: april, as cheap as possible

16. Typical human-human dialogue S(alesman), C(ustomer) S: hi C: flight to paris S: when do you want to travel? C: april, as cheap as possible How do you want to travel? Where do you want to travel? What price range?

17. Question accommodation • If questions are part of the DGB, they too can be accommodated • If the latest move was an answer, and there is an matching question which is relevant in the activity at hand, then • put that question on QUD

18. Coordination and accommodation • “... someone says something at t which requires X to be in the conversational scoreboard, and X is not in the scoreboard at t...” • This may indicate a case of lack of coordination • but may also be used as a strategy for conveying implicit information (Grice) • In any case, accommodation can be used to adjust to some presupposition of what is shared

19. Feedback in dialogue • Feedback: signals for achieving coordination (alignment, grounding) on several levels of action (Allwood, Clark) • Contact / attention: +/- • Perception: +/- • Understanding: +/-/? • Reaction: accepting and rejecting utterances • The hearer can react to whole utterances or sentences, or to some part of an utterance • A word, a phrase, a grammatical construct, or in general any linguistic construct • Example: Clarification Requests (Ginzburg)

20. Grounding • ”To ground a thing … is to establish it as part of common ground well enough for current purposes.” (Clark) • Common Ground includes • general facts about the world (commonsense knowledge) • More specific facts about the world (e.g. facts about history) • facts about words (dog can mean ”canine animal”) • and more • Henceforth, we will use grounding in a more limited sense • The process of adding information to the ”Dialogue Gameboard” (DGB) or Conversational Scoreboard

21. Semantic plasticity

22. Structuralism • The sign relation, i.e. the connection between words (linguistic form) and concepts is arbitrary • The way that linguistic material is divided into words is arbitrary • The way that the world is divided into concepts is arbitrary • Focus on study of language as a structure (langue); the concrete use of language (parole) assumed too unruly for scientific study

23. Poststructuralism • Langue is continuously being affected by parole • Words change their meanings over time as a result of language use • If or concepts determine how we understand the world... • Concrete language use changes our understanding of the world • Communication is not (just) transmission of information • Science studies, Critical Discourse Analysis & related disciplines study how people fight over the use of certain words; “man”, “gender”, “gene”, “terrorism”, …

24. Phonetic plasticity • Cutler, McQueen, Norris (2005) • ACL paper • Experiment: • Ambiguous phoneme /?/ between /f/ and /s/ • Group A hears words where /?/ replaces /f/, e.g. ”carafe” • Group B hears words where /?/ replaces /s/

25. Result • For group A, the /f/ category became more inclusive (tested by phoneme categorisation) • For group B, the /s/ category became more inclusive • Exposure to /?/ in non-word context had no effect • Effect generalised to new words, and thus facilitates word recognition

26. Semantic plasticity • As with phonemes, semantic categories can (presumably) gradually expand, contract, and shift

27. Kinds of semantic plasticity • Semantic systems exist on several levels • National • Regional • Domain, activity, language game • Personal (idiosyncratic) • Particular interactions (dialogues) • Semantic system can be adapted • Example: “the one with the handle going across”: map “handle” to certain shape on card (Brennan) • to a new activity or domain • to a certain individual (who has an idiosyncractic way of using some concept) • to a certain interaction (ad-hoc) • Associated issue: • When & to what extent does “idiosyncratic” usage of some word in a single dialogue affect its meaning in general? • Plasticity vs. elasticity • Indeed, are there “meanings in general” which are adapted to specific activities or are meanings just borrowed from other (equally specific) activities?

28. Sketch of a formal general account of semantic plasticity • Meaning emerges from a multitude of interactions where the DPs of a linguistic community shape each other’s usage dispositions • A language-user A observes some linguistic construct c being used in a set of situations Sc (situational collocation for c) • A generalises over Sc; this generalisation we call the usage disposition [c] • The way [c] is updated after a use depends on the feedback given by other DPs • A: “Alignment is automatic” • B; “Uhuh” / “I agree” / “Automatic?” / “No way!” / “Eh?” / ... • [“automatic”] may get updated for A or B or both

29. Usage dispositions • Instead of modelling semantic plasticity directly, we will model usage dispositions • Private (“in the head”), but socially conditioned • Do not completely determine usage, but affects it • Dispositions are affected by feedback • The usage-dispositions of the members of a linguistic community must be sufficiently similar to allow for coordinated behaviour • The model will allow for creative language use • The following is an abstract “skeleton” account that can be filled out in may different ways

30. Situational collocation • Abstractly, each individual A can be associated with a situational collocation ScA • Contains the situations in which A has observed c to be used (including uses by A) • ScA = { s | c was observed by A to be used in situation s } • We won’t say more about what a situation is here; can be fliled out in various ways to include e.g. • Linguistic context • Referential context • “Connotational” aspects

31. Successful and unsuccessful uses • Consider a situation where c is used in s, and c is addressed to B. • B can understand c or fail to do so • Provided B understands (or thinks he understands), B can choose to accept or reject c • In the following, we will (for the most part) restrict feedback to • acceptance (and understanding) • negative understanding (and no acceptance) • positive understanding, rejection • Successful use: understood and accepted • Unsuccessful use: not understood, or rejected

32. Usage dispositions: more • Presumably, an individual A somehow generalises over situational collocations • (Alternatively, remember completely all situations where each constructs was used) • We refer to this generalisation as A's usage disposition for c • An agent A’s usage disposition for c, written as [c]A, is a function of the situational collocation for c • [c]A = fdisp-A(ScA) • Related points: • This function may capture any aspects of the situation that are relevant to language use; physical, biological, psychological, sociological, linguistic • To handle ambiguity, this function should allow for differentiated generalisations over subsets of ScA • Does the generalisation function differ • Between speakers? • Between linguistic categories (nouns, verbs, grammar rules, …)?

33. Judging appropriateness • We assume that something like “estimated degree of appropriateness” or “consistency with previous uses” plays a role in language use • Appropriateness of using c in s is a function of the usage disposition for c, and s • appr(c, s)A = fappr([c]A,s) = fappr(fdisp(ScA), s) • In a simple case, we can model fappr with a boolean function (either appropriate or non-appropriate) • Or it could be a numerical value, e.g. between 0 and 1 • Related points • This is not meant to imply a conscious judgement… • … nor that it is relevant or even possible to say whether the estimation of appropriateness is “correct” • May be used to model some notion of “linguistic intuition” governing own language use including how one gives feedback

34. Indeterminacy • Upon hearing c in new situation s, A’sreaction (the kind of feedback A gives) partly depends on [c] • But A’s behaviour is not determined by [c], or even Appr(s,c) • This means that A can understand and accept uses of c that deviate from [c] • A’s own future uses of c are partly determined by [c] • Again, A’s own use of c is not determined by [c], or even Appr(s,c) • A can use c in ways that deviate from [c]

35. Disposition updates • If follows from the definition of dispositions that whenever a construct c is used, Sc will be extended, and so the usage-disposition [c] may change • This is a disposition update • Disposition reinforcement • This use of c is consistent with usage disposition, i.e., c is appropriate in s • No drastic change; previous disposition is reinforced • Disposition revision • This use of c is non consistent with usage disposition • More or less drastic change of meaning; previous meaning is revised • Related points • Arguably, all language uses are creative since all situations are different • But intuitively, some uses are more creative than others • To what extent can we distinguish revision from reinforcement? • Are updates continuous or discrete • Depends on the kind of linguistic construct? • Scalar terms, e.g. colour terms appear to be continuous

36. The usage equation • use(c, s)A = fuse( fappr(fdisp(ScA), s), X ) • What does this mean? • Whether A uses c in s depends on • s: The current situation • ScA: Situational collocation for c - previous situations where c has been used, in A’s experience • fdisp: The way A generalises over these • fappr: The way A uses this generalisation do determine the appropriateness of c in s • Any additional factors X • the ways in which these, in conjunction with the appropriateness judgement, affect whether c is actually used (fuse)

37. Possible outcomes of using c in s

38. From usage dispositions to meaning • The meaning of a construct c in a linguistic community L emerges from the coordinated use of c by the members of L • Meaning is inherently social, and arises out of coordinated behaviour in a linguistic community • In interaction, members of a community “mould” each others’ usage dispositions by giving feedback and accommodating usage • This keeps language use sufficiently coordinated for meaning to arise • By modelling plasticity of usage dispositions, we indirectly model semantic plasticity

39. Meaning accommodation

40. Possible outcomes of using c in s

41. Meaning accommodation • For each construct used in an U, the addressee in a dialogue is (usually) expected to react if he thinks a construct in U was incomprehensible or inappropriately used • If a breakdown occurs during interpretation of U by B, it may be due to a mismatch between the situation in which c was being used by A, and B’s usage disposition for c • The addressee B may now • reject this use of c explicitly: negative feedback on understanding or acceptance level • or quietly alter B’s usage disposition for c so that c can be counted as appropriate after all. • The latter process we may call usage accommodation, or meaningaccommodation • This extends the notion of accommodation beyond the DGB, to include the language system

42. Two variants of meaning accommodation • Accommodated creative use • Speaker non-appropriate, Hearer non-appropriate, successful • New use not “appropriate” according to speaker but speaker tries it anyway • revise [c]A and [c]B • Accommodated conservative use • Speaker appropriate, Hearer non-appropriate, successful • Not really creative since the speaker followed her appropriateness judgement, but hearer had not heard that use before • reinforce [c]A, revise [c]B • Example (of either of the above) • (in 1991 or so) • A: What are you doing? • B: I’m surfing the web • A: ... Ah, OK.

43. Latent subgames and “gears” in dialogue • Cohen (1978), Severinsson-Eklundh (1983) • Dialogue involves “latent subgames” or “tacit moves” that can become explicit if necessary • Example: referent identification subgame, perception subgame • A: Pick up the red one • B: (hears “pick up the red one”) • B: (identifies red object; there are two but only one has been discussed previously) • B: (picks up red object) • A: Pick up the red one • B: (hears “pick up the red one”) • B: Do you mean this one? (points to x) • A: No, this one (points to y) • B: (picks up red object) • A: Pick up the red one • B: Pardon? • A: Pick up the red one • B: …

44. Accommodation as tacit negotiation • Meaning accommmodation can be seen as tacit negotiation of meaning • When dialogue proceeds smoothly, speakers shift to high gear and assume that latent subgames will succeed • Optimistic strategy • Tacit moves / games • When problems (breakdowns) occur, participants shift into low gear and give explicit feedback, request confirmation, etc. • Cautious strategy • Latent subgames are played out explicitly • Explicit negotiation can often be regarded as occurring in “low gear” • Accommodation can be seen as a tacit dialogue move(or game), which replaces an explicit subgame of negotiation

45. Explicit negotiation vs. accommodation • Meaning can change as a result of explicit negotiation or accommodation • Explicit negotiation • Hearer detects problem and initiates negotiation subgame to resolve it • Accommodation • Hearer detects has problem understanding or finds utterance otherwise inappropriate or problematic, but nevertheless is able to adapt to the situation and proceed with the interaction • Commonsense background and general redundancy in linguistic interaction enables efficient tacit negotiation through accommodation

46. Conclusions

47. Coordination of meaning through miscommunication • “Perfect communication” • Perception, understanding and agreement are unproblematic • No negotiation, feedback or accommodation is needed • “Miscommunication” • Perception, understanding and agreement are fallible • Feedback is needed to monitor success • Accommodation is needed for efficiently adapting to differences in DGB and language system • These mechanisms enable coordination of meaning through the interactive adaptation of usage dispositions • Miscommunication is thus the very basis for linguistic plasticity and adaptation • If there was no miscommunication, language would be static and unadaptable • Unless changes where instantaneous and universal, which does not seem very plausible

48. Summary • Three types of mechanisms for dealing with miscommunication • Explicit negotiation • Feedback • Accommodation • … enable coordination of... • Dialogue Gameboard updates • Dynamics of semantic system (semantic plasticity) • Coordination of DGB presupposes a sufficiently coordinated language system • In “low gear”, explicit negotiation of usage / meaning is played out explicitly • In dialogue in “high gear”, accommodation enables adaptation of usage dispositions (and thus coordination of usage) • Meaning emerges and changes not only as a result of explicit negotiation, but also as a “side-effect” of tacitly resolving problems in information sharing

49. Additional material

50. future work