Generating Feedback and Sequencing Moves in a Dialogue System

1. Generating Feedback and Sequencing Moves in a Dialogue System AAAI Spring Symposium 2003 Staffan Larsson Göteborg University, Sweden

2. Overview • Interactive Communication Management (ICM) • ”Verification” in dialogue systems • Classifying and formalising ICM • ICM for a dialogue system • Examples • Conclusions & Future work

3. ICM (Allwood) • Interactive Communication Management • As opposed to Own Communication Management (OCM): self-corrections, hesitations, etc. • Feedback moves • (short) utterances which signal grounding status of previous utterance (”mm”, ”right”, ”ok”, ”pardon?”, ”huh?” etc.) • Sequencing moves • utterances which signal dialogue structure (”so”, ”now”, ”right”, ”anyway” etc.) • Dialogue structure part of / modeled by common ground • Turntaking moves

4. Grounding and ICM in current commercial systems • Limited to ”verification” • Examples (San Segundo et. al. 2001) • I understood you want to depart from Madrid. Is that correct? [”explicit v.”] • You leave from Madrid. Where are you arriving at? [”implicit v.”] • Involves repetition or reformulation • Appears in H-H dialogue, but not very common

5. From verification to ICM in dialogue systems • ”Verification” is just one type of ICM behaviour • Perhaps the one most cruicial in dialogue systems given poor speech recognition • Could a wider range of the ICM behaviour occurring in H-H dialogue be useful in dialogue systems? • We want a typology of ICM moves for H-H dialogue • Feedback and sequencing moves • We want to formalise it and use it in a system • Still we will implement only a subset, but more than verification

6. Classifying feedback • Level of action • Polarity • Eliciting / noneliciting • Form (syntactic realisation) • Content type: object- or metalevel

7. Feedback levels • Action levels in dialogue (Allwood, Clark, Ginzburg) • Contact: whether a channel of communication is established • Perception: whether DPs are perciveving each other’s utterances • Understanding: Whether DPs are understanding each other’s utterances • Non-contextual (”semantic”) meaning • Contextual (”pragmatic”) meaning • Acceptance: Whether DPs are accepting each other’s utterances • The function of feedback is to signal the status of utterance processing on all levels

8. Feedback polarity • Polarity (Allwood et.al. 1992) • Positive: indicates contact, perception, understanding, acceptance • Negative: indicates lack of contact, perception, understanding, acceptance • We add a ”neutral” or ”checking” polarity – there is one or more hypotheses, but the DP lacks confidence in them • Examples • ”I don’t understand”: negative • ”Do you mean that the destination is Paris?”: checking • ”To Paris.”: positive • ”Pardon”: negative

9. Formalising ICM dialogue moves • Action levels • con: contact • per: perception • sem: semantic understanding (no context) • und: pragmatic understanding (relevance in context) • acc: acceptance • Polarity • pos • neg • chk (”int” in paper)

10. Feedback move notation • icm:Level*Polarity{:Args} • icm:per*pos:String – ”I heard you say ’londres’” • icm:und*neg – ”Sorry, I don’t understand” • icm:und*chk:AltQ – ”Do you mean x or y?” • icm:und*pos:P – ”To Paris.” • icm:acc*neg:Q – ”Sorry, I can’t answer Q” • icm:acc*pos – ”Okay”

11. System feedback for user utterances in GoDIS • contact • negative (”I didn’t hear anything from you.”, ”Hello?”) [icm:con*neg] • perception • negative: fb-phrase (”Pardon?”, ”I didn’t hear what you said”) [icm:per*neg] • positive: repetition (”I heard ’to paris’”) [icm:per*pos:String] • semantic understanding: • negative: fb-phrase (”I don’t understand”) [icm:sem*neg] • positive: reformulation (”Paris.”) [icm:sem*pos:Content] • checking: reformulation (”Paris, is that correct?”, ”Paris?”) [icm:sem*chk:Content]

12. System feedback, cont’d • pragmatic understanding • negative: fb-phrase (”I don’t quite understand”) [icm:und*neg] • positive: reformulation (”To Paris.”) [icm:und*pos:Content] • checking: reformulation (”To Paris, is that correct?”, ”To Paris?”) [icm:und*chk:Content] • acceptance/integration • negative:fb-phrase with reformulation (”Sorry, I cannot answer Q”, ”Sorry, Paris is not a valid destination city.”) [icm:acc*neg:Content] • positive: fb-word (”okay.”) [icm:acc*pos]

13. Feedback selection • Feedback polarity selected depending on (for each level respectively): • Contact: Whether any speech is observed • Perception: Speech recognition score • Semantics: Whether system can find a non-contextual interpretation • Pragmatic understanding: Whether system can find a relevant contextual interpretation • Acceptance: Whether system can accept what’s been said

14. Feedback on multiple levels Negative perception feeback S> Lets see. What can I do for you? U> (mumble mumble) S> Sorry, I didnt hear what you said. What can I do for you? U> siridus S> I heard you say siridus . Sorry, I dont understand. What can I do for you? U> channel one S> channel one. I dont quite understand. Do you want to add a program or change channel? Positive perception Negative understanding Positive understanding (word meaning) Negative understanding (relevance) (Task clarification)

15. Positive understanding Rejection (negative acceptance feedback) U> turn on the t v S> turn on the TV… Sorry, I cannot control the TV set. Do you want to add a program or change channel?” U> pardon S> Sorry, I cannot control the TV set. Do you want to add a program or change channel? U> change channel S> Okay. User negative perception System repetition Positive acceptance

16. Sequencing ICM • Discourse particles • ”so” may be used to reraise a previously discussed question • ”right”, ”anyway”, ”let’s see…” • Verbose sequencing • ”Let’s talk about …”, ”Let’s return to the issue of …” • How do we classify these? • We will correlate them with updates to the common ground

17. Sequencing moves generated by GoDiS • Formal notation schema: • icm:Type{:Content} • Loading a plan • icm:loadplan – ”let’s see.” • Question accommodation • icm:accommodate – ”alright.” • Reraising an issue • icm:reraise – ”so, ” • icm:reraise:Q – ”returning to the issue of …”

18. Sequencing ICM: reraising issue, loading a plan Signals loading a plan S> Welcome to the travel agency! U> price information S> (…) Lets see. How do you want to travel? U> by flight S> (…) What city do you want to go to? U> paris S> (…) What city do you want to go from? U> do I need a visa ? S> (…) Lets see. What country are you from? U> sweden S> Okay. Yes, you need a Visa. S> Returning to the issue of price. Lets see. What city do you want to go from? Reraising an issue explicitly

19. Sequencing ICM: Accommodation S> Lets see. What can I do for you? U> channel five today S> Alright. add a program… Lets see. Okay. channel five… today… What time do you want to start recording? U> from six thirty to seven thirty S> Okay. from six thirty… to seven thirty… The program has been added. ” Signals accommodation

20. Conclusions • Current dialogue systems use only a fraction of the range of ICM used by humans • By extending the range of ICM used by systems, their communication becomes more natural and comprehensive • We have provided an initial classification of feedback and sequencing ICM useful in a dialogue system, and implemented it • (ICM can be correlated with updates to common ground, and used to signal these updates to the user)

21. Future work • Replace current ad-hoc forms based on corpus studies and existing work • ”discourse markers” (Schiffrin) • ”cue phrases” (Grosz & Sidner and others) • ”Real” generation rather than templates • Integrate ICM with other utterances, rather than just appending them • Explore other ways of generating ICM • Intonation, facial expression, etc. • Extend the range of ICM generated (and interpreted) by the system

22. Questions?

23. Object- or metalevel content • Utterances with metalevel content explicitly refer to contact, perception, understanding or acceptance • Object-level utterances instead refer to the task at hand • Example • S: What city are you going to? • U: Paris • S(1a): Did you say you’re going to Paris? [meta] • S(1b): Are you going to Paris? [object] • S(2a): Do you mean Paris, France or Paris, Texas? • S(2b): Do you want to go to Paris, France or Paris, Texas? • This dimension does not apply to all feedback, e.g. ”Paris.”, ”Pardon?” • (Is 2b feedback or simply an alternative question?)

24. Realisation of feedback moves • Syntactic form: • declarative: ”I didn’t hear what you said.”; ”The destination city is Paris.” • interrogative: ”What did you say?”; ”Do you want to go to Paris?” • imperative: ”Please repeat your latest utterance!” • elliptical • interrogative: ”Paris?”, ”To Paris or from Paris?” • declarative: ”To Paris.” • In general, the exact formulation of ICM phrases may depend on various contextual factors • including activity, noise level, time constraints etc.

25. Correcting the system S> Lets see. What can I do for you? U> add program S> Okay. Add a program… S> Lets see. What channel do you want? U> channel six S> OK, channel five. U> no S> not channel five. So, What channel do you want? U> channel six S> Okay. channel six.” (dialog4_edit) Interrogative feedback Negative answer to fb. Signals reraising an issue (Sequencing ICM)

26. Reactions to positive and checking feedback • After positive feedback, system leaves a pause for the user to agree or disagree • if user gives no response, system assumes it was right • After check feedback, positive response is needed • or system will assume it was wrong

27. Implicit feedback? • Clark: ”relevant followup” to U • What is relevant? • simple cases for followups to questions: • answer to question • ”subquestion” • feedback concering question • Complex cases: all other utterances • In general, complex inference and knowledge may be needed (implicatures) • Currently, irrelevant followup counts as negative feedback • What about no followup at all? • in reaction to ask-move or interrogative feedback, counts as negative • in reaction to answer or positive feedback, counts as positive

28. Rejection? S: ”Where do you want to go?” U1: ”Nowhere” U2: ”I don’t know” • Should these count as rejections? • U1: negative answer? presupposition failiure? rejection? • U2: rejection? • but not as definite as ”No comment!”

29. Relation to Traum’s computational theory of grounding • Focus on understanding-level • ”grounding” here refers only to the understanding level • Acceptance and rejection seen as ”core speech acts” • Focus on positive feedback and corrections (self and other) • Based on the TRAINS corpus of H-H dialogue • Deals with the question, when does a contribution end? • Corrections not included here; involves turntaking and OCM • Does not include sequencing ICM

30. GoDiS: an issue-based dialogue system • Explores and implements Issue-based dialogue management (Larsson 2002) • Based on Ginzburg’s notion of a dialogue gameboard involving Questions Under Discussion (QUD) • Uses (mostly pre-scripted) dialogue plans • Extends theory to more flexible dialogue • Multiple tasks, information sharing between tasks • Feedback and grounding • Question accommodation, re-raising, clarification • …

31. Eliciting / nonelciting feedback • (Allwood et. al. 1992) • Eliciting feedback is intended to evoke a response from the user • Noneliciting feedback is not so intended • But may nevertheless recieve a response

32. Simplifying assumptions regarding feedback • We only represent action level and polarity • In polarity, we replace ”neutral” by ”checking” • We exclude feedback which is neutral but not check-questions • Eliciting/noneliciting dimension implicit • Negative feedback is eliciting; since something went wrong, it must be fixed • Checking feedback is also eliciting, since it poses a question that must be adressed • Positive feedback is not eliciting (we assume) • Syntactic form not included

33. Grounding • ”To ground a thing … is to establish it as part of common ground well enough for current purposes.” (Clark) • making sure that the participants are percieving, understanding, and accepting each other’s utterances