1.15k likes | 1.62k Views
Alignment in human-human and human-computer interactions. Holly Branigan Jamie Pearson University of Edinburgh. Collaborators. University of Edinburgh Martin Pickering Stanford University Clifford Nass John Hu. Outline. Our approach Features of Human-Human interaction (HHI)
E N D
Alignment in human-human and human-computer interactions Holly Branigan Jamie Pearson University of Edinburgh
Collaborators University of Edinburgh • Martin Pickering Stanford University • Clifford Nass • John Hu
Outline • Our approach • Features of Human-Human interaction (HHI) • Audience design • Alignment • Patterns of alignment • Explanations for alignment • Relevance to HCI • Methodology for studying HCI • Data from experiments investigating lexical/syntactic alignment in HCI
Our approach • Psycholinguistic accounts of HHI: • Characterising linguistic behaviour • Identifying underlying psychological mechanisms • Focus on lexical and syntactic processing • Applying to HCI: • Predicting human (user) behaviour • Simulating human behaviour • Modifying human behaviour
Features of HHI • Linguistic behaviour in HHI is highly flexible and adaptive: • An individual speaker’s behaviour is partially contextually-determined. • Linguistic choices (semantic, lexical, syntactic, phonological..) choices may vary according to context. • Relevance to HCI: • Features of HCI communicative context may influence linguistic behaviour.
Determinants of Behaviour • Influence of addressee on speaker: • Indirectly – speaker’s beliefs about addressee’s state of knowledge, interests etc • Audience Design • Directly – via addressee’s own linguistic behaviour • Alignment • Both influences may be active simultaneously
Audience Design • Tailoring utterance to fit addressee: • Their knowledge, interests, beliefs etc • May be based on speaker’s a priori beliefs or assumptions about addressee • Or may be based on direct evidence from addressee • E.g., addressee’s feedback
Audience Design – lexical choice • Fussell & Krauss (1992): • speakers adjust their choice of lexical terms to fit the assumed knowledge of their addressees: • Proper names when addressee likely to know referent: Clint Eastwood • More detailed descriptions when addressee less likely to know referent: Media mogul, was married to Jane Fonda, owns CNN, has a moustache and grey hair • Assessment of addressee knowledge is based on a priori judgements about social distribution of knowledge • movie stars > industrialists
Audience Design – lexical choice • Isaacs & Clark (1987): • A priori assessment of addressee knowledge can be rapidly adjusted on basis of direct evidence from feedback • Lexical choices then reflect this adjusted assessment • ‘experts’ in a domain (New York landmarks) quickly adjust their choice of lexical terms to fit the apparent expertise of their addressees: • Proper names when addressee is apparently fellow expert Chrysler building • More detailed descriptions when addressee is apparently non-expert Building with a tall pointy roof and a spike on top
Audience Design - syntax • Audience design may influence syntactic choices: • Speakers may choose syntactic structures that are most easily understood by addressees. • Ambiguity-avoidance: Put the penguin in the cup on the star vs Put the penguin that’s in the cup on the star When two penguins (and an empty cup), temporary ambiguity at penguin in the cup…
Audience Design - syntax • Audience design may influence syntactic choices: • Speakers may choose syntactic structures that are most easily understood by addressees. • Ambiguity-avoidance: Put the penguin in the cup on the star vs Put the penguin that’s in the cup on the star When two penguins (and an empty cup), temporary ambiguity at penguin in the cup… Speakers include that’s more often (removing ambiguity) when there are two penguins than where is only one penguin. Haywood, Pickering & Branigan (2005)
Audience Design - summary • Speakers in HHI may engage in strategic planning of aspects of utterances: • Calculations of addressee knowledge, beliefs etc • Based on a priori assumptions or direct evidence • A priori assumptions are quickly updated in light of direct evidence; audience design reflects this • Strategic planning may affect lexical choice, syntax, semantic choices • E.g., choice of reference frame (Schober, 1993)
Alignment • Speakers are affected directly by addressee’s linguistic behaviour: • Tendency for speakers in a dialogue to converge linguistic behaviour. • After hearing an interlocutor use particular linguistic behaviour, speakers tend to repeat that behaviour. • This tendency towards alignment is robust and highly pervasive. • Rhetorical structure, semantic structure, syntactic structure, lexical choice, accent, speech rate, ….
Alignment • Alignment can be implicit: • Almost always arises without explicit negotiation • Speakers often align on different expression than negotiated expression • Speakers usually unaware of effects • Can sometimes (but often cannot) report awareness of meaning-related alignment • Usually unaware of alignment of form
Alignment – semantic choices • Alignment occurs for aspects of language associated with meaning: • e.g. reference frames (Watson, Pickering & Branigan, 2005) The dot is left of the cameravs The dot is below the camera
Alignment – semantic choices • Alignment occurs for aspects of language associated with meaning: • e.g. reference frames (Watson, Pickering & Branigan, 2005) The dot is left of the cameravs The dot is below the camera • e.g., description schemas(Garrod & Anderson, 1987) I’m at B5 vsI’m two along and two up
Alignment – lexical choice • Also alignment of lexical choice: • Use same words in same ways • boxvs node • square = single nodevsconfiguration of nodes
Alignment – lexical choice • May align on unusual/rare lexical choice: • e.g., if first person to name object is a non-native speaker
Alignment – lexical choice • May align on unusual/rare lexical choice: • e.g., if first person to name object is a non-native speaker Rocking chair
Alignment – lexical choice • May align on unusual/rare lexical choice: • e.g., if first person to name object is a non-native speaker Rocking chair The chair that can go back and forth
Alignment – lexical choice • May align on unusual/rare lexical choice: • e.g., if first person to name object is a non-native speaker Rocking chair The chair that can go back and forth You can shake your body Bortfeld & Brennan, 1997
Alignment • Alignment at semantic and lexical levels may be linked to different types of meaning: • Alignment of perspectives on a situation (ways of thinking about the world) • E.g., rainbow troutvscoloured fish • But other alignment seems to be unrelated to convergence on types of meaning • e.g., speech rate • e.g, syntax (alternatives express same meaning)
Alignment – syntactic choices The artist is selling the dancer the gun(Double Object) vs The artist is selling the gun to the dancer (Prepositional Object)
Alignment – syntactic choices The artist is selling the dancer the gun(Double Object) vs The artist is selling the gun to the dancer (Prepositional Object) • A: The chef handing the book to the teacher • B: The artist selling the gun to the dancer
Branigan et al. (2000) • Picture-description/-matching game • Stooge and naïve subject alternately describe pictures • Stooge produces scripted descriptions (50% PO, 50% DO) • Stooge describes source picture to subject: • The nunshowing the monk the banana • Subject chooses matching picture: • Yup
Branigan et al. (2000) • Picture-description/-matching game • Stooge and naïve subject alternately describe pictures • Stooge produces scripted descriptions (50% PO, 50% DO) • Stooge describes source picture to subject: • The nunshowing the monk the banana • Subject chooses matching picture: • Yup • Subject describes target picture to matcher: • The artistselling the dancer the gun • Matcher chooses matching picture: • Uh huh
Branigan et al. (2000) • Picture-description/-matching game • Stooge and naïve subject alternately describe pictures • Stooge produces scripted descriptions (50% PO, 50% DO) • Stooge describes source picture to subject: • The nunshowing the monk the banana • Subject chooses matching picture: • Yup • Subject describes target picture to matcher: • The artistselling the dancer the gun • Matcher chooses matching picture: • Uh huh
Confederate-scripting paradigm Naïve participant Confederate Blah Blah Blah Confederate’s script Picture cards to be matched Branigan et al (2000)
Alignment - syntax • Strong tendency to align syntax with interlocutor: • More likely to produce a Prep Obj description after hearing a Prep Obj description than after a Double Obj description, and vice versa. • 77% aligned descriptions when verb repeated • 63% aligned descriptions when verb not repeated • Chance = 50% • NB: speakers align sentence form: both alternatives have same denotational meaning • (cf. semantic/lexical alignment) • Speakers align dynamically (produce both PO and DO)
Alignment - syntax • Similar effects found: • for other structures: • E.g., NP structure: a red catvs a cat that’s red (Cleland & Pickering, 2003) • NB: Relative clause = strongly dispreferred in null context • in multi-party dialogues: • Speakers syntactically align, whether directly addressed or not. A: The waitress is giving the monk the banana C:Yup
Alignment - syntax • Similar effects found: • for other structures: • E.g., NP structure: a red catvs a cat that’s red (Cleland & Pickering, 2003) • NB: Relative clause = strongly dispreferred in null context • in multi-party dialogues: • Speakers syntactically align, whether directly addressed or not. A: The waitress is giving the monk the banana C:Yup B:The teacher is handing the cowboy the jug C:Uh huh Branigan et al. (in press)
Alignment - syntax • Similar effects found: • in different communicative contexts: • e.g., during walkie-talkie communication • in ‘special’ populations such as bilinguals, L2 learners, children…
Alignment - syntax • Similar effects found: • in different communicative contexts: • e.g., during walkie-talkie communication • in ‘special’ populations such as bilinguals, L2 learners, children… El cerdo llora El pingüino baila (SV)
Alignment - syntax • Similar effects found: • in different communicative contexts: • e.g., during walkie-talkie communication • in ‘special’ populations such as bilinguals, L2 learners, children… El cerdo llora El pingüino baila (SV) Llora el cerdoBaila el pingüino (VS) Flett, Branigan & Pickering, in prep
Alignment - summary • Alignment with addressee’s linguistic behaviour: • is robust • can be very strong determinant of behaviour • occurs for many (all?) levels of linguistic structure • occurs for aspects of structure concerned with form, not just meaning
Alignment and audience design • Alignment and audience design may co-occur. Put the penguin in the cup…vs the penguinthat’s in the cup…
Alignment and audience design • Alignment and audience design may co-occur. Put the penguin in the cup…vs that’s in the cup… • Priming: • tendency to say The penguin that’s in the cupafter hearing The sheep that’s on the plate • Audience design: • tendency to say The penguin that’s in the cupwhen there are two penguins Haywood, Pickering & Branigan, 2005
Explanations of alignment • Social psychological approaches to alignment: • Identification with a particular social group (inc. addressee) • Reciprocity effects (linked to politeness norms?) • Such approaches explain why alignment of linguistic form occurs (in absence of meaning differences) • Perceived social identity of addressee is important • e.g., politeness is only relevant for addressees that are perceived as social agents
Explanations of alignment • Alignment as audience design: • Alignment may reflect strategic effects (choosing to adopt other person’s perspective to enhance communication) • Choosing e.g., the same description schema or referential expression maximises the chances of effective communication • Such approaches do not really explain why alignment of linguistic form occurs (in absence of meaning differences) • Perceived identity of addressee is important • e.g., depends on addressee being intentional agent • inanimate entities do not have perspectives or beliefs
Explanations of alignment • Alignment as an automatic behaviour: • Primitive, default behaviour • children: • align significantly more than adults (e.g., NP structure: 75% vs 30%) • tend to align form, even when this leads to misunderstanding: • e.g., using same term with different reference (e.g., square) • must learn to suppress tendency towards alignment where appropriate (Garrod & Clark, 1994).
Explanations of alignment • Alignment may be based on automatic priming mechanisms: • activation-based account (Branigan et al., 2000) • comprehension of word or structure activates associated linguistic representations • VP V NP PP • representations retain residual activation • residual activation facilitates subsequent selection
Explanations of alignment • Alignment may be fundamental to efficient communication (Pickering & Garrod, 2004) • Efficient communication arises when interlocutors come to have the same understanding of relevant aspects of the world. • This arises from alignment of their situation models(e.g., Zwaan & Radvansky, 1998). • Alignment of situation models arises from alignment of other aspects of language (e.g., syntax, lexical choice). • Alignment at one level promotes alignment at others Lexical alignment syntactic alignment semantic alignment
Explanations of alignment • Explanations are not mutually exclusive: multiple factors may underline alignment. • Most likely that at least some implicit element • Participants generally report lack of awareness of alignment • But other factors may also contribute to overall effect: • Basic (automatic) alignment effect may be enhanced by e.g. social factors (e.g., social status) • Alignment at some levels of structure may be differentially susceptible to additional influences
Implications for HCI • Does alignment occur in HCI? • Pickering & Garrod (2004): • alignment is fundamental to effective communication • So evidence that alignment is absent in HCI would imply that HCI is necessarily ineffective/inefficient. • May highlight inherent difficulties in HCI • Conversely, evidence that alignment is present in HCI suggests possibility of effective communication in HCI
Alignment in HCI? • Why might we expect alignment in HCI? • Alignment seems to be a default linguistic behaviour: • May be consequence of architecture of human language processor • Might therefore expect to find it in any communicative context • Existing evidence that people treat computers as social actors Reeves & Nass (1996) • They interact with them and evaluate them as they do other people • People should align with computers
Leveraging alignment in HCI • Alignment may be useful behaviour: • Facilitates predictions of user behaviour • Reduces decision space in e.g., speech recognition • Allows simulation of human behaviour • Users may feel more comfortable with systems that display human-like behaviour • Users might feel more positively towards systems that align than those that do not • May allow [implicit] modification of human behaviour • Users may modify their linguistic behaviour to align with system • May allow modelling of desired input
Patterns of alignment in HCI • How might alignment in HCI pattern? • If alignment is purely automatic priming: • Alignment would occur whenever a linguistic structure is encountered • But in HHI, speakers’ (lexical and syntactic) choices are also influenced by: • a priori beliefs about addressee knowledge, capability etc • direct evidence about addressee knowledge, capability etc • Hence beliefs about the knowledge, capability etc of a system might influence the extent to which speakers align with it.
Relevant factors in HCI alignment • A priori beliefs about social identity of system: • If systems are treated as social agents just like humans, then alignment should occur just as with humans. • But which group of humans would count as comparable social agents? • Limited capability? • If systems are not treated as social agents just like to humans, then alignment may differ: • Less alignment if alignment has substantial component related to e.g. politeness
Relevant factors in HCI alignment • Beliefs about system capability: • Users may assume computers to be less capable (generally and/or specifically linguistically). • May increase likelihood of aligning, relative to HHI • Possibly different degrees of alignment with different systems • May increase likelihood of aligning with unusual structure/expression • Effects of feedback: • A priori beliefs may be updated in light of feedback • Positive feedback of understanding may lead to comparable alignment as in HHI