Jens Allwood SCCIIL, Linguistics, University of Gothenburg

1. Jens AllwoodSCCIIL,Linguistics,University of Gothenburg Berlin, February, 2009 Feedback, coactivation and coconstruction

2. Feedback, coactivation and coconstruction- an analysis of the relation of common ground to dialog (communication) on the basis of Activity based Communication analysis

3. Why interesting? Human Communicative Feedback a place where Top-down and Bottom-up meet • Top-down: anchorable and derivable from theory of Human Communication • Bottom- up: anchorable and derivable from frequent type of data in human communication • Many applications

4. Top - down • Humans are social, rational animals • The primary means for this is Communication (dialog) Communication = def. • sharing of old or new factual, emotive and conative aspects of cognition through • coactivation and coconstruction of content, information or understanding • as a part of and means for joint social activities involving coordination, attunement, collaboration and/or cooperation • Often multimodal and interactive Dialog = def. Interactive communication • Often multimodal and non-competitive and non-conflictual

5. Thus communication involves • Participating in a number of processes that need to be managed so that communicative behaviors can be adapted to reach the criteria of success of these processes • We have therefore evolved several mechanisms for communication management, e.g. turn management, feedback, sequencing and own communication management • The perhaps most important of these is the system for communicative feedback

6. Communicative feedback • Provides mechanisms and criteria for management of the success of Communication = i.e. sharing through coactivation and coconstruction of factual, emotive and conative cognition Through means for eliciting and giving • positive and negative feedback with varying degrees of awareness and intentionality • These enable interactive management and success of communication

7. Positive and negative feedbackwith varying degrees of awareness and intentionality • What is Positive feedback wrt sharing through coactivation (attunement) and coconstruction of cognition (factual, emotive and conative) ? • Body movements and vocal • What is Negative feedback wrt sharing through coactivation (attunement) and coconstruction of cognition (factual, emotive and conative) ? --> Body movements and vcocal • How do we elicit reactions, responses - feedback? --> Using Body movements and vcocal means

8. Interactive view on embodied communication Two communicators form one dynamic system by establishing stabilizing communication links Feedback employs all of these levels, too. Feedback

9. What is embodied com. feedback? Communication, in which two or more embodied communicators are taking turns in contributing information, requires a feedback system to make sure the contributed information is really shared. • Use of vocal and non-vocal channels for feedback (speech, prosody, laughter, facial expression, gaze, head gesture, manual gesture, posture, body position and orientation). • There is continuous and simultaneous sharing of information between speaker and hearer (e.g. Argyle & Cook, 1976). • Feedback can range from conscious, intentional signals to nonconscious, unintentionalindicators of how information processing is going. • Underlying are automatic processes of mimicry, co-activation,synchrony of bodily or vocal action, and emotional contagion(e.g. Wallbott, 1995; Grammer et al., 1996).

10. Communicative Feedback(Allwood, 1976; Allwood, Nivre & Ahlsen, 1993) Every communicative expression can evoke reactions and it can respond to evocative aspects of a previous contribution. Responsive feedback includes short unobtrusive expressions whereby a recipient of information informs the contributor about • Ability & willingness to communicate (have contact) • Ability & willingness to perceive the information • Ability & willingness to understand the information • Ability & willingness to accept the main evocative intention (can statement be believed, question answered, request complied with) • Emotions and attitudes triggered by the perceived information.

11. Dimensions of embodied feedback Every feedback expression can be classified with respect to several dimensions, the most relevant being: • Types of expression or modality used (e.g. Vocal, gestural) • Types of function/content of the expressions • ± Contact (C) • ± Perception (P) • ± Understanding (U) • ± Acceptance of main evocative intention (A) • Emotions/attitudes (E) • Degrees of control and awareness • innately given causal influence, without control or awareness • potentially aware and intentionally controllable • actually aware and intentionally controlled

12. Dimensions of embodied feedback (ctd.) • Types of communicative intentionality(Allwood, 1976) • Indicated information that the sender is not aware of ”showing”, or intending to convey • Displayed information is intended by the sender to be shown • Signaled information is intended by the sender to be recognized by the receiver as being displayed • Types of reception its production is linked to • reactive and automatic vs. responsive and aware • Degrees of continuity • digital and discrete vs. continuous and analogue • Semiotic information carrying relations • indices vs. icons vs. symbols

13. Degrees of awareness, intentionality and conventionalization

14. Laughter • Modality of expression Vocal and gestural • Function/content of the expressions • ± Contact (C) • ± Perception (P) • Emotions/attitudes (E) • -------------------------------------------- • ± Understanding (U) • ± Acceptance of main evocative intention (A) • Both give feedback and elicit reaction • Degrees of control and awareness Mostly innately given causal influence, without control or awareness but is • potentially aware and intentionally controllable or actually aware and intentionally controlled • Mostly indicated, sometimes displayed, seldom signalled • Mostly index. Seldom icon or symbol • Types of reception, mostly reactive, spontaneous, contagious • Degrees of continuity, continuous and analogue rather than digital and discrete

15. When does a communicative sound become a word (symbol)? • Criteria • Similarity of expression and Similarity of function/content between communicators • Conventionalization of link btween expression and content (arbitrary) • Systematic difference between communication-communities Most feedback expressions like mm, ng etc meet these criteria • Inclusion in written language as a written word ( if this were necessary, no language without written languages would have words)

16. Feedback: Degrees of awareness, intentionality and conventionalizationOntogenetic and phylogenetic development

17. Dimensions of feedback in different types of expression

18. Control architecture of a feedback-giving communicator Must combine processes of high-level evaluation (e.g. Allwood et al., 1993) and low-level appraisal (e.g. Scherer, 1999)

19. Example A: always raining in Gothenburg B: (nodding) mhm yeah it is (depressed) The functions of A’s contribution A: 1. ? Responsive: (? B present as pot. communicator) 2. Expressive: opinion (belief) - predication 3. Referential: implicit (meteorological situation) made relevant by predication 4. Evocative: Continue Perceive Understand Share belief Obligations: 1. A considers B cognitively and ethically 2. A desires continuation Commitments: A believes predication

20. B’s reaction/response and obligation • Nonconscious reactions (both internal and behavioral) • Evaluate own willingness/ability concerning evocative functions, i.e. Given my present emotional/volitional/cognitive state/social role Can I/do I want to Continue Perceive Understand Accept evocative function Other reactions 3. React/respond on the basis of the evaluation, i.e. feedback + possible continued coactivation of information B: mhm yeah it is Functions: 1. Reactive/responsive: Contact, Perception, Understanding 2. Expressive: CPU + Depression reformulating opinion/ belief-agreement 3. Referential: same as in A’s contrib. 4. Evocative: CPU (share belief)

21. Shared information/content understanding (common ground?) Dialog creates/activates • Shared contact (automatic reactions) • Shared perception • Shared understood content/information (including emotions) • Accepted evocative fuctions (agreements , commitments to joint action) • Continued relevant coconstruction Drawing on a shared context/background of • Shared perceptual environment (com & noncom) • Jointly engaged in activity (com & other instr action) • Other information activated by communication and other instrumental action (iv) Other common traits (physical, biological, social, psychological, e.g. cultural Representing a jointly activated and constructed content containing • multimodal information - often using short 1-word utterances and gestures • other instrumental action + feedback concerning willingness/ability to perceive, understand, accept

22. How do we and/or the communicators know if information is shared (is common ground)? • Coherence of discourse i.e. • That evocative functions have a fit with responsive functions (local relevance), • esp. relevant fedback (indicated, displayed, signalled), • referential coherence, • continued relevant coconstruction • Global relevance with joint purpose of social activity • Relevance to shared interests • Lack of negative feedback • Requests for clarification handled

23. The build up of common ground is also influenced by the fact that Communication involves Cooperation, Ethics & Trust • Communication Sharing through coactivation and coconstruction of information, content or understanding mediated through • mutual causal influence between organisms(mechanisms) engaging in •(motivated, intentional, rational) action and interaction

24. The cooperative features of communication mean that • Common ground of coactivated and coconstructed old and new iformation can be more easily established • Coordination, collaboration and cooperation facilitate reaching different levels of common ground

25. ButCommunication (dialog) can be • Unethical • Distrustful And Unsuccessful communication (dialog) Can be • uncoordinated and • lack collaboration on understanding and providing of information

26. Unsuccessful communication • Even in these cases some types of common ground can be established And is sometimes presupposed • Subconcious CG • shared perception CG • -shared understanding CG • Conflict and disagreement can sometimes involve lack of coordination, perception and understanding but does not always do so. • Some types of conflict and disagreement presuppose understanding to be effective

27. Bottom -upEmpirical data on feedback • Data on • Indicative FB behavior automatic processes of mimicry, co-activation,synchrony of bodily or vocal action, and emotional contagion • Displayed FB behavior Displayed processes of mimicry, imitation,coordination of bodily or vocal action, and emotional and conative attunement • Signalled-symbolic FB behavior e.g gestural and verbal, see below Collaborative and cooperative behavior

28. Indicative and/or Displayed FB30 interacting pairs of students, systematically varied with respect to sex and mutual acquaintance.Task to find out as much as possible about each other within 3 min. Self-reported rapport L (00:00:13): Bist du im ersten Semester $ R (00:00:15): Ich bin eigentlich im fünften Semester aber die ersten zwei hab ich nicht wirklich was gemacht und dann // $ L (00:00:18): aha $ R (00:00:20): jetzt bin ich im dritten $ L (00:00:23): Zoologie oder Botaniker oder was $ R (00:00:26): entweder Anthro oder Zoologie das weiss ich noch nicht so genau $ L (00:00:28): aha die Anthropologen sind viel besser $ L (00:00:33): mhm /// hast du schon den Seidler gemacht $ R (00:00:35): ja $ ….

29. Some empirical data on signalled, symbolic Feedback • Most common complete utterances in Swedish and German

30. German Swedish 23 mhhm men 24 nee nähä 25 ach so precis 26 nich så 27 jawohl m ja 28 nein nein öh 29 na ja ja tack 30 ja bitte javisst 31 ich jaja 32 ja eben just det 33 okay ja okej 34 mhm jaja m 35 wiedersehn eh 36 also ja ja ja 37 natürlich varsågod 38 richtig tack så mycket 39 danke ja det är det 40 hm hm hm visst 41 mh ja det 42 ja hm ja men 43 das det är det 44 sicher vad 45 ja sichervadå 46 ähä det 47 guten tag m m m 48 ja mhm tack ska du ha 49 gell oj 50 mmbra 51 ja genauja ha 52 doch hej då 53 ja aber nä just det 54 wiederhören tjugo 55 oh nej nej 56 hm ja ah 57 tjaoch 58 so två 59 eh äh 60 ach tre 61 ach jaja absolut 62 vier vier eins sieben sieben sieben japp

31. German Swedish 63 ja gut sju 64 dankeschön tjugofem 65 danke schön nä nä nä 66 ja klar näeh 67 wiedersehen ja ja visst 68 auf wiederhören hm hm 69 ah so fem 70 genau ja fyra 71 dann ja då 72 oder åtta 73 ja undyes 74 m det tror du 75 bitte schön tack hej 76 was hej hej 77 hier hallå 78 herr ja det tror jag 79 ja das jaha ja 80 ja natürlichjag vet inte 81 gut ja det tycker du 82 ja ich ett 83 warum jag 84 ähm gula sidornas informationsservice god middag 85 ah tolv 86 bitte sehr det är 87 hm hm hm hm ja det tycker jag 88 aha janä men 89 vielleicht eller 90 das is sex 91 schön m hm 92 moment nio 93 selbstverständlich femton 94 vielen dank ja jo 95 nee neeja ja just det 96 bitteschön tio 97 jo tjugotre 98 hallo du 99 so ist es arton 100 mhmh sexton

32. 100 most common utterances

35. Parts of speech and one word utterances

36. Swedish Feedback reduplications

37. Some examples of reduplication

39. Concluding remarks This talk has agued that communicative feedback both from a top-down and bottom-up perspective is an essential feature og human communication • Feedback is a means for managing the sharing of information throughcoactivation, and coconstruction in activity based interactions which is characteristic of all human communication • Feedback occurs with several degrees of awareness, intentionality and conventionalization; from indicative behavior to signalled symbols