Søren Brier & Ole Nedergaard Thomsen

1. Integrative evolutionary Communication – towards a Cybersemiotic foundation of Functional Discourse Grammar Søren Brier & Ole Nedergaard Thomsen

2. Part 2: Towards a Cybersemiotic Discourse Pragmatics Ole Nedergaard Thomsen & Søren Brier, CBS, IKK. one.ikk@cbs.dk

3. Cybersemiotics, setting the scene • Potential challenge: • ”A conception of language and gesture as a single integrated system is sharplydifferent from the notion of a ”bodylanguage” – a communicationalprocessutilizing signals made up of bodymovements, which are regarded … as separate from and beyond normal language.” (McNeill 1992: 11; emphasisadded, ONT)

4. The integrative approach • ”Ratherthancausingus to slice a person analyticallyintosemi-isolatedmodules, • takinggesturesintoaccountencouragesus to seesomethinglike • the entire person as a theoreticalentity– • his thinking, speaking, willing, feeling, and acting, as a unit.” • (McNeill 1992: 11; emphasisadded, ONT)

5. ”The entire person” • theoreticobject of investigation (observation, description, and explanation) of the latefunctionallinguist Simon C. Dik, the ”father” of FunctionalGrammar(FG), the ”godfather” of FunctionalDiscourseGrammar (FDG): • ”The NaturalLanguageUser” (NLU): • Cybersemiotics: ”a linguisticcyborg” • CybersemioticDiscoursePragmatics: ”integral communicator” (+geneticthought-gesture-speech link)

6. Life/LivingSystems/Semiosis Life World/Consciousness/Meaning Existential Development Organic Evolution Embodiment Inner mental world  Language  Consciousness Social Semiotic Practice Physical nature The other & language Big Bang Cosmology History of Culture Matter/Energy/Information Mentality/Language/Knowledge

7. Cybersemiotics, synopsis

8. Threelevels of communication • Signaling– somaticlanguaging: coordination of coordination of behaviorLASC! • Reflexive,stimulus-response; informational • Sign games – psychosomatic • Instinctual, emotional, motivational, volitional • Language games – psychosocial • Intentional, normative, symbolic

9. Signaling: EyeContactMutual gazing ”Whenwe look intoeachother'seyes, I projectmy gaze whilereceivingyours, youprojectyourswhilereceiving mine, and thesefour events occursimultaneously.”

11. Threelevels of autopoiesis • Biological system • the body/organism • Psychologicalsystem • the psyche/mind (motivation, intentionality, self-perception/-value/-interest, conatus: striving for self-preservation of individual and species) • Socio-communicative systems • the person (individualmember of speech community) • the society (collective)

13. Levels of internal semiosis • Body: endosemiosis • Psyche: intrasemiosis (psychosomatic) • Psyche: phenosemiosis • Person/Society: thought semiosis: Slobin: thinking for speaking (psychosocial)

15. Threelevels of eco-semiosis • Body: structuralcouplingsto the invironment (information) • Psyche: instinctualsignification(psychosomatic) • Psyche: conceptualsignification(private, concepts?!) • Person/Society: conceptualsignification (psychosocial: shared, public concepts, in Brier 2008)

16. ECOSEMIOTICS CONCEPTUAL SIGNIFICATION INSTINCTUAL SIGNIFICATION SOCIO-COMMUNICATIVE AUTOPOIETIC LANGUAGE GAMES INFORMATION THROUGH STRUCTURAL COUPLINGS INDIVIDUAL SIGNIFICATION SPHERE ENVIRONMENT

17. Twosignificationspheres • Individualsignificationsphere ( Person) • Informational (somatic/signal: cognitive domain, Maturana/ Umwelt, von Uexküll) • Instinctual (psychosomatic/sign) • Conceptual (psychological: psycho-social/language) • Bio-psycho-social integration • Culturalsignificationsphere ( Society/ Culture) • Result of communicationbetweenindividuals

19. Possible solutions to the challenge • the interpenetrationsbetween the differentautopoietic systems yield an integratedwhole, i.e. a body-psyche-person(bio-psycho-social system): ”an entire person” (S. Dik: NLU) • Searle (e.g. 2009): human societies/institutions are the result of languagegames/linguisticcommunication, the application of communicativecompetences of entire persons

20. Threeinterpenetrating systems … • Person – Logos (o/ratio; polis) • psycho-social:  language games: linguisticcommunication: Speech/Symbolic– Speaker • Psyche – Pathos ( = emotion) • psycho-somatic:  sign games: nonlinguisticanimalcommunication: Gesture/Iconic-indexical– Signer • Body – Ethos ( = behavior) • somatic:  signaling: subconscious signals – Emitter

21. … Three exosemiotic processes as integrated parts of a wholebio-psycho-socialprocess …

22. … yieldingonemacrosystem: … • The threedifferentlevelscouldbeviewed: •  segregationally: i.e. talkingheads vs. communicating ’bodies’ (i.e. minusheads), i.e. language vs. ”bodylanguage” •  integrationally: i.e. total, integral communication: human communication is an integral whole: communicatingbodies (i.e. plusheads): integrative evolutionarycommunication: the wholebody (plus extensions) is the articulator; the body-psyche-personis the integral communicator(responsible/liableactor); the differentsemiotic displays are coexpressive

23. … a total, integral communicativecompetence… • Part of the Body-Psyche-Person(partlydefined as a set of dispositions, abilities, faculties, …), i.e. the integral communicator • I.e. one single process of ”thinking-for-communicating” getsprocessed via the communicativecompetence and is output via differentcommunicationchannels (media)

24. The bio-psycho-social macrosystem

25. Entire person: integral communicator

26. Communicatingbodies/persons

27. Communicatingbodies/persons

28. … Total, integrativecommunication … • Generation of utterances in 2 stages (McNeill 1992): • multidimensional meaning (thinking for/in communicating) is developed in twocomplementarymodes of thoughtof a single integratedprocessof utterance formation whichgetsinstantiated in two stages (modes of representation): • (1) idiosyncraticimagistic-iconicgesture(gesticulation) • (2) conventionalpropositional-symbolic speech (verbalization) • The two stages are synchronized in the gesturalstroke and the focal part of the verbal utterancevia a rhythmic pulse • The gestural stage anticipates the verbal stage in the gesturalpreparation part

29. Generation of utterances in 2 stages • *Sign games: stage 1: spontaneousgestures: iconic(-indexical) • imagistic form of thought;icons • rhematic (qualitative) • idiosyncratic, not coded, non-syntacticallystructuredholophrasticicon • motivation of semiosis: non-idiom • monadic: noduality of patterning, i.e: • minus symbolizationrule (meaning=form, kinesic form: nolevel) • non-standardized, non-distinctive form, minus correctness(”private”) • analogic: holistic, non-linear; non-hierarchical, non-combinatoric • multidimensional (space: visual) • global (wholedeterminesmeanings of parts) • synthetic (onegesturecancombinemanymeanings) • Instantaneous • Indexical: deictics, e.g. pointing (dyadic) • *Language games: stage 2/final stage: speech: symbolic • symbolic form of thought; symbols • propositional-argumental • sociallyregulated, coded, hierarchicallystructuredstring of symbols: convention • arbitrariness of semiosis: idiomatic • triadic: duality of patterning, i.e.: • plus symbolizationrule(meaning => form, phonetic form: separate level) • standardized, distinct forms, structuralcorrectness / grammaticality (”public”) • digital: linear-segmented, hierarchic, combinatoric, recursive(+ SYNTAX!) • unidimensional (time: auditory) • compositionality (meaning of whole is co-determined by parts) • anlytic (distinctmeaningsattached to distinctwords) • temporally extended, successive

30. Dialectic evolution of utterance

31. Semioticunderstanding • Dynamicalobject (same idea unit) • Immediateobject (aspect 1) • Immediateobject (aspect 2) • (1)  rhematicinterpretant: imagery (actional & visuospatial; alsometaphoric) • (2)  dicenticinterpretant: categorialcontent • (1)  iconicrepresentamen: gesture (iconics) • (2)  symbolicrepresentamen: speech

32. Substantialevidence • Gesture-speech is a unitybecause (McNeill 1992: 24): • Gesture and speech developtogether in children (psycholinguistics, ontogenetics) • Gesture and speech break downtogether in aphasia (neurology, pathology)

33. Microgenesis • generative starting point: ”the smallest componentthat has a capacity to grow, • to developintosomethingelse – the final utterance” (McNeill 1992: 218) • ”sense” (context-specificaspects of thought) • dominates over ”meaning” (generalizedcontext-independentaspects) • role of linguisticsigns is to mediateconsciousness

34. The starting point: GP • ”growth point” (GP; McNeill 1992: 219 ff., 2005) • the pragmaticpeak of communicativedynamism: focus (psychologicalpredicate/rheme): point of differentiation of newsworthycontent from a background • semantic pivot: imaginal (iconic) + categorialcontent (symbolic) • the stroke (gesturalpeak) • the intonationalpeak (linguisticpeak) • constructed in advance and held in abeyancewhile the rest of the utterance is built up around it.

35. FunctionalDiscourseGrammar • basedonmodularized information processingpsychology (Levelt 1989); i.e.: • Conceptualizer thought • Formulator speech • Articulator acousticoutput • Mackenzie, J.L., 2000. ”Firstthingsfirst: towards an IncrementalFunctionalGrammar”. Acta LinguisticaHafniensia 32. 23-44.

37. Verbal channel The mind harbors multiple representational systems that can mutually interact. But to formulate any representation linguistically requires its translation into a semantic, “propositional” code

38. FDG – a grammaticalcompetence model • ”FDG starts with the speaker’s intention and thenworksdown to articulation. This is motivated by the assumptionthat a model of grammarwillbe more effective the more itsorganizationresembleslanguageprocessing in the individual. Psycholinguistic studies (e.g. Levelt 1989) clearly show thatlanguageproduction is a top-downprocess, which starts with intentions and endswitharticulation of the actuallinguisticexpression. [Cont’ed]

39. ”The implementation of FDG reflectsthisprocess and is accordinglyorganized in a top-downfashion. Thisdoes not meanthat FDG is a model of the speaker: FDG is a theoryaboutgrammar, but onethattries to reflectpsycholinguisticevidencein itsbasicarchitecture.” (Hengeveld & Mackenzie 2008)

40. Basicarchitecture of FDG

42. Conceptual, contextual and output components Conceptual component is the driving force behind the grammatical component Contextual component is the discourse domain on the basis of which new utterances are produced in the grammatical component Output component generates acoustic, signed, or orthographic expressions on the basis of information provided by the grammatical component

43. Problems with FDG • Model of grammarratherthan model of a NLU, orbetter: of an integral communicator • That is, does not take total, integral communicationintoaccount • Therebybeingpsychologically and pragmaticallyinadequate

44. A remedy Kopp, S.; Bergmann, K; & Wachsmuth, I. Multimodal communication from multimodal thinking – towards an integrated model of speech and gesture production. International Journal of Semantic Computing2008, 2.1, 115-136

45. CybersemioticDiscoursePragmatics • Subscribes to microgenesis (incrementality) • Is not an information processing model. It recognizesfeed back from formulation to conceptualization(= reconceptualization) • But alsoincludescommunicative intentions in the thinking for communicating (McNeillseems to neglectthis part, onlydescribing the propositonal/intentionalcontent) • Thought-language-hand link original in evolution