Cognitive semiotics and neurodynamics. RT: Cognitive semiotics, Cognitive science 15th of September 2006

1. Cognitive semiotics and neurodynamics.RT: Cognitive semiotics, Cognitive science15th of September 2006 Wolfgang Wildgen Early Fall School in Semiotics 11th to 18th of September 2006 Sozopol, Bulgaria

2. Dynamic vs. cognitive semiotics • Dynamic semiotics starts from biological morphogenesis and expands this thought to linguistics. In his holistic strategy René Thom (1923-2002) preferred a Gestaltist view on biology and not so much a mechanistic one, which takes the brain as the central (and finally) only “organ” responsible for thought, language and culture. This view is further developed in Wildgen (1994) introducing modes of cognitve self-organization in style of synergetics (cf. Haken and Stadler, 1991). • The research line of cognitive semiotics was first developed in the philosophical atmosphere of logical empiricism (Quine) and formal syntax (Carnap). Whereas Chomsky created a compact mentalistic theory, Lakoff (since 1975) and with him Langacker and Talmy combined insights of gestalt-psychology with ideas stemming from issues of generative semantics. Cognitive semiotics have extended this methodology (cf. Brandt, 2002). Wolfgang Wildgen

3. Ways of a neurodynamic expansion of theses models • The current advances in neurodynamics due to the techniques of neuro-imaging and research in the neurodynamics of animals (mice, rats etc.) during perceptual and motor tasks give us a new understanding of what the brain does, when a subject (animal or human) has to solve specific problems. • Dynamic semiotics (in Thom‘s tradition) must consider the neural “machinery” and specify the dynamics in the context of the brain. • Cognitive semiotics must leave the theoretical isolation of structuralism, its primary concern with grammar and go beyond psychological analogies towards a foundation in the dynamics of the brain. Wolfgang Wildgen

4. The representation of dynamics in cognitive grammar (Langacker) Langacker's analysis of the verb ENTER. Wolfgang Wildgen

5.  Descriptionof a process in cognitive semantics (Talmy) The ball kept rolling because of the wind blowing on it. • intrinsic force tendency of the Agonist (right): towards rest (●), • the Antagonist (left) is stronger (), • intrinsic force tendency of the Antagonist: action (), • result of the force interaction: action ( ). Schematisation of force-dynamics by Talmy (1988). Wolfgang Wildgen

6. The representation of dynamics in dynamic semantics Derivation of a process-schema (right) from a path p in the vector-field of the cusp (A3): capture; A catches B. Wolfgang Wildgen

7. Are neural representations a map of external dynamics? • Both the quasi-imaginistic and the dynamic model of meaning assume that a kind of abstract “pictures” of external events exist in the mind. Although neural representations must have an enactive capacity and be able to govern action in the external word (and social coordination in such actions), they have a very specific format and rather independent dynamics. • At the local level, where specific observation and experimentation in a neuro-cognitive experiment can be achieved, one can not assume the appearance of such global mappings. This ma only appear at a higher level, where learned schemata are activated top-dawn. They must , however, be able to use the more basic binding dynamics explained in the forgoing sections. Wolfgang Wildgen

8. Shortcomings of both cognitive and dynamic semiotics • If images filling Langacker’s articles represent only vague and fluctuating after-effects observed in the analysis of linguistic structures, on may just dispense of them and return to classical models. • All concepts taken from gestalt psychology have since undergone a redefinition in terms of neurodynamics. Why stick to the state of the art fifty years ago? • The concepts taken from morphogenesis by Thom (cf. his discussion with Waddingston) have also been redefined using more specific higher order knowledge on neurochemical processes. • The neurodynamics of perception, motor programs and memory are nearer to language than basic morphogenetic processes. Wolfgang Wildgen

9. The dynamics of composition (Frege’s principle) • In a simple case which avoids the complexity of verbal valence, and rather takes nominal syntax as a basic example, one may consider a noun related to the form of an object, say a “square”, an adjective of colour, say “red”, and a present participle of motion, say “moving”: • red moving square • How does the brain compose a head-noun referring to form with two satellites referring to colour and motion? Wolfgang Wildgen

10. Some elements of a model of neurolinguistic binding • For Andreas Engel (2004) the binding process is one of temporal synchronization of neural assemblies, which form wholes (gestalts) from parts and one of desynchronization which distinguishes figure and ground. • This type of analysis concerns only the composition in perception, attentiveness and memory, but one may conjecture a parallel process for words (related to perceptual information) and their composition in syntactic constructions involving nouns and adjectives. Wolfgang Wildgen

11. The basic idea of temporal binding Parts or features of a visual whole are linked by the synchronic firing of a set of neurons (an assembly) during a short time interval. In the example the parts and features of the cat and those of the woman are bound together by the internal synchrony of the assemblies 1 and 2 and they are distinguished by the asynchrony of these assemblies. From: Engel et alii, 1997: 572 Wolfgang Wildgen

12. Ambiguity and Binding Picture a is ambiguous. If it is seen as one face (and a candle in front of it) the zones (1,2) and (3,4) (see series d) are bound; in the case two faces looking at each other are seen, the zones (1,3) und (2,4) are bound. The binding may be recognized by the synchronic firing rates in the series d versus e. From: Engel, Fries und Singer, 2001: 707 Wolfgang Wildgen

13. Top down effects due to expectation and memory The remembered object produces higher synchronization at the γ-level (30 to 60 Hz) At the left a Kanitza-triangle At the right a non-Kanitza-triangle If the tested person is instructed to recognize the non-Kanitza-triangle , the synchronization is higher for this configuration, although basic gestalt laws would predict the contrary. From: Hermann, Munk und Engel, 2004:349 Wolfgang Wildgen

14. Two different pathways starting from the visual center V1 spezialize on the analysis of: Where was it seen? What was seen? “ ‘What’ and ‘Where’ are the basic questions to be answered in visual perception. Not only must we recognize what we are looking at, but also we need to know where it is in order to respond appropriately.” Gazzaniga u.a. (1998: 165) Wolfgang Wildgen

15. Restrictions on compositionality due to temporal binding Teisman (1999: 108) writes: • „It [the binding-by-synchrony hypothesis] also provides a plausible reason for the attentional limit of around four objects that is widely observed in the perception of brief displays and in studies of visual working memory. The different firing rates that can be easily discriminated on a background of inherent noise and accidental synchronies may set a low limit to the number of objects that can be simultaneously bound.” • The restrictions of valence patterns and of embeddings (recursive operations) have been discredited by Chomsky as performance effects. To the contrary, they are hints to the nature of the compositional process in language and thus more interesting than the algebraic notion of recursive operations. Wolfgang Wildgen

16. Some conclusions • Cognitive semiotics must try to have a neurodynamic foundation of such basic concepts as the composition of meanings to a whole (Frege’s problem) • Such a model should consider the topology of the brain, synchronization and desynchronization, and the coupling of subnetworks with self-organization (filtering, choice of dominant modes), self-reference and monitoring in consciousness, etc. • As cognitive semiotics goes beyond cognitive semantics and grammar, it must also consider the composition of meaningfull entities in in the visual (art, media, architecture) and in the domain of audition (music) and motion (dance) Wolfgang Wildgen

17. Some bibliographical indications • Engel AK, Fries P, Singer W (2001) Dynamic predictions: oscillations and synchrony in top-down processing. Nature Reviews Neuroscience 2: 704-716 • Engel AK, Roelfsema PR, Fries P, Brecht M, Singer W (1997) Role of the temporal domain for response selection and perceptual binding. Cerebral Cortex 7: 571-582 • Herrmann CS, Munk MHJ, Engel AK (2004) Cognitive functions of gamma-band activity: memory match and utilization. Trends in Cognitive Sciences 8: 347-355 • Gazzaniga, Michael S., Richard B. Ivry und George R. Mangun, 1998. Cognitive Neuroscience. The Biology of the Mind, Norton, New York. • Machery, Edouard, Markus Werning, Gerhard Schurz (Hg.), 2005. The Compositionality of Meaning, Bd. II: Applications to Linguistics, Psychology and Neuroscience, Ontos, Frankfurt/Main. • Riesenhuber, Maximilian und Tonaso Poggio, 1999. Are Cortical Models Really Bound by the “Binding Problem”?, in: Neuron, 24: 87-93. • Singer, Wolf, 1999. Neuronal Synchrony: A Versatile Code for the Definition of Relations?, in: Neuron 24: 49-65. • Teisman, Anne, 1999. Solutions to the Binding Problem. Progress Through Controversy and Convergence, in: Neuron, 1999: 105-110. • Werning, Markus, Eduard Machery, Gerhard Schurz (Hg.), 2005. The Compositionality of Meaning and Content, Bd. I: Foundational Issues, Ontos, Frankfurt/Main. • Wildgen, Wolfgang, 1994a. Process, Image, and Meaning. A Realistic Model of the Meanings of Sentences and Narrative Texts, Benjamins, Amsterdam. Wolfgang Wildgen

18. My home page is:http://www.fb10.uni-bremen.de/homepages/wildgen.htm You may find my presentation in Sozopol (when I have returned) together with other conference papers of the last two years in a list at the end of my homepage.