Three ways to bridge the gap between perception and action, and language

1. Three ways to bridge the gap between perception and action, and language Jean-Luc PETIT Université de Strasbourg& Laboratoire de Physiologie de la Perception et de l’Action (Collège de France) jean-luc.petit@college-de-france.frwww.jlpetit.com

2. Plan of the talk • The ‘embodiment of language’ : A spectrum of possible meanings • From modularity to interaction • The motor system not a mere output • Binding doesn’t need abstract supramodal computation • From Mentalese to ‘the language of neurons’ • Broca’s area: an all-purpose processor of structural complexity • That neuroscience can not do without a phenomenology of language: • Let’s not replace questions of essence with evolutionary narratives • Piecemeal solutions only conceal the full extent of the dilemma • The challenge: to anchor in the body verbal (or preverbal) meaning morphologies • The phenomenological tradition and its deceptive proxy: • Liberman, Gibson, Goldman, et alii • Merleau-Ponty, the one epistemically acceptable phenomenologist • Husserl, the founder and transcendantal scarecrow • Three ways to bridge the gap between perception and action, and language: • Tracing the flow of neural events in brain circuits of the speaker-hearer • Descriptive eidetic phenomenology and its geometric modelling • Transcendental constitution of a verbally articulate Lebenswelt • Conclusion: A plea for a threefold approach to the embodiment of language

3. The ‘embodiment of language’ : • A spectrum of possible meanings • From modularity to interaction • The motor system not a mere output • Binding doesn’t need abstract supramodal computation • From Mentalese to ‘the language of neurons’ • Broca’s area: an all-purpose processor of structural complexity

4. The ‘embodiment of language’: A spectrum of possible meanings • The expression ‘embodiment of language’ refers to a new trend of research on the neural basis of language. • Apart from a convenient label to bring together different research teams, this expression suggests that there is a special relationship between language and the body. • The philosophical observer may wonder whether he will find in it the traditional philosophical problem of the incarnation of mind. • But, as it would be foolish to project interpretations of a philosopher on empirical science, we should first get familiar with the use of ‘embodiment’ by the researchers themselves. • If we consider all the contributions to the investigation of the brain bases of language without omitting any of the protagonists in the ongoing controversy, we will find a spectrum of, at first sight not very homogeneous, uses : • Are these different uses of the term ‘embodiment’ mutually compatible, and can they be reduced to one unequivocal sense?

5. From modularity to interaction • After the provocative revival of Gall’s Phrenology by Fodor (1983), it looked as though major cognitive functions, such as object or sentence recognition, could be fully carried out by specialized peripheral systems that operate independently of each other without exchanging information, so that the organism, in spite of its anatomic unity, is like a hydra at the cognitive point of view. • To the extent that new research on the foundations of language suggests that its functions are not underpinned by specialized modules but rather by an extensive network of distinct areas of the brain that sustain a permanent dialogue with each other, such research tends assuredly to some form of embodiment: the recovery of the integrative unity of the organism on its fragmentation into multiple modules. • Cf. Fadiga, Gallese, Pulvermüller, Rizzolatti ...

6. The motor system not a mere output • The classic model of the bases of language (Lichtheim 1885, Geschwind 1965) strictly subordinates the production of speech sounds by articulators to the cognitive processing of linguistic information. • This model limits the contribution of the motor system to the role of slavishly executing a motor program developed elsewhere in auditory areas and in the upper levels of the hierarchy of the cognitive system. • This representation of the muscular production of speech rests on the traditional prejudice concerning the body as an instrument of thought. • This prejudice is shaken by the discovery of retroactive influences, sometimes modulatory and sometimes formative, performed by the articulator system on auditory reception and semantic interpretation of phonemes and expressions. • This rehabilitation of the cognitive function of movement in speech amounts to an embodiment of language.

7. Binding doesn’t need abstract supramodal computation • The synthesis of sensory qualities of perceived objects raises, at the level of the neuron or neuronal group, the problem of binding of unimodal signals of different pathways: visual, auditory, olfactory, vestibular, proprioceptive and visceral in a supramodal concept of the object. • This integration function is classically delegated to a central cognitive system hierarchically superior to the various sensory systems and exerting an influence on their operation, especially through the orientation of attention. • This hypothesis of a purely conceptual, amodal thus disembodied, center is unlikely given the ubiquity of the mixture of influences of the various modalities that occurs at synaptic connections. • The introduction of a transversal process of integration between modalities would allow us to dispense with this hypothesis, assuming that the motor system is able to preform perception through action.

8. From Mentalese to ‘the language of neurons’ • In the history of cognitive science Chomskyan idea of competence, with its strict distinction as to performance and its priority over the latter for the study of language has had a founding role. • This distinction, and hierarchy resulting from it, tended to assimilate the core structure of the linguistic capability of man to a language of symbolic logic and its implementation to an application of syntactic rules to strings of symbols (a calculus). • Once the language of thought has been internalized in this deep structure its realization in acts of communication could only appear as a contingent coating surface structure. • The Mentalese (Fodor 1975) limited the contribution of neuroscience to the study of language to the realization of the logical structure of competence in a brain-machine indifferent to its program. • The identification of linguistic information processing with neural dynamics itself and its laws of association is yet another form of embodiment of language (Pulvermüller 2002).

9. Broca’s area: an all-purpose processor of complexity • Whether converging or diverging, the various trends expressing themselves through the theme of embodiment are represented in the debate on the interpretation of the functions of Broca's area. • Traditionally regarded as a center for motor realization of speech at the end of cognitive processing, the contribution of Broca's area was found to take place earlier and to be more complex, since it is recruited at all levels of verbal conduct: for perception as well as for production, for syntactic construction as well as for semantic interpretation. • This redefinition of the linguistic functions of Broca's area coupled with a phylogenetic hypothesis about its origins in a monkey's premotor area site of mirror neurons crosses the issue of embodiment: • Should we assign this key node in the brain circuits of language to the motor system ? Or should we not rather focus on the emancipation of this area from its former utilitarian functions and the acquisition by it of a capacity to process any multimodal cognitive complexity?

10. That neuroscience can not do without a phenomenology of language: • Let’s not replace questions of essence with evolutionary narratives • Piecemeal solutions only conceal the full extent of the dilemma • The challenge: to anchor in the body verbal (or preverbal) meaning morphologies

11. Merleau-Ponty: « On ne comprendra jamais ces deux idées à la fois si l'on continue d'osciller entre la notion de "motricité" et celle "d'intelligence", et si l'on ne découvre pas une troisième notion qui permette de les intégrer, une fonction, la même à tous les niveaux, qui soit à l'œuvre aussi bien dans les préparations cachées de la parole que dans les phénomènes articulaires, qui porte tout l'édifice du langage, et qui cependant se stabilise en processus relativement autonomes(Phénoménologie de la Perception, 228). » • Do recent work on the cerebral bases of language have surpassed this vacillation between motricity and representation? • As everyone knows, such work is divided into two schools: 1) Theory of Mind 2) Embodiment, although some researchers are leaving the path of controversy for a search for complementarity. • Is this a confirmation of MP’s diagnosis? His remark would shift from diagnosis to prognosis and even premonition: should we go that far? The turn taken by empirical research barely helps to fix our ideas.

12. Let’s not replace questions of essence with evolutionary narratives • The concern of biologists for the evolutionary origins of human capabilities may sometimes cause puzzlement to the philosopher. • It may seem natural to think that the primary issue of any inquiry, philosophical, empirical or otherwise, is the question: ‘What is it?’, a question concerning the essence of the thing itself and not its becoming, its origins, its cause, its effects, etc.. • The transition from a study of language to a study of its neural basis led to a replacement of: ‘What is it?’ by: ‘Where did that come from?’ even though the story answering the second question does not necessarily provide the definition expected in response to the first. • Moreover, blurring the differences can be detrimental to our understanding: If speaking, hearing and understanding are to be conceived henceforth as species of movement or imitation of movement, surely our concepts of movement and of speech will have to be altered. Will moving still mean moving one’s body and will understanding someone still mean knowing what he means?

13. Piecemeal solutions conceal the full extent of the dilemma • The shortest way to account for the embodiment of language goes through observation of articulator gestures and hand gestures that accompany or replace speech: these gestures are what is seen in linguistic communication. • If the embodiment of language in all its dimensions has little meaning, an obvious short circuit is to look first in the cartographic representations of the body of sensorimotor cortical areas (Penfield 1950) for the potential correlates of the lexicon of action verbs. • From there, one will address the following problem: whether it is possible to extend what is true for gesture (or the motor repertory) to the entire verbal behavior in the hope that the generalization reveals the uninterrupted passage between phoneme production / perception and construction / interpretation of expressions or sentences. • But if gesture is already language, basing language on gesture cannot do much to clarify the nature of language. • If action verbs derive their meaning from actions they express, this is not the case in the rest of lexicon. Their metaphorical usage is a semantic innovation that might rather undermine than safeguard the link with action.

14. The challenge: rooting meaning morphologies in the body • The possibility for human beings to express through linguistic expressions perceived forms of the visual field and goals or affordances of the practical field depends on an underlying mediation between the categories of perception and of action presumably to be performed in brain circuits. • The abstract principle of the semantic universality of natural language presupposes such mediation without accounting for it : “If it is possible in general to talk of anything whatsoever in a sensible way, then it is also possible to talk of that thing in everyday language” (Tarski 1936). • One must understand step by step how it is possible that any configurations, whether objects of visual attention or goals of intended actions – configurations that emerge and stabilize in a silent experience – are promoted and safeguarded in terms of their expression in linguistic forms. • To clarify the transition between the morphologies of different eidetic types (not just linguistic type) that inform the conduct of agents-observers-speakers it is not sufficient to trace courses of events in brain circuits. • The problem of embodiment is not settled at the level of neurons because it is both and inextricably eidetic and psychophysiological.

15. The phenomenological tradition and its deceptive proxy: • Liberman, Gibson, Goldman, et alii • Merleau-Ponty, the one epistemically acceptable phenomenologist • Husserl, the founder and transcendantal scarecrow

16. The phenomenological tradition and its deceptive proxy • Without lapsing into an outdated imperialism, a philosopher may be surprised by what appears to be a revival of the theme of incarnation in the literature on embodiment coupled with a misunderstanding, if not a systematic attribution error, of the original sources of this theme. • The fact is that the requirement of thinking together, and inter-relating, bodily experience with the understanding the actions of others, action with the perception of the environment, imitation with intersubjective communication, dates back to the phenomenological movement in the early twentieth c. (Lipps, Dilthey, Husserl, Scheler, Stein, Reinach, Heidegger...). • However, everything happens as if current neuroscience sought a basically inadequate substitute for this phenomenology in authors who hesitate between behaviorism and cognitivism, between mentalism and physicalism, between computation and simulation (Liberman, Gibson, Goldman...).

17. Liberman, Gibson, Goldman, et alii • Instead of relying on a phenomenology, Liberman’s (1985) conception of articulator gesture seeks to frame phenomena in two successive doctrines: 1) a behaviorist concern to assign phonetic units to a coarticulation resistant, recordable movement ; 2) an assignment of phonetic encoding and decoding to a peripheral system, conforming to the modularity of mind doctrine. • Gibson’s concept of affordance and ecological theory of perception (1979) resembles a phenomenological description of the morphological structures of Umwelt for a living being, but is marred by a physicalism for which the perceptual invariants are due to information actually residing in the optical flow. • Goldman (1989) does not provide a satisfactory alternative (Gallese & Goldman 1998) to the theory of the mind that subordinates recognition of others to an inference of the cognitive subject, because his ‘simulation’ remains solipsistic and representational, insofar as the observer is supposed to have his own motor system objectified as a representation in mind and to use it for predicting the future behavior of an observed agent.

18. Merleau-Ponty, the one acceptable phenomenologist • Merleau-Ponty enjoys in cognitive science a favorable view that is denied to the philosopher from whom he borrows his ideas, namely Husserl, especially in his later texts on the body and intersubjectivity. • This unfair attribution goes so far as to conceal the Husserlian origins of the themes of his phenomenology of perception, despite the fact that Merleau-Ponty himself made no mystery of their provenance. • To counterbalance this trend, it is noteworthy that MP’s assimilation of the own body (Leib / Körper) with the body schema of neurologists (Head & Holmes 1911) is perhaps not the bridge one might think would lead to the functional somatotopic maps of current neuroscience. • MP’s uncritical adherence both to Goldstein’s dogma of Gestalt and observations (1920) of a single case: Schneider, a probable simulator (Goldenberg 2002), casts serious doubt on his description (J-L Petit in Merleau-Ponty: le corps en acte, Berthoz & Andrieu eds 2010).

19. Husserl, the founder and transcendantal scarecrow • In science philosophers are considered producers of theories to be tested experimentally. But in Husserl the issue is not of theory but of a lived experience – even if it's a thought experiment – which requires a conversion of attitude in anyone who wants to follow suit. • That's what is needed to understand a paradox of his phenomenology of language pointed out by Merleau-Ponty (1960): starting from an eidetic science of ideal essences of meaning as a priori norms for any language (Log. Untersuchungen IV, 1901), he came to the truly phenomenological point of view of later texts where: • « …le langage apparaît comme le corps de la pensée pour le sujet parlant qui use de sa langue comme d’un moyen de communication avec une communauté vivante… ». • We are invited to take part in a thought experiment: that of the gradual lifting of intellectual obstacles enabling the scholar to go beyond the rigid dichotomies of a logical understanding to reach a harmonious integration of the two dimensions of language: ideality – incarnation in a single constitutive process.

20. Body/language ambivalence of expressions reconsidered • Whoever approaches language through logical ideality opposes linguistic expression and bodily expression, while an embodied conception of language allows for a founding continuity leading from one to another: • „Zu betonen ist, dass auch die so genannten unwillkürlichen „Ausdrücke“ unserer Seelenlebens, wie Mienenspiel und Geste, zur ausgeschlossenen Sphäre gehören, obwohl die gewöhnliche Rede es bei ihnen wie bei den sprachlichen Ausdrücken zu sagen gestattet, dass ihre Bedeutung verstanden ist.“ (Bedeutungslehre 1908) • „Für Sehende, für Hörende, Sprechende sind die Worte „Ausdrücke“, sind die Leiber Ausdrücke, die einen für Mitteilungen an andere Menschen, die anderen als Ausdrücke vom Dasein von Personen. Wortausdrücke setzt im Ausgedrückten Menschen als ausgeredete und nicht nur redende. Der erste und einfachste Ausdruck ist der des leiblichen Aussehens als Menschenleib, er setzt natürlich „Sehende“ und verstehende voraus. “ (Phänomenologie der Intersubjektivität III 1935) • The overcoming of solipsism which posed communication as inessential to thought promotes body expression to the status of linguistic expression and the foundation of subjective experience in intersubjectivity involves the founding of expression in communication. • Einfühlung – a natural mode of perceiving the body of the other as a direct expression of inner life, but also an ethical imperative one should practice in expressing one’s own inner life (Ricœur: Soi-mêmecomme un autre).

21. Three ways to bridge the gap between perception and action, and language: • Tracing the flow of neural events in brain circuits of the speaker-hearer • Descriptive eidetic phenomenology and its geometric modelling • Transcendental constitution of a verbally articulate Lebenswelt

22. Kinaesthesia in the Constitution of Lebenswelt • How is it possible that the chain of physical events do not unfold in me without me, but that I have a sensible experience? How is it that visual forms have for me the value of independent things in the world or that movements of this body carry my (or the other’s) intentions? • How is it that expressions heard or produced do not simply obey the rules of phonology, syntax and semantics of some language (or reflect its statistic regularities), but are endowed with sense for communicating subjects? • The constitution of Lebenswelt is an intersubjective foundation of the ordinary life world of personal agents in a community, which perceive, act and communicate through speech drawing on their own resources and actively mobilizing their bodily capabilities of giving meaning. • The experience of one’s body, and its extension through the intropathic experience of the other’s body are operative in giving sense to objects through the progressive recruitment of the kinaesthetic systems of the body, from ocular to manual movements, and to locomotion. • NB: Neglect of the kinaesthetic system by Merleau-Ponty made of Husserl’s hantieren (dealing with) a ghostly relationship: « Le corps hante le monde ».

23. Three ways to bridge the gap (not only one) • For the first time in history of the knowledge of man we see on the basis of data of empirical research a possibility to trace the uninterrupted course of events inside the organism that goes from perception and action to communication through language. Not content with tracking correlative activity patterns in the circuits of an isolated brain, neuroimagery reveals synchronizations of such patterns in the brains of communicating individuals (Wilson 2007; Schippers 2009, 2010; Stephens 2010). • However the narrative of brain events involved is far from answering all questions, despite the reductionist appeal of such chains of events for any naturalistic explanation of human behavior and capability. ‘The effort of the organism toward meaning’ (Lord Adrian) falls far short of giving a univocal ontological genesis. • It remains a sequence of mere facts that keep the contingency of what is empirical despite their derivation from the history of phylogenetic evolution and ontogenetic development.

24. Tracing neural events in brain circuits of the speaker • As cognitive science, the work on the neural basis of language is not just to go back to pure physical events: the occurrence of a change of brain state. • First, the events that they seek as sciences of nature are the regulatory ideas of a consensus expected at the end of an ongoing controversy in the community of neuroscientists: Should we say ‘modulation’ or ‘induction’ or simply ‘spreading of activation’? • These are not perfectly objective entities that are only what they are and whose unambiguous description might be immune to ‘the conflict of interpretations’, but rather the likely signature of a behavioral task or the language capability this conduct denotes. • The fact that correlative brain events cannot be taken in isolation from the verbal behavior in ‘ecological’ conditions captured by the experimental protocol restores priority to phenomenology of language. • Speaking of communication between brains as if the dialogue between speaking subjects were a tale for the public remains a misnomer.

25. The challenge of part-whole semantic dependency • The electrophysiological discharge of a nerve cell is an individual event so riveted to the present modality of its occurrence, that it contains no reserve of being to be further determined. • Nothing in common with the entity of meaning expressed by a syncategorematic expression, one which realizes its function by its completion with other expressions of which it contains (not in explicit form but in the signifying intention) the empty place, a place quite determined, nevertheless, since it specifies a priori the category of suitable complementary expressions in the sentence: • „Synkategorematika werden als Träger inhaltlich bestimmter Bedeutungsmomente aufgefasst, die nach einer gewissen Ergänzung verlangen, und zwar einer Ergänzung, die, obschon der Materie nach unbestimmt, doch ihrer Form nach durch den gegebenen Inhalt mitbestimmt und somit gesetzlich umschrieben ist.“ (Log. Untersuchungen IV § 5) • The generality of this morphological structure of incompleteness-dependence is especially supported by the lastly revived structural syntax of Tesnière (1965): • « Les connexions entre les mots ne sont indiquées par rien. Mais il est indispensable qu’elles soient aperçues par l’esprit sans quoi la phrase ne serait pas intelligible (Eléments A, I, 4). »

26. Can connectionist modelling meet the challenge? • How is it possible that the brain frees itself from the transient and contingent flux of events in its neural circuitry so as to be sensitive to semantic idealities, to give them life and support over time? • The question is no longer an absolute enigma posed by the phenomenology of language for empirical sciences, since the latter were split into a more observational neurophysiology (brain imagery) and a theoretical neuroscience working on hypothetical models: • One might consider that some naturalization of eidetics (not necessarily under that name) is in progress in so far as the modelling of the detection of nomological relations of dependence between spatiotemporally non-adjacent elements of the verbal flow by neural networks implicitly naturalizes Husserl’s Log. Untersuchungen IV: • McCulloch, Pitts (1943); Minsky, Papert (1969); Pulvermüller (2002).

27. Eidetic phenomenology and its geometric modelling • Phenomenology claims to describe the verbal or perceptual semantic forms ‘without prejudice’, that is to say without making assumptions about the causal mechanisms of basic brain substrate. • His approach is essentialist, but not fixist, because the semantic forms of expression (words, phrases) are driven by the Ergänzungs-bedürftigkeit: the need or requirement of completion which leads these forms to become parts of wholes (sentences, speech). • Which brings them back to the mereological standard of perception (Log. Untersuchungen III), suggesting a transition from the muteness of perceptual (or practical) forms to the expressive forms of language. • A geometric morphodynamics (Thom 1988, Petitot 2011) undertook to model the morphogenetic dynamism with which semantic forms emerge from the physical substrate, by stabilizing them at the phenomenal plane and then transforming each into the other with a view to structuring the sense of experience of the speakers.

28. Lebenswelt : the limit of body foundationalism • The extension carried out by Einfühlung of the circle of actions and intentions of the ego to actions and intentions of others stumbles on social acts:(1) accomplished through speech (2) dependent on the reception by the addressee (3) separating in time the utterance and the realization (4) building supratemporal and immaterial relationships. • The socially basic act of promising something to someone, and keeping one’s word, owes nothing to empathy (Reinach 1921). • In addition to kinaesthesia and Einfühlung the constitution of the Lebenswelt, including idealities of the Law regulating social acts, requires the recognition of speech and even the whole ‘formalism’ of language as fully-fledged (but non bodily) constitutive operations. • The constitutive power of language in relation to institutional realities no longer depends on kinaesthesia or Einfühlung: hence the skepticism one may have towards the ambitions of a ‘social neuroscience’ seeking its roots in mirror neurons (or in cortical maps).

29. Conclusion • Faced with issues unresolved by the mere narrative of events in a brain, that which is currently presented as a univocal ontological genesis of an embodied meaning will predictably break out in three directions: • 1) neurophysiologicinvestigation of the organic substrate of the continuous linkage between perception and action, and language; • 2) eidetic-geometricmorphodynamics as norm a priori backing the transformation of forms/schemes in syntactic or semantic structures; • 3)transcendentalconstitutionof the Lebenswelt of a community of perceiving-acting personal subjects who interact by words and gestures drawing on bodily capabilities and other operations of meaning-giving. • Of these lines of approach only the first unquestionably ranks in the ideological framework of a naturalistic science, while the remaining lines cannot simply be fitted into traditional metaphysical dualism. • Therefore we plead for an epistemology of language that is neither monistic nor dualistic, but rather trinitarian, as an alternative to the physicalism of current neuroscience (under its embodied disguise).

30. For more: Thank you! www.jlpetit.com phenomenologica