1 / 31

WP5 Cognitive Integration and Bootstrapping

This article provides an overview of the integration of cognitive capabilities in robots, focusing on language as a cognitive tool and the role of motor affordances. It discusses the roadmap for research in this area, planned experiments, and the relationship between neuroscience and human-robot interaction.

cruzt
Download Presentation

WP5 Cognitive Integration and Bootstrapping

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WP5 Cognitive Integration and Bootstrapping Angelo Cangelosi,Alessandra Sciutti, Metta, Sagerer, Nolfi, Nehaniv, Fischer ,Tani, Belpaeme, Fadiga, Nori, Rohlfing, Wrede, Tuci, Tikhanoff, Schillingmann, Lohan, Mirolli, Saunders, Dautenhahn, Zeschel, Marocco, Peniak, Foerster, Massera, Sato, Lyon, Pitsch, Gijsberts, Beck, et al.

  2. Overview WP5 Overview: The integration WP? Task 5.1 (Roadmap) Road Map preparation Results and dissemination Task 5.3 (HRI and neuroscience) Planned experiments Main result(s) Task 5.2 (language as cognitive tool) Planned experiments Integration Outlook ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  3. Overview WP5 Overview: The integration WP? Task 5.1 (Roadmap) Road Map preparation Results and dissemination Task 5.3 (HRI and neuroscience) Planned experiments Main result(s) Task 5.2 (language as cognitive tool) Planned experiments Integration Outlook ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  4. The Integration WP? • Different levels of integration of work • Intra-WP: Each WP includes integration of cognitive capabilities • E.g. WP1 action withlanguage (T1.4  T4.3) • E.g. WP3 social withlanguage (T3.1) • Inter-WP: WP5 for additional aspects of cognitive integration (bootstrapping) • Roadmap vision (T5.1) • Extra-linguistic roles of language (T5.2) • From neuroscience to HRI (T5.3, T5.4) • Work (eg. results, software) integration • Standardisation process (iCub/ITALK repository)

  5. WP5 Objectives • WP Objectives (Year 1) • to identify the underlying methodological, theoretical and technological principlesfor research on the integration of linguistic, motor and cognitive capabilities in robots; • to uncover, from both a theoretical and an application point of view, the relationships between the different levels of integration and transfer of action and language knowledge through the involvement of the mirror-neuron system and the role of motor affordances in human-robot communication experiments • the understand the role of language as a cognitive tool and its role in the bootstrapping of cognitive capabilities such as attention and gaze sharing, syntax processing, and object representation through motor affordances ROADMAP Gaze experiments Language as cognitive tool

  6. Overview WP5 Overview: The integration WP? Task 5.1 (Roadmap) Road Map preparation Results and dissemination Task 5.3 (HRI and neuroscience) Planned experiments Main result(s) Task 5.2 (language as cognitive tool) Planned experiments Integration Outlook ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  7. ROADMAP: Process • Meeting 1: Planning of process • Meeting 2: Rome Workshop • Drafting • 5 Thematic groups • Email co-writing • Dissemination • Website • Submission to IEEE TAMD journal

  8. ROADMAP: Intl. Workshop • Participants • All ITALK principal scientists • ISAB/Advisors: • I. Pepperberg (Harvard USA) • M. Bowermann (Max Planck) • A. Glenberg (Arizona State) • R. Kubat (MIT USA, Roy’s lab) • K. Plunkett (Oxford UK) • S. Harnad (UQAM & Southampton University.) • R. Grush (UCSD USA) • Related EU projects: • L. Steels (ALEAR), T. Ziemke (ICEA), A. Borghi (ROSSI), P. De Ruiter (JAST), P. Dominey (CHRIS)

  9. ROADMAP: Intl. Workshop

  10. ROADMAP: Post-Workshop • Thematic Groups • Introduction/Vision/Milestones: led by A. Cangelosi (PLYM) with contribution from T. Belpaeme and all other staff • Action Learning Challenges: led by G. Metta and L. Fadiga (IIT) with contribution from PLYM staff • Category Learning Challenges: led by S. Nolfi (CNR) with contribution from PLYM and RIKEN staff • Social Learning Challenges: led by G. Sagerer (BIEL) with contribution from UH and USD staff • Language Learning Challenges: led by K. Fischer (USD) with contribution from BIEL, PLYM and RIKEN staff

  11. ROADMAP Results:Key Challenges • Understanding how agents learn and represent compositional and hierarchical actions; • Understanding how agents acquire and represent compositional lexicons and linguistic knowledge; • Understanding the dynamics of social interaction and social learning, • Understanding of how compositional action and language representations are integrated to bootstrap cognition.

  12. ROADMAP Results: Milestones

  13. ROADMAP Results: Milestones

  14. ROADMAP: Publication • Roadmap white paper (website) • Publication (submitted) • Workshop proceedings (CDROM and on-line)

  15. Overview WP5 Overview: The integration WP? Task 5.1 (Roadmap) Road Map preparation Results and dissemination Task 5.3 (HRI and neuroscience) Planned experiments Main result(s) Task 5.2 (language as cognitive tool) Planned experiments Integration Outlook ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  16. Task 5.3: goals • Human-robot experiments and mirror-neuron system: Gaze behaviour during observation of actions performed by a humanoid artefact • Human-human interaction: mo 6-24 • Human-robot interaction: mo 24-30 ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  17. Proactive gaze behaviour and mirror neurons • Gaze behaviour is proactive (Flanagan and Johansson 2003)  fundamental in interactions (also human – robot). • Which are the characteristics of the observed action that evoke the pro-active gaze behaviour? Eye movements are used to reflect the activity of a motor resonance involving a mirror-neuron system. • Proactive gaze behavior as a function of: • Observed action kinematics • Cognitive cues ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  18. Experiments Effects of the observation of liftingobjects of different weight on: • Mirror neurons involvement • Previous experiment • Proactive gaze behaviour • New ongoing pilot experiment ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  19. Object lifting task Optotrak cameras Optotrak markers Eyelink system ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  20. Stimuli Object appearance Demonstrator’s kinematics Visible heavy vs. Visible light Hidden heavy vs. Hidden light Demonstrator’s kinematics only Labeled heavy vs. Labeled light (both heavy) Object appearance only: cognitive cues ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  21. Mirror neurons involvement Normalized MEP area (Z-score) EMG : FDI TMS : 120% of MT 50 ms after lifting Protocol: 8 subjects, 10 trials,Conditions randomized ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  22. Hand – Gaze Measures ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  23. Preliminary results Light Placing Lifting HAND Kinematics Heavy HAND & GAZE Observer Demonstrator ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  24. Current and future steps • Currently: setup ready, starting data collection. • Next steps: • Modulating predictability of demonstrator’s motion (e.g. alternating different arrival points) • Robot as demonstrator • Modulation of kinematics • Modulation of interaction ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  25. Overview WP5 Overview: The integration WP? Task 5.1 (Roadmap) Road Map preparation Results and dissemination Task 5.3 (HRI and neuroscience) Planned experiments Main result(s) Task 5.2 (language as cognitive tool) Planned experiments Integration Outlook ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  26. T5.2: Language as a cognitive tool and its role in enhancing cognition (e.g. inner speech) The aim of this task is that to verify, through robotic experiments, whether: • the co-development of behavioral and linguistic skills represent a crucial prerequisite for the development of complex cognitive skills • linguistically mediated social interactions cause a radical transformation of elementary cognitive abilities, as proposed by Vygotsky (1962, 1978) • Talking-to-oneself (inner speech) as a cognitive aid tool (Vygotsky 1978)

  27. Objects manipulation scenario • A robot located in front of few objects will be trained (through a trial and error process) for displaying: • few behavioural skills (e.g. “touching the blue object”, “indicating the red object”, or “moving the green object”) • Associated linguistic skills that will consists in “verbally describing the behaviour that the robot is currently exhibiting” and/or “listening to a verbal command and executing the corresponding behaviour”

  28. Shape recognition/drawing scenario • A robot located in front of a white board will be trained (through a trial and error process)for: • Drawing a shapes (e.g. simple geometrical shapes such us a square or a circle or more elaborate shapes) by moving a pen after having observed the experimenter drawing the same shape • Verbally describing the shape being drawn by the experimenter or by the robot itself

  29. Research Objectives • Through these experiments we will investigate: • Whether the possibility to be exposed to linguistic inputs might facilitate the development of the required behavioural and cognitive skills. • Whether the need to produce linguistic outputs can facilitate the development of the required behavioural and cognitive skills • Whether the possibility of the agent to access own linguistic outputs (inner speech) can represent a prerequisite for the development of certain behavioral and cognitive skills • How the internal categories developed by the agents are co-shaped by the development of behavioral and linguistic skills

  30. Overview WP5 Overview: The integration WP? Task 5.1 (Roadmap) Road Map preparation Results and dissemination Task 5.3 (HRI and neuroscience) Planned experiments Main result(s) Task 5.2 (language as cognitive tool) Planned experiments Integration Outlook ITALK Year 1 ReviewDüsseldorf, 1 July 2009

  31. WP5 Outlook: Integration for Cognitive Bootstrapping • Additional dimensions in cognitive integration • E.g. Neuroscience and mirror neuron system (Tasks 5.3-5.4) Mirror neuron involvement on gaze in humans   Mirror neuron involvement on gaze in HRI   Mirror neuron involvement on linguistic complexity in HRI • E.g. Self-talk (Tasks 5.2) Language for communication   Language for taking-to-oneself  • Language as cognitive aid tool  • language/cognition relativism

More Related