1 / 20

Outline IM-CLeVeR: Intrinsically Motivated Cumulative Learning Versatile Robots

IM-CLeVeR: Intrinsically Motivated Cumulative Learning Versatile Robots Gianluca Baldassarre , Marco Mirolli , Francesco Mannella, Vincenzo Fiore, Stefano Zappacosta, Daniele Caligiore, Fabian Chersi, Vieri Santucci, Simona Bosco.

ellis
Download Presentation

Outline IM-CLeVeR: Intrinsically Motivated Cumulative Learning Versatile Robots

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IM-CLeVeR: Intrinsically MotivatedCumulative LearningVersatile Robots Gianluca Baldassarre, Marco Mirolli,Francesco Mannella, Vincenzo Fiore, Stefano Zappacosta,Daniele Caligiore, Fabian Chersi, Vieri Santucci, Simona Bosco Gianluca Baldassarre

  2. OutlineIM-CLeVeR: Intrinsically Motivated Cumulative Learning Versatile Robots • The “numbers” of the project • The partners • The project vision • The 3 pillars of the project hypothesis + 4 S/T objectives • WP3: Experiments • WP4: Abstraction • WP5: Intrinsic motivations • WP6: Hierarchical architectures • WP7: Integration and demonstrators • Conclusions Gianluca Baldassarre

  3. The “Numbers” of the ProjectIM-CLeVeR: Intrinsically Motivated Cumulative Learning Versatile Robots • Integrated Project • Coordinator: ISTC-CNR • Call: Cognitive Systems, Interactions and Robotics • EU funds: 5.9 ml euros • 7 Partners • Start: May 2009 • End: April 2013 Gianluca Baldassarre

  4. Partners Institutes Groups 1. CNR-ISTC-LOCEN Coordinator (Gianluca Baldassarre, Marco Mirolli) 1. CNR-ISTC-UCP (Elisabetta Visalberghi) 1. UMASS (Andrew Barto) 2. USFD (Peter Redgrave & Kevin Gurney) 3. UCBM-LBRB (Eugenio Guglielmelli) 3. UCBM-LDN (Flavio Keller) 4. FIAS (Jochen Triesch) 5. AU (Mark Lee) 6. UU (Ulrich Nehmzow) 7. IDSIA (Juergen Schmidhuber) Gianluca Baldassarre

  5. Vision: the problem • How can we create “truly intelligent” robots? • Versatile: have many goals; re-use actions • Robust: function in different conditions, with noise • Autonomous: learning is paramount • Weng, McClelland, Pentland, Sporns, Stockman, Sur, Thelen, (Science, 2001): • …knowledge-based systems (e.g. production systems)… • …learning systems focussed on single tasks (e.g. RL)… • …evolutionary systems… •  Important results, but limited autonomy and scalability. . . . . . on the contrary . . . • . . . organisms do scale, are flexible, and are robust! Gianluca Baldassarre

  6. Vision: the idea • Why are organisms so special? • Let’s give a closer look at children… Gianluca Baldassarre

  7. Vision: the idea Ingredients: • Powerful abstractions: “elefant on table leg”, “it slides down” • Explore andrecord interesting states: • Based on intrinsic motivations (novelty, learning rates, …) • Such states motivate to reach them (= goals) • Furnish learning signals which guide learning • Acquired skills are: • Re-used to explore and discover new goals • Composed to produce new skills • Science: which brain and behaviouralmechanisms are behind these processes? • Technology: Can we reverse engineer them? Gianluca Baldassarre

  8. Vision: 2 promises • Science: we can understand the mechanisms in organisms • Technology: we can develop a new methodology for designing robots… … in particular robots that (we will get 3 iCubs!)… Learn actions cumulatively: …on the basis of intrinsic motivations… …on the basis of abstraction (sensory andmotor)… …on the basis of already learned actions. Gianluca Baldassarre

  9. Vision: how we will do it:3 pillars + 4 S/T objectives From Technologyto Science FromScience to Technology WP4: Abstraction and attention 2. Computational bio-constrained models: mechanisms underlying brain and behaviour Technology Science Suitable representations 4. Two robotic demonstrators: - CLEVER-B - CLEVER-K 1. Empirical investigations: -Monkeys - Children - Adults - Parkinson patients WP5: Intrinsic motivations 3. Machine-learning models: powerful algorithms and architectures Focussing learning WP6: Hierarchical architectures to support cumulative learning Gianluca Baldassarre

  10. WP3: Experiments and mechatronic board From Science to Technology WP3 WP4: Abstraction and attention 2. Computational bio-constrained models: mechanisms underlying brain and behaviour Technology Science Suitable representations 4. Two robotic demonstrators: - CLEVER-B - CLEVER-K 1. Empirical investigations: -Monkeys - Children - Adults - Parkinson patients WP5: Intrinsic motivations 3. Machine-learning models: powerful algorithms and architectures Focussing learning WP6: Hierarchical architectures to support cumulative learning Gianluca Baldassarre

  11. Tactile sensors Inertial/magnetic unit + battery + wireless WP3: Empirical Experiments: “Board experiment” • UCBM-LBRB (Eugenio Guglielmelli); • UCBM-LDN (Flavio Keller): children • CNR-ISTC-UCP (Elisabetta Visalberghi): monkeys; Sabbatini, Stammati, Tavares, Visalberghi, 2007,Amer. J. Primatology Campolo, Taffoni, Schiavone, Formica, Guglielmelli, Keller, 2009, Int. J. Sicial Robotics Gianluca Baldassarre

  12. WP4: Abstraction WP4 From Science to Technology WP4: Abstraction and attention 2. Computational bio-contrainedmodels: mechanisms underlying brain and behaviour Technology Science Suitable representations 4. Two robotic demonstrators: - CLEVER-B - CLEVER-K 1. Empirical investigations: -Monkeys - Children - Adults - Parkinson patients WP5: Intrinsic motivations 3. Machine-learning models: powerful algorithms and architectures Focussing learning WP6: Hierarchical architectures to support cumulative learning Gianluca Baldassarre

  13. WP4 Abstraction: motor, perception, attention, vergence, • Abstraction is a key ingredient for action hierarchies • Abstraction is a key ingredient for intrinsic motivations Schembri, Mirolli, Baldassare, 2007,ICDL, ECAL, EPIROB Neto Nehmzow, 2007, Rob. & Aut. Syst. Gianluca Baldassarre

  14. WP5: Novelty detection From Science to Technology WP4: Abstraction and attention 2. Computational bio-contrainedmodels: mechanisms underlying brain and behaviour Technology Science Suitable representations 4. Two robotic demonstrators: - CLEVER-B - CLEVER-K 1. Empirical investigations: -Monkeys - Children - Adults - Parkinson patients WP5: Intrinsic motivations WP5 3. Machine-learning models: powerful algorithms and architectures Focussing learning WP6: Hierarchical architectures to support cumulative learning Gianluca Baldassarre

  15. WP5 Intrinsic (extrinsic) motivations • Intrinsic motivations (skill/knowledge acquis.): • Psychology: motivate actions for their own sake • Drive actions whose effects are an increase in:(a) knowledge or prediction ability;(b) competence to do • Terminate to drive actions when knowledge or competence is acquired • Extrinsic motivations (e.g. food, sex, money): • Psychology (Berlyne, White, Deci & Rayan):motivate actions to achieve specific goals • Drive actions whose effects directly increase fitness • Come back again with the homeostatic needs they are associated with Gianluca Baldassarre

  16. WP5 Intrinsic motivations • CNR-LOCEN (Gianluca Baldassarre, Marco Mirolli) • Young robot: low level of hierarchy develps skills based on evolved ‘reinforcers’ (knowledge-based intrinsic motivations) • Young robot: high level of hierarchy selects skills which produce the highest suprise (competence-based intrinsic motivations) • Adult robot: high level of hierarchy performs skill composition to achieve salient goals (external rewards fitness measure) Adult robot tasks Adult robot: results Child robot task Young robot: results Before learning After learning Gianluca Baldassarre Schembri, Mirolli, Baldassare, 2007, ICDL, ECAL, EPIROB

  17. WP6: Hierarchical architectures From Science to Technology WP4: Abstraction and attention 2. Computational bio-mimeticmodels: mechanisms underlying brain and behaviour Technology Science Suitable representations 4. Two robotic demonstrators: - CLEVER-B - CLEVER-K 1. Empirical investigations: -Monkeys - Children - Adults - Parkinson patients WP5: Intrinsic motivations 3. Machine-learning models: powerful algorithms and architectures Focussing learning WP6: Hierarchical architectures to support cumulative learning WP6 Gianluca Baldassarre

  18. WP6 Bio-inspired / bio-constrained hierarchical reinforcement learning • CNR-LOCEN (Gianluca Baldassarre & Marco Mirolli) • Piaget theory: actions support learning of other actions • Camera, dynamic arm, reaching tasks • Continuous state/action reinforcement learning • Hierarchical RL: segmentation, Piaget Gianluca Baldassarre Caligiore Borghi Parisi Mirolli Baldassarre, ongoing From Fuster, 2001, Neuron

  19. Conclusions: A timely project! • Timely research goals:sensorimot. abstraction, intrinsic motiv., hierarchical architect. • Within important trends: • Developmental robotics • Computational system neuroscience • Emotions/motivations • In synergy with various events:EpiRob, ICDL, IEEE Journal Automonous Mental Development • In line with EU calls:“Cognitive Systems, Interactions and Robotics” • First EU Integrated Project wholly focussed on these topics www.im-clever.eu Gianluca Baldassarre

  20. What we need from iCub • Robustness! • Usability (assistance): • Software: Yarp, simulator for rapid prototyping • Hardware: when it will break • One standardised simulator (e.g., based on Bullet) • Compliance: for safety, for more bio-realism • That it actually becomes a standard in EU research Gianluca Baldassarre

More Related