Developmental Artificial Intelligence

1. Developmental Artificial Intelligence 1er April 2014 Olivier.georgeon@liris.cnrs.fr http://www.oliviergeorgeon.com t oliviergeorgeon.com

2. Outline • Example • Demos with robots. • Conclusion of the course. • Learning through experience. • Intermediary level of AI: semantic cognition. • Exercise • Implement your self-programming agent (follow-up). oliviergeorgeon.com

3. Robotics research Panoramic camera Bumper tactile sensor Ground optic sensor oliviergeorgeon.com http://liris.cnrs.fr/simon.gay/index.php?page=eirl&lang=en

4. Experimentation oliviergeorgeon.com

5. Developmental Learning in AI Theories Validation paradigm Implementations Phenomenology (Dreyfus, 2007) “even more Heideggarian AI” Radical Interactionism, ECA (Georgeon et al., 2013) perception-action inversion (Pfeifer & Scheier, 1994) Behavioral analysis rather than performance measure Horde (Sutton et al., 2011) SMC theory (O’Regan & Noë, 2001) Cybernetic control theory (Foerster, 1960) Learning by experiencing “intrinsic motivation” (Oudeyer et al. 2007) Learning by registering (Georgeon, 2014) Embodied AI. “Constructivist Schema mechanisms” (Drescher, 1991). Kuipers et al., (1997) Naively confusing perception and sensing (Crowley, 2014) Non-symbolic Machine Learning. Reinforcement Learning. POMDP, etc. Non-symbolic CLARION ACT-R Soar Symbolic Newell & Simon (1972) goals drive problem solving; (1976) Physical Symbol Systems.

6. Symbolic AI The “environment” passes “symbols” to the agent as input. We encode the “semantics” of symbols in the agent. We implement a “reasoning engine”. (“symbolic” should not be mistaken with “discrete”)

7. Developmental Learning in AI Theories Validation paradigm Implementations Phenomenology (Dreyfus, 2007) “even more Heideggarian AI” Radical Interactionism, ECA (Georgeon et al., 2013) perception-action inversion (Pfeifer & Scheier, 1994) Behavioral analysis rather than performance measure Horde (Sutton et al., 2011) SMC theory (O’Regan & Noë, 2001) Cybernetic control theory (Foerster, 1960) Learning by experiencing “intrinsic motivation” (Oudeyer et al. 2007) Learning by registering (Georgeon, 2014) Embodied AI. “Constructivist Schema mechanisms” (Drescher, 1991). Kuipers et al., (1997) Naively confusing perception and sensing (Crowley, 2014) Non-symbolic Machine Learning. Reinforcement Learning. POMDP, etc. Non-symbolic CLARION ACT-R Soar Symbolic Newell & Simon (1972) goals drive problem solving; (1976) Physical Symbol Systems.

8. Learning by registering 2 2 1 4 4 3 The “environment” passes “observations” to the agent as input. The relation state -> observation is “statistically” a surjection. We implement algorithms that assume that a given “state” induces a given “observation” (although partial and subject to noise).

9. Developmental Learning in AI Theories Validation paradigm Implementations Phenomenology (Dreyfus, 2007) “even more Heideggarian AI” Radical Interactionism, ECA (Georgeon et al., 2013) perception-action inversion (Pfeifer & Scheier, 1994) Behavioral analysis rather than performance measure Horde (Sutton et al., 2011) SMC theory (O’Regan & Noë, 2001) Cybernetic control theory (Foerster, 1960) Learning by experiencing “intrinsic motivation” (Oudeyer et al. 2007) Learning by registering (Georgeon, 2014) Embodied AI. “Constructivist Schema mechanisms” (Drescher, 1991). Kuipers et al., (1997) Naively confusing perception and sensing (Crowley, 2014) Non-symbolic Machine Learning. Reinforcement Learning. POMDP, etc. Non-symbolic CLARION ACT-R Soar Symbolic Newell & Simon (1972) goals drive problem solving; (1976) Physical Symbol Systems.

10. Positionnement dans le cadre de l’IA Theories Validation paradigm Implementations Phenomenology (Dreyfus, 2007) “even more Heideggarian AI” Radical Interactionism, ECA (Georgeon et al., 2013) perception-action inversion (Pfeifer & Scheier, 1994) Behavioral analysis rather than performance measure Horde (Sutton et al., 2011) SMC theory (O’Regan & Noë, 2001) Cognition située (Clancey 1992) “intrinsic motivation” (Oudeyer et al. 2007) Apprentissage par l’expérience Apprentissage « désincarnée » Apprentissage par l’observation (Georgeon, 2014) En confondant naivement “input” et “perception” (Crowley, 2014) Reinforcement Learning. Neural networks. Machine learning. A* etc. Non-symbolic Symbolic Newell & Simon (1972) goals drive problem solving; (1976) Physical Symbol Systems.

11. Change blindness http://nivea.psycho.univ-paris5.fr/ oliviergeorgeon.com

12. Learning by experiencing Time The environment passes the “result” of an “experiment” initiated by the agent. This is counter intuitive ! We implement algorithms that learn to “master the laws of sensorimotor contingencies” (O’Regan & Noë, 2001).

13. Accept the counter-intuitiveness • We have the impression that the sun revolves around the earth. • False impression! (Copernic, 1519) • We have the impression to receive input data about the state of the world. • False impression! (Philosophy of knowledge since the enlightenments, at least). • How to translate this counter-intuitiveness into the algorithms? oliviergeorgeon.com

14. The stakes: semantic cognition Reasoning and language Rule-based systems, Ontologies, traditional AI. Knowledge-grounding, sense-making, Self-programming. Semantic cognition Stimumuls-response adaptation Reinforcement-learning, neural nets, traditional machine learning. oliviergeorgeon.com

15. Conclusion • Think in terms of interactions • Rather than separating perception et action. • Think in terms of generated behaviors • Rather than in terms of learned data. • Keep your critical thinking • Invent new approaches ! http://e-ernest.blogspot.fr/

16. Invent new approaches “Hard problem of AI” Formalized problem A E A Etc. E http://e-ernest.blogspot.fr/

17. Exercice Part 4. oliviergeorgeon.com

18. Environnement 3 - modified • Behave like Environment 0 up to cycle 5, then like environment 1 up to cycle 10, then like environment 0 . • Implementation • If (step <= 5 or step > 10) • If (experiment = e1) then result = r1 • If (experiment = e2) then result = r2 • Else • If (experiment = e1) then result = r2 • If (experiment = e2) then result = r1 • Step++ oliviergeorgeon.com

19. Agent 3 in Environment 3 Environnement 0 Environnement 1 Environnement 0 0. e1r1,-1,0 1. e1r1,-1,0 learn (e1r1e1r1),-2,1 activated (e1r1e1r1),-2,1 propose e1,-1 2. e2r2,1,0 learn (e1r1e2r2),0,1 3. e2r2,1,0 learn (e2r2e2r2),2,1 activated (e2r2e2r2),2,1 propose e2,1 4. e2r2,1,0 activated (e2r2e2r2),2,2 propose e2,2 5. e2r1,-1,0 learn (e2r2e2r1),0,1 6. e2r1,-1,0 learn (e2r1e2r1),-2,1 activated (e2r1e2r1),-2,1 propose e2,-1 7. e1r2,1,0 learn (e2r1e1r2),0,1 8. e1r2,1,0 learn (e1r2e1r2),2,1 activated (e1r2e1r2),2,1 propose e1,1 9. e1r2,1,0 activated (e1r2e1r2),2,2 propose e1,2 10. e1r1,-1,0 learn (e1r2e1r1),0,1 activated (e1r1e2r2),0,1 activated (e1r1e1r1),-2,1 propose e2,1 propose e1,-1 11. e2r2,1,0 activated (e2r2e2r1),0,1 activated (e2r2e2r2),2,2 propose e2,1 12. e2r2,1,0 activated (e2r2e2r1),0,1 activated (e2r2e2r2),2,3 propose e2,2 13. e2r2,1,0 oliviergeorgeon.com

20. Principle of Agent 3 AGENT Activated (i11,i11) (it-1,it) (i12,i12) it-1 it i11 Choose e1 i12 i11 Learn Execute Propose Activate it-2 it-1 it= i11 it-4 it-3 i12 … Time PAST FUTURE PRESENT oliviergeorgeon.com

21. Environment 4 • Returns result r2 only after twice the same experience. • e1 -> r1, e1 -> r2, e1 -> r1, e1-> r1, … e1->r1, e2 -> r1, e2->r2, e2 -> r1, …, e2 -> r1, e1 -> r1, e1 -> r2, e2 -> r1, e2 -> r2, e1 -> r1, e1 -> r2, … • If (experiencet-2!=experiencet &&experiencet-1==experiencet) result = r2;else result = r1; oliviergeorgeon.com

22. Rapport • Agent 1 • Explanation of the code. • Traces in environments 0 and 1 with different motivations. • Explanation of the behaviors. • Agent 2 • Explanation of the code. • Traces in environments 0 to 2 with different motivations. • Explanation of the behaviors. • Agent 3 • Explanation of the code. • Traces in environments 0 to 4 with différent motivations. • Explanation of the behaviors. • Conclusion • What would be the next step towards Agent 4 able to adapt to Environments 1 to 4? oliviergeorgeon.com