Embodied Learning of Qualitative Models

1. Embodied Learning of Qualitative Models Jure Žabkar joint work with xpero partners Exploration and Curiosity in Robot Learning and Inference, DAGSTUHL, March 2011

2. problem “How shoulda robotchooseitsactions and experiences so as to maximizethe effectivenessof its learning?”

3. goals • to learn comprehensible models • no extrinsic reward • intrinsic reward: improved prediction model about the environment

4. our way • learning from scratch(no explicit background knowledge, but given a learning algorithm) • real robots, real-time learning

5. learning loop • observe the environment (collect data) • learn a model • use the model to predict the effect of each action • choose the best action (w.r.t. active learning strategy) • observe the environment and check whether the predictions match new observations

6. starting scenario Q: how does the area of the ball (as observed by the robot)change w.r.t. robot's actions? area := #pixels of the red blob in the image from robot's camera actions: sL, sR(the distance of the L/R wheel)

7. area = area(sL,sR) task: find the appropriate model equation discovery? we tried several algorithms, no success

8. motivation why learning qualitative relations? people most often reason qualitatively AI: robots should mimic human intelligence

9. the area problem, qualitatively if action=forward then the area increases until it becomes constant(blob occupies the whole image) if orientation<0 and action=left (increasing the absolute value of the angle)then the area decreases until it becomes constant(zero) ...

10. qualitative rules prediction model gets much more accurate, but the predictions are not that precise.

11. methods • active learning + planning • learning methods: PadéŽabkar, Možina, Bratko, Demšar Learning Qualitative Models from Numerical Data, AIJ, 2011 STRUDELKošmerlj, Bratko, Žabkar Embodied Concept Discoverythrough Qualitative Action Models, IJUFKS, 2011 QubeŽabkar et al Preference Learning from Qualitative Partial Derivatives, ECML Preference Learning Workshop, 2010 Hyper (with predicate invention mechanism)Leban, Žabkar, Bratko An experiment in robot discovery with ILPProc. ILP 2008 • tested on simulated (billiards) and real data (medical application, robotics)

12. ceteris paribus "all other things being equal" • e.g. partial differentiation • observe a qualitative relation between two selected features, other features held constant • qualitative relations of 3 types: • x increases  f(x) increases (Padé) • preference relation: x y  f(x) f(y) • structural: on(A,B,t1), on(A,C,t2)

13. qualitative models data Padé, Qube, STRUDEL qualitative changes machine learning, statistics qualitative models

14. qualitative models data Padé, Qube, STRUDEL qualitative changes machine learning, statistics qualitative models

15. qualitative models data Padé, Qube, STRUDEL qualitative changes machine learning, statistics qualitative models

16. learning with structured data • ILP with predicate invention too complex for real-time learning • we use ILP to learn smaller subtasks – structural qualitative changes

17. www.ailab.si/xpero

18. the concept "movable" the discovered condition which distinguishes different effects of actions: p1(Obj):- at(T1, Obj, Pos1), at(T2, Obj, Pos2), neq_pos(Pos1, Pos2). move(T, Obj):- p1(Obj), f1(T, Obj). move(T, Obj):- not p1(Obj), f2(T, Obj). the discovered effects of actions: • f1(T1, Obj):- at(T1, Obj, Pos1), at(T2, Obj, Pos2), Pos1 \== Pos2, {T2 = T1+1}. • f2(T, Obj):- not f1(T, Obj). p1 is true if the objectwas observed at two different positions