Update on Learning By Observation Learning from Positive Examples Only

1. Update on Learning By ObservationLearning from Positive Examples Only Tolga Konik University of Michigan

2. GOAL • Generate AI agents by observing expert task execution • Engineering Goal • Reduce the cost of agent development • Reduce the expertise required to develop agent development. • AI Goal • Agents that improve themselves observing experts

3. Agent Architecture Agent Program external Internal Learning Framework Environmental Interface Environment Expert Behavior trace Annotations Behavior Recorder Annotated Behavior trace Knowledge Generator Episodic Database Background Knowledge rules Concept Learner (ILP) Training Set Generator examples

4. Agent Architecture Agent Program external Internal Learning with Redux Redux Environmental Interface Environment Expert Behavior trace Annotations Behavior Recorder Annotated Behavior trace Knowledge Generator Episodic Database Background Knowledge rules Concept Learner (ILP) Training Set Generator examples

5. Agent Architecture Agent Program external Internal Current Experiments Expert Soar Agent Environmental Interface Environment Expert Behavior trace Annotations Behavior Recorder Annotated Behavior trace Knowledge Generator Episodic Database Background Knowledge rules Concept Learner (ILP) Training Set Generator examples

6. Experiments in Haunt 2 Domain

7. i3 i4 r3 r2 d3 d4 d5b d6b d2 d1 d5 d6 r1 r4 Move-to example r3 d1 d2 d3 d4 move-to-area move-to-via-node move-to-connected-node

8. r3 r2 d3 d4 d2 d1 d5 d6 r1 r4 An Example in Haunt Domain move-to-area(Area) move-to-via-node(Node) move-to-connected-node(Node)

9. r3 r2 d3 d4 d2 d1 d5 d6 r1 r4 An Example in Haunt Domain move-to-area(Area) move-to-via-node(Node) move-to-connected-node(Node)

10. An Example in Haunt Domain r3 d1 r1 move-to-area(Area) move-to-via-node(Node) move-to-connected-node(Node) • Correct selection condition for move-to-via-node

11. Example GenerationOperator Concepts • Termination(A) A negative positive

12. Selection(A) Example GenerationOperator Concepts A B negative positive

13. Learning Examples i3 i4 r3 r2 d3 d4 d5b d6b d2 d1 d5 d6 r1 r4 • A Positive Example: • selection(Sit20, move-to-via-node(d1) )

14. General to Special Search with positive and negative examples

15. General to Special Search with positive and negative examples

16. General to Special Search with positive and negative examples

17. General to Special Search with positive and negative examples

18. General to Special Search with positive and negative examples

19. i3 i4 r3 r2 d3 d4 d5b d6b d2 d1 d5 d6 r1 r4 Problem in Choosing Parameters • Selection(move-to-via-node) move-to-via-node move-to-connected-node

20. i3 i4 r3 r2 d3 d4 d5b d6b d2 d1 d5 d6 r1 r4 Problem in Choosing Parameters • Selection(move-to-via-node) Negative Positive move-to-via-node move-to-connected-node

21. General to Specific Learning with Positive Examples Only Positive

22. General to Specific Learning with Positive Examples Only d1 Positive

23. Learning Examples i3 i4 r3 r2 d3 d4 d5b d6b d2 d1 d5 d6 r1 r4 • A Positive Example of move-to-via-node:

24. i3 i4 r3 r2 d3 d4 d5b d6b d2 d1 d5 d6 r1 r4 Learning Examples • Random Examples of move-to-via-node • For each positive example, use the same situation with parameters selected in other situations

25. Nuggets • Move-to operators are learned in Haunt domain • ~ 3 mins of trace • ~ 35000 situations • ~ 10 min to prepare examples • ~20 min for learning.

26. Coals • Missing Components • It is still research not a tool