PLOW: A Collaborative Task Learning Agent

1. PLOW: A Collaborative Task Learning Agent Author: James Allen, Nathanael Chambers, etc. By: Rex, Linger, Xiaoyi Nov. 23, 2009

2. Outline • Introduction • The PLOW System • Demonstration • Learning Tasks • Evaluation • Strength & Weakness • Related Works • Q&A and Discussion

3. Introduction • Aim to further human-machine interaction • Quickly learn new tasks from human subjects using modest amounts of interaction • Acquire task models from intuitive language-rich demonstration

4. Background • Previous Work: Learn new tasks by observation, learn throw observing experts’ demonstration. • Paper’s Contribution: Acquire tasks much more quickly, typically from a single example, maybe with some clarification dialogue.

5. The Interface

6. Agent Architecture

7. Language Processing • Focus on how it is used for task learning, rather than how it is accomplished • TRIPS system • Based on a domain-independent representation

8. Instrumentation • The key issue is to get the right level of analysis for the instrumentation DOMs

9. A demo

10. Task Learning • Challenges • Identifying the correct parameterization • Hierarchical structure • Identifying the boundaries of iterative loops • Loop termination conditions • Task goals

11. Primitive Action Learning • NL Interpretation + GUI Interpretation • Heuristic search through DOM • Semantic metric • Structural distance metric

12. Primitive Action Learning Natural Language Interpretation Heuristic search GUI Interpretation Structural Distance Metric Semantic metric

13. Parameterization Learning • Identify appropriate parameterization • Object roles • Input/output parameter • Constant • Relational dependency • Information from language

14. Parameterization Learning • Output: Hotels • Input: Address • Constant: Hotels • Relational Dependency: Zip is Role of Address

15. Hierarchical Structure Learning • Beginning of new procedures • A Goal • End of procedure

16. Iteration Learning • Iterative procedure learning • Language support • PLOW’s attempt for Iteration • User corrections/more example • Rule/Pattern learned

17. Iteration Learning

18. Evaluation • 16 test subjects with general training • 3 other systems: • One learned entirely from passive observation • One used a sophisticated GUI primarily designed for editing procedures but extended to allow the definition of new procedures • One used an NL-like query and specification language that requires detailed knowledge of HTML producing the web page

19. Evaluation – First part • Subjects taught different systems about some subset of predefined test questions • Evaluators created new test examples by specifying values for input parameters, then scored the execution results • PLOW scored 2.82 out of 4 (not mention other systems’ scores)

20. Evaluation – Second part • 10 new test questions designed by an outside group, unknown to the developers prior to the test • Subjects had one work day to teach whichever of these tasks they wished, using whichever of the task learning system • PLOW was used to teach 30 out of 55 task models

21. Evaluation – Second part • 10 new test questions designed by an outside group, unknown to the developers prior to the test • Subjects had one work day to teach whichever of these tasks they wished, using whichever of the task learning system • PLOW was used to teach 30 out of 55 task models • Also, PLOW received the highest average score in the test (2.2/4)

22. Strength • Integrating natural language recognition and understanding (TRIPS, 2001) • “Play by play” mode, great user experience • Easier to identify parameters, boundaries of loops, termination conditions, build hierarchical structure, realize goals • Generalization from one short task • Learn not only the task, but also the rule

23. Strength • Error correction from users • “This is wrong. Here is another title” • PLOW will confirm correctness from users when generating data from lists • Less domain knowledge required, less training • Close to “one-click automation”

24. Weakness • Some remarks of Evaluation: • PLOW was ensured of being able to learn 17 pre-determined test questions, other systems? • 10 new tasks have different levels of difficulties: ex: For what reason did <person> travel for <project> between <start date> and <end date>? • No detailed analysis of evaluation result, so does PLOW really learn robust task models from a single example? Or just better on certain types of tasks?

25. Weakness • Learning and reasoning relied on NL understanding: • encounters new concepts? • require certain patterns of speaking? Enough NL understanding capabilities? • Still need one full work day to teach 3 simple tasks/person • Users have to construct good task models, no error detection mechanism for users in PLOW

26. Related works • Sheepdog, 2004 • an implemented system for capturing, learning, and playing back technical support procedures on the Windows desktop • Complex technical supporting procedures – relatively simple procedures in PLOW • Record traces to form alignment problem, use I/O HMMs to build procedure models, need many training examples

27. Related works • Tailor, 2005 • a system that allows users to modify task information through instruction • Recognize user’s instruction: combine rules with parser, JavaNLP • Map sentences to hypothesized changes • Reason about the effects of changes, detect the unanticipated behavior • Also relatively simple tasks

28. Related works • CHINLE, 2008 • a system which automatically constructs PBD systems for applications based on their interface specification • Learning from incomplete data and partial learning from inconsistent data – PLOW can learn subset of certain tasks, but users cannot make mistakes

29. Questions?

30. More time for the latest PLOW demo?

31. Thank you!