An Agent Capable of Learning to Create and Maintain Websites

1. An Agent Capable of Learning to Create and Maintain Websites Anthony Tomasic, Ravi Mosur Alex Rudnicky, Raj Reddy, John Zimmerman Carnegie Mellon University 18 April 2003

2. Project vision Problem Assumptions Inputs Outputs Missing capabilities Our approach Impact Evaluation Conclusion Outline

3. Project Vision – Honeydew • Honey – An agent • Learns by observation • Obtains advice and consent • Creates and maintains public project websites

4. Project Vision – Honeydew • Honey Can Perform Tasks • Organize, manage and update a complex project website • Delegate tasks • Generate periodic briefing folders • Email extracted material and online documents with planning and summarization capabilities • Respond to specific information queries • Extract relevant information • WWW & mailing lists • Perform teaching • Communicate with other Honeys and EPCAs

5. Problem Assumptions • Scope of a “project” website is predefined and not learnt: • Publications • Presentations • Milestone status • News updates • Demonstrations • Links to collaborators • Software releases • Documentation • FAQs

6. Problem Assumptions • Things that Honey will not do • System administration • Capacity planning • Graphic design • Font selection • Site design • Some layout design possible … • Content creation

7. Inputs to Honeydew • E-mail messages with updates to website • Volunteered and solicited information • Minutes from project meetings • Tracking project participants and events • Queries from external sources • Inferred information needs • Click sequences • Publicly visible events, not explicitly provided to webmaster • Conference appearances, news stories, etc.

8. Inputs to Honeydew • Sequences of UI actions performed • Receive e-mail request to add paper to WWW site • Extract title, author, abstract, publication forum, funding agency • Think up file name • Copy attachment to conference paper directory • Update WWW page with info and link • Notify user of change • React to advice from user about change

9. Expected Outputs • Project websites (5 subprojects, 1 project) • Report generation • Overviews of activity over time (e.g., quarterly reports) • Briefing generation • Overviews of current project activities • Question-answering agents • Google-like search, summarization in response to specific queries • Semi-automatic FAQ generation

10. Expected Outputs • Shared knowledge base of learned tasks • Toolkit for rapid construction of new assistants • Assistant Monitoring and Interaction GUI • Requirements for “Assistant Aware Applications” • Stream of papers • Stream of masters and Ph.D. students

11. Missing Capabilities – What • Identify significant webmastering events • Represent webmaster activities through generalizable descriptions • Create consistent and complete task representations • Formulate key clarification dialogs • Adapt to errors in task execution

12. Our Approach • Ethnographic study of Webmasters • WoZ system for domain definition • Human webmasters with Honey observing activities • Information-sharing among EPCAs

13. Our Approach • In-line data labeling by humans • Interactive clarification of human actions • Lightly-supervised learning • Generalizable and sharable representations of activities • Learn by being told • Enthnographic study of Honey users

14. Impact • Relieve the user of routine maintenance tasks associated with web pages • Illustrate portability by using in other WWW task domains • HCII web page • Pittsburgh Post Gazette web page • Workflow systems

15. Impact • Dramatic reduction of human effort in construction and maintenance of web sites • Improve time productivity by 100 to 500% • Little or no loss in quality of site • EPCAs that can learn the skill of cooperatively structuring and managing information • Dramatic reduction in the cost of construction of assistants • Reduce size of backroom knowledge engineers • Assistants become trainers of new Web masters

16. Evaluation: mid-term and finals • Honey performance to be compared with 5 human subjects • 5 other human coaches to be used in providing data and inputs needed for the Honey to learn from Experience • Operational Version 0 in 3 months • Evaluation – 6 months • Honey performs 50% of tasks correctly • Evaluation – 12 months • Elapsed time from e-mail to WWW update improved by 1.3 • Elapsed time to assemble report improved by 1.3 • Honey performs 75% of tasks correctly • Quality of WWW site and reports comparable to human

17. Users Architecture Functional Specification Text Filtering Summarization Extraction Quality Assurance Quality Assurance Evaluation Knowledge Representation Dialog Infrastructure Input

18. Conclusion • Key emphasis • Learning • Coaching • Retargetting • Potential huge impact to Webmaster job • Large amount of shared infrastructure • Many similar problem domains • Many, many research problems

19. Research Issue • Architecture Execution Recommend Learning Matching Summarization KB User GUI Event Stream Coaching GUI Monitor/Do KB GUI E-Mail Server Monitor/Do GUI E-Mail GUI File System Editor Command