Providing Expert Advice by Analogy for On-Line Help

1. Providing Expert Advice by Analogy for On-Line Help • Henry Lieberman and Ashwani Kumar • Media Laboratory • Massachusetts Institute of Technology • Cambridge, MA, USA • http://www.media.mit.edu/~lieber/

2. What happens when interaction with the Web goes wrong? • Increasingly, people use the Web to perform procedures (buy things, vote, date, seek jobs) as well as just browse • What happens when things go wrong? • Debug it yourself • Get help • Increasingly, confidence in Web interactions depends on effectiveness of help

3. How do you get help? • Look up things in problem-solution database • Telephone support • On-line chat

4. Elicititation & Explanation • Elicitation: Helper asks user what is wrong. Get enough information from user to determine problem and choose solution. • Explanation: Helper tells user how to fix the problem. Explain why the solution worked, and how to avoid such problems in the future.

5. Mismatch between expert and novice models • The helper is an expert; the user is a novice • Novice may lack technical vocabulary to understand the elicitation questions • Novice may lack background knowledge to understand explanation of soultion • Expert may be unable to empathize with novice

6. Problem/Solution Database

7. AI Expert Systems • Expert Systems are the traditional way to encode expert behavior • Knowledge Engineer experts to get their knowledge and procedures • Encode Expert Model • Deep but narrow knowledge • If expert and novice don’t share vocabulary and knowledge, it’s difficult for the novice to interact directly with the Expert Model

8. We need a model of novice knowledge • How do we model the knowledge of someone who is not supposed to know very much? • Traditionally, novice is modeled as subset of expert knowledge • Intelligent Tutoring Systems, Qualitative Physics • But how do you model novice knowledge “in general”?

9. Open Mind - Push Singh • Common Sense statements collected from volunteer contributors on the Web • 770,000 English sentences • Conventional data/knowledge bases: Do a lot of organizing work up front, so you can get the stuff out easily • OM: Get the stuff in any which way, then do a lot of work on the back end to interpret it • Query expansion, Parser, Semantic net miner

10. Open Mind - Push Singh

11. How do human helpers succeed in bridging Expert/Novice gap? • We interviewed helpers on the America Online (internet service provider) help desk • Helpers tend to provide “cookbook” solutions • We asked for successful interactions where users came to understand the problem and solution • Helpers best explained things by making analogies to everyday life situations!

12. Problem/Solution Database • Problem: “My browser can see my company’s home page, but it won’t let me access my workgroup’s Web site” • Solution Procedure: “Check if cipher strength is '0’. Upgrade Browser to 128 bit Encryption.” • Explanation: Generally, Websites require 128-bit encryption in order to process information securely. If the cipher strength of your browser is inadequate, you will not get into secure Websites. Upgrading your browser's encryption may help it better handle secure Websites. • NOTE: You only need to do this when unable to get to secure Websites.

13. Better: Explain by Analogy • Encryption in a browser is like security clearance to enter a building. If you don't have the proper security clearance, you may be able to get into the building, but not into certain areas. You must upgrade your security clearance status to go further. So without the proper encryption, your browser may be able to access a website, but not log in.

14. Introducing SuggestDesk • SuggestDesk is an agent that “listens” to the helper/user chat • Provides suggestions to helper (unseen to novice user) • Tries to recognize opportunities for making analogies between problem/solution DB and Common Sense knowledge

15. SuggestDesk

16. Example • User: “My browser runs slowly” • Problem/Solution DB: • Browser might run slowly because of network congestion • Browser might run slowly because the browser is infected by a virus

17. Elicitation & Explanation • Elicitation: • (Traffic) When did you try to log in? • (Virus) Have you downloaded any new applications lately? • Solution: • (Traffic) Try again at a later time. • (Virus) Run an anti-virus program.

18. SuggestDesk’s help (Traffic) • What makes things slow down? • Traffic runs slowly at rush hour. • Analogy: • The AOL service is like a road. • The users are like cars. • If too many users try to use the service at the same time, it slows down. • The solution is to try at a time when things are less crowded or find an alternative route.

19. SuggestDesk’s help (Virus) • What makes things slow down? • People slow down when they are tired. • Being sick can make you tired. • Analogy: • A computer virus is like a biological virus • If you have the flu, you can’t do things as fast as you normally would • An antivirus program is like medicine.

20. SuggestDesk Architecture • The Natural Language Processor (NLP), • The Commonsense Processor (CP), • The Expert Analyzer (EA), • The Analogy Mapping Engine (AME), and • The Elicitation and Explanation Processor (EEP).

21. Natural Language Processing • Via MontyLingua, Common Sense part-of-speech tagger Result= [{prep_phrases_tagged=[], verb_phrases_tagged=[is/VBZ running/VBG], verb_arg_structures_concise=[("run" "browser" "slow")], noun_phrases=[browser], noun_phrases_tagged=[browser/NN], adj_phrases_tagged=[slow/JJ], verb_arg_structures=[[is/VBZ running/VBG, browser/NN, [slow/JJ]]], modifiers_tagged=[slow/JJ], prep_phrases=[], verb_phrases=[is running], parameterized_predicates=[[[run, [past_tense, passive_voice]], [browser, []], [slow, []]]], modifiers=[slow], adj_phrases=[slow]}]

22. ExpertNet (EffectOf 'surf internet' 'download files') (EffectOf 'surf internet' 'download applications') (EffectOf 'download files' 'browser cache is large') (EffectOf 'download applications' 'browser infected by virus') (EffectOf 'PC infected by virus' 'browser run slow')

23. Analogy Mapping Engine Analogies:[[computer, [[UsedFor, surf internet, 1.1887218755408673], [CapableOfReceivingAction, run slow, 1.1887218755408673], [CapableOfReceivingAction, crash, 1.1887218755408673], [CapableOfReceivingAction, start, 1.1887218755408673]], 6.1887218755408675], [car, [[CapableOfReceivingAction, damage, 1.1887218755408673], [CapableOfReceivingAction, crash, 1.1887218755408673], [CapableOfReceivingAction, start, 1.1887218755408673]], 5.930167946706389], [software, [[CapableOfReceivingAction, run slow, 1.1887218755408673], [CapableOfReceivingAction, crash, 1.1887218755408673], [CapableOfReceivingAction, install, 1.1887218755408673], [CapableOfReceivingAction, install, 1.1887218755408673]], 5.855516191543203]]

24. Weighting • log(f+0.5*i+4), • where f=outgoing edges and i = incoming edges

25. SuggestDesk Architecture • The Natural Language Processor (NLP), • The Commonsense Processor (CP), • The Expert Analyzer (EA), • The Analogy Mapping Engine (AME), and • The Elicitation and Explanation Processor (EEP).

26. User Testing

27. Woodstein (with Earl Wagner) • Debugger for Web procedures • By analogy to debugger for programming • Provides visualization, explanation, tools for incremental exploration • Self-help debugging • Co-operative debugging between expert and novice

28. Woodstein

29. Conclusion • The effectiveness of online help is key to the success of Web interactions • Help needs to bridge the gap between expert and novice knowledge • Common Sense Reasoning can help find analogies that allow expert and novice to communicate • We need debuggers that help us systematically explore the causes of problems • Don’t worry, help is on the way!