1 / 30

GoogleBuddy: Towards a Collaborative System for Learning and Sharing Search Knowledge

GoogleBuddy: Towards a Collaborative System for Learning and Sharing Search Knowledge. Suresh K. Bhavnani School of Information University of Michigan GoogleBuddy Team Andrew Paulsen (SI) Mark LaRosa (SI) Robert Thompson (SI) Adam Ward (EECS) Warner Washington (Art and Design)

prema
Download Presentation

GoogleBuddy: Towards a Collaborative System for Learning and Sharing Search Knowledge

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GoogleBuddy: Towards a Collaborative System for Learning and Sharing Search Knowledge Suresh K. Bhavnani School of Information University of Michigan GoogleBuddy Team Andrew Paulsen (SI) Mark LaRosa (SI) Robert Thompson (SI) Adam Ward (EECS) Warner Washington (Art and Design) Dino Anastasia (MSI, CAEN) This research is supported in part by a GROCS Award

  2. Overview • Motivation to develop GoogleBuddy • Learning and Sharing of Search Knowledge • Design of GoogleBuddy • Learning Interface • Sharing Interface • Early evaluation results of Learning Interface • Discussion of Issues About Sharing Interface • Future Research

  3. Motivation • Large numbers of users access the web for information in different domains • Estimated half of all Americans adults have searched the Web for healthcare information (Fox & Rainie, 2003) • Tasks include to become informed, to preparation for appointments and for surgery, and to share information • Explosion in the number and size of sources for information • National Cancer Institute (NCI) site has information for about 118 cancers • Hundreds of vendors selling electronic products • Given such vast resources, and search engines, one might expect that a user can quickly get comprehensive information about a topic

  4. Motivation Counter to evidence of actual search behavior • Users can often find information for factual searches (e.g.Wallace et al., 2000, Bilal, 2001) • What is a melanoma? • Users have difficulty finding information for comprehensive searches (e.g. Fidel et al., 1999, Bhavnani, 2003) • What are the risk/prevention factors for melanoma? • Who should or should not get a flu shot and why? • Find a low price for reliable digital camera with 5 Megapixel (resolution) and Optical Zoom greater than 2X

  5. Example of an Expert Healthcare Searcher Experiment that compared expert healthcare searchers to novices (Bhavnani, 2001) • Search task: Tell me three categories of people who should, or should not get a flu shot and why? • Expert search procedure • Find a reliable source for flu shot information •  MEDLINEplus • 2. Find flu shot information in reliable source •  adam.com • 3. Verify flu shot information in a pharmaceutical company •  wyeth.com • Outcome • - Visited 3 reliable healthcare sites (as listed in CAPHIS) • - Total task time = 7 minutes • - Accessed a comprehensive list of 9 categories of people who should get a flu shot, and 5 categories who should not

  6. Example of a Novice Healthcare Searcher Search task: Tell me three categories of people who should, or should not get a flu shot and why? • Novice search procedure • Find flu shot information through Google •  Query=“who should or should not receive flu shots” • 2. Search through hit list in the sequence provided •  Visited 13 sites • Outcome • - Visited 13 sites, none of which were listed by CAPHIS • - Total task time = 20 minutes • - Completed the task by retrieving bits and pieces of information from many unreliable sites

  7. Why is it Difficult to Find Comprehensive Information About a Topic? 1. Information Scatter • No single webpage or website appears to have all the information about a topic • Distributions of facts about a topic follow highly skewed distributions (Bhavnani, 2005a, Bhavnani, 2005b) y = 33.308e-0.0276x LR=20.98 p=.051

  8. Why is it Difficult to Find Comprehensive Information About a Topic? 2. Information Density • Webpages about a topic appear to have different densities of information (Bhavnani, 2005a, 2005b) • General, Specific, Sparse pages 3. Information Specialization • Websites fall into different genres (Crowston & Williams, 1997) • Shopping: Product Review, Price Comparison, Coupon • Healthcare: Ask-a-doc, Calculator pages

  9. Expert Search Knowledge • Experts appear to have developed domain-specific search procedures to address emergent regularities in information • Shopping • 1. Find high-quality products from review sites • 2. Find low price from price comparison sites • 3. Find discounts from coupon and bargain sites • Healthcare • 1. Find overview information in general site • 2. Verify information in specific site • Writing a paper • 1. Find important references by visiting a general source like an encyclopedia • 2. Find key concepts about different topics within references • 3. Use key concepts to search in a specific database (HCI bibliography)

  10. Attempts to Provide Search Assistance • Implicit Feedback • Query reformulation (Gauch & Smith, 1993) • Suggest adding “and”, “or” when returned set is very large • Query refinement (Spink et al., 2001) • suggest synonyms from a thesaurus • Relevance feedback (Harman, 1992) • search based on terms from a document considered relevant by a user • Similar queries (Anick, 2003) • suggests similar queries by previous users • Spelling (Jansen et al., 2000) • suggest spelling corrections • Pathfinders • Textual descriptions of how to do searches for different topics (Canfield, 1972, e.g. IPL Pathfinders) • Very little research on online learning and sharing of domain-specific search procedures

  11. Early Attempt to Provide Search Procedures Select a link in a step of the search procedure Page is displayed in a new window

  12. Strategy Hub Effectiveness • Controlled experiment comparing Strategy Hub to Google and MEDLINEplus • Strategy Hub enables novice searchers to find more accurate and comprehensive information, when compared to conventional search tools, in the same amount of time (Bhavnani et al., 2006) • Limitations of the Strategy Hub • Does not teach user how to search on their own • Does not easily scale up • Does not address rapidly changing search knowledge

  13. GoogleBuddy • Goal: Develop and evaluate a learning and sharing system for search knowledge in specific domains • Learning Interface (current system) • Learning by observation of overall steps in a search strategy • Learning by doing by interacting directly with Google • Sharing Interface (future system) • Members can add and modify search strategies • All users can vote on alternate search strategies

  14. Design of GoogleBuddy

  15. Design of GoogleBuddy

  16. Design of GoogleBuddy

  17. Design of GoogleBuddy

  18. Design of GoogleBuddy

  19. Demo of GoogleBuddy

  20. Early Interface Evaluation Results • Heuristic Evaluation without users (Nielsen, 1993) • Four rapid user evaluations with design iterations • 2-6 users • Identification of problems and refinement of design • Example Task Your friend wishes to buy a NEW, reliable digital camera at the lowest price you can find with the following criteria: • 5 Megapixel (resolution) • Optical Zoom greater than 2X Please use GoogleBuddy to find a price quote for a digital camera that meets the above criteria and that you would recommend. • Open-ended interview

  21. Study-1 • Observed interactions and interview • 1/3 users did not use keywords in drop down box • User hit return quickly causing the drop down to not appear • User thought that GB would automatically search for each step • Possible cause • Delay too long in drop down box • Instruction about drop down box was not explicit • Redesign • Delay shortened to show drop down box while typing • More text and explanation under “Guide Me”

  22. Study-2 • Observed interactions and interview • 3/6 users did not use drop down box • Users did not notice drop down box – thought it was the history • Possible cause • Drop down box looked too much like history box • Instruction from “Guide Me” not heard • Redesign • Changed design of drop down box to look like GB • Bolded keywords • Added error message if keyword was not chosen

  23. Study-3 • Observed interactions • 2/2 users used the drop down box • 1 user did not come back to use other steps • Possible cause • GoogleBuddy perceived as a search tool, not a learning tool • Redesign • No interface change • Tested a short oral introduction in the experiment “GoogleBuddy is a teaching tool, which provides you a step-by-step strategy to make better use of Google.”

  24. Study-4 • Observed interactions • 2/2 users used the drop down box • 1 user did not come back to use other steps • Possible cause • GoogleBuddy perceived as a search tool, not a learning tool • Redesign • Future design will have an introduction the system “GoogleBuddy is a system to teach you how to search …”

  25. Future Research: Experiments • Evalulate efficacy of search guided by GoogleBuddy • Comparison with users just using Google • Evaluate learning • Transfer of knowledge to just using Google • Retention of knowledge over time • Evaluate affective variables • Engagement, satisfaction, motivation, etc.

  26. Future Research: Sharing Interface • Motivation • There exist domain experts who do have strategies • We are not aware of existing online mechanisms to enable experts to share and refine search strategies • Goals • Provide domain experts with a collaborative environment to share and refine strategies • Allow all users to evaluate strategy effectiveness • Ensure the best strategy is provided on the Learning Interface

  27. GoogleBuddy Sharing Interface Current Strategy Edit Steps Keywords Visit a product review site review, consumer review Visit a price comparison site Price comparison, Visit a site with coupons Discount, coupon Comments Make sure you do not get side-tracked by prices and coupons on review sites, and by coupons on price comparison sites … Visitor (36) **** Expert (24) ***** Strategy #2 Edit Steps Keywords Visit a product review site review, consumer review Some Initial Ideas: Sharing Interface Modify Strategy Rate this Strategy ***** Login

  28. Architecture of the Sharing Interface

  29. Some Issues Related to Sharing Interface • Should strategies be identified by experts, or synthesized through a forum, or both? • What are the criteria to include domain experts? • Should strategies have visible owners? • Should strategies be approved and voted by other experts before being posted? • If a strategy is modified, what happens to its ratings? (fresh start, or some form of inheritance) • What motivates a change in strategy? (Poor ratings, changes on the web, etc.)

  30. Summary • Need for a learning and sharing search knowledge environment • Development and testing of the GoogleBuddy Learning Interface • Future Research • Experiments related to efficacy and learning • Design and development of the Sharing Interface

More Related