1 / 25

Assessment of Value

Assessment of Value. Presented by Ben Hunter and Gretchen Scronce INLS 180: Human Information Interactions October 12, 3004. Images courtesy of http://thesimpsons.com/bios/bios_family_index.htm. Characteristics of Usefulness . Authority Quality Relevance Value. Authority.

richard_edik
Download Presentation

Assessment of Value

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment of Value Presented by Ben Hunter and Gretchen Scronce INLS 180: Human Information Interactions October 12, 3004 Images courtesy of http://thesimpsons.com/bios/bios_family_index.htm

  2. Characteristics of Usefulness • Authority • Quality • Relevance • Value

  3. Authority • According to Amento: “an authoritative document is one that many other documents link to” (297) • Not all links are equally valuable • A good authority is linked to by good hubs and a good hub links to good authorities

  4. Playing the system? • Google bombing • 1. Humor • 2. Ego • 3. Money • 4. Justice • Deterioration of quality

  5. Quality • Subjective • Amento: “The quality of a web site inherently is a matter of human judgement” (296)

  6. Relevance “Relevance refers to the binary state of whether a document is on the same topic as the query or not.” (Glover, E.J., et al)

  7. Value • Based on individual human judgment • Subjective • Determined by user’s information need • Offering search constraints can improve likelihood of value

  8. Does “Authority” Mean Quality? • Brian Amento, Loren Terveen and Will Hill • Published in 2000

  9. Problems faced by Internet users • Finding collections of items relevant to their interests • Identifying high-quality items • Finding items that contain a certain category of information • Creating and maintaining personalized subsets of items

  10. Link-based Algorithms • Do their results correlate with human standards of quality? • Are human standards of quality consistent? Does a shared notion of quality even exist? • Do the different link-based algorithms produce consistent results?

  11. Research Method • 40 subjects from a local university identified sites relevant to specific popular-culture themes • 16 self-identified experts rated the quality of each site • Various link and content-based algorithms used to evaluate sites

  12. Results of the Study • Generally, experts agreed with each other • Search engines were relatively consistent with each other • Experts and link-based algorithms came up with similar results

  13. Surprises for the Researchers • In-degree performed at least as well as more sophisticated methods • Simple page count was almost as good as link-analysis methods

  14. Conclusions • Topic experts make consistent quality judgments • Link-based metrics and a simple content metric both do a good job of identifying high quality items

  15. Problems with Study • Self-selecting, probably homogenous test group (self-identified expert?) • Are broad topics really what people typically search for? • Does pop-culture represent other uses of search engines?

  16. Web Search--Your Way Eric J. Glover, Steve Lawrence, Michael D. Gordon, William P. Birmingham, and C. Lee Giles (2000)

  17. Standard search engine • Searches based on user-generated keyword queries • Ordering policy determines which results are listed first

  18. Metasearch engine • Single interface to multiple search engines • Results are ordered into one list • Metasearches often have difficulty effectively ordering results taken from different search engines--quality is inconsistent

  19. Standard and Metasearch Engines

  20. Inquirus 2 • Source selection based on user preferences: information need categories • Query modification: prepends/appends • Ordering policy: Multi-Attribute Utility Theory

  21. Inquirus 2

  22. Benefits/Drawbacks • User-controlled • May produce results of higher value • Constructing search is more involved for user • Designed for sophisticated searchers--confusing to average Google user?

  23. Conclusion The authors believe that searching features such as those used by Inquirus 2 will enable search engines to produce results of higher value to the user.

  24. Keep in mind… • Not a user study • Inquirus 2 is used by NEC Research Institute, a sophisticated user group • Authors work for NEC • Now at Inquirus 3--searches non-Web based sources

  25. Discussion Questions • Does one characteristic of usefulness seem more important than the others? • Does the popularity of Google, a search engine that uses a link-based metric to assign quality, validate the success of such metrics? Do you think such systems erode over time (e.g. Google bombing) or improve? • How might a typical user react to a complicated search engine like Inquirus 2? • What impact does the continued evolution of the web have on the concepts discussed in these articles?

More Related