1 / 53

Finding and Re-Finding Through Personalization

Finding and Re-Finding Through Personalization. Jaime Teevan MIT, CSAIL. David Karger (advisor), Mark Ackerman, Sue Dumais, Rob Miller (committee), Eytan Adar, Christine Alvarado, Eric Horvitz, Rosie Jones, and Michael Potts. Thesis Overview. Supporting Finding How people find

papina
Download Presentation

Finding and Re-Finding Through Personalization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Finding and Re-Finding Through Personalization Jaime Teevan MIT, CSAIL David Karger (advisor), Mark Ackerman, Sue Dumais, Rob Miller (committee), Eytan Adar, Christine Alvarado, Eric Horvitz, Rosie Jones, and Michael Potts

  2. Thesis Overview • Supporting Finding • How people find • Individual differences affect finding • Personalized finding tool • Supporting Re-Finding • How people re-find • Finding and re-finding conflict • Personalized finding and re-finding tool

  3. Old New

  4. Thesis Overview • Supporting Finding • How people find • How individuals find • Personalized finding tool • Supporting Re-Finding • How people re-find • Finding and re-finding conflict • Personalized finding and re-finding tool

  5. Supporting Re-Finding • How people re-find • People repeat searches • Look for old and new • Finding and re-finding conflict • Result changes cause problems • Personalized finding and re-finding tool • Identify what is memorable • Merge in new information

  6. Supporting Re-Finding • How people find • People repeat searches • Look for old and new • Finding and re-finding conflict • Result changes cause problems • Personalized finding and re-finding tool • Identify what is memorable • Merge in new information Query log analysis Memorability study Re:Search Engine

  7. Related Work • How people re-find • Know a lot of meta-information [Dumais] • Follow known paths [Capra] • Changes cause problems re-finding • Dynamic menus [Shneiderman] • Dynamic search result lists [White] • Relevance relative to expectation [Joachims]

  8. Query Log Analysis • Previous log analysis studies • People re-visit Web pages [Greenberg] • Query logs: Sessions [Jones] • Yahoo! log analysis • 114 people over the course of a year • 13,060 queries and their clicks • Can we identify re-finding behavior? • What happens when results change?

  9. Re-Finding Common Unique click Repeat click 40% 86% of queries of queries Repeat query 33% 26% of queries of queries 87% 38% of repeat queries of repeat queries

  10. Change Reduces Re-Finding • Results change rank • Change reduces probability of repeat click • No rank change: 88% chance • Rank change: 53% chance • Why? • Gone? • Not seen? • New results are better?

  11. Change Slows Re-Finding • Look at time to click as proxy for Ease • Rank change  slower repeat click • Compared with initial search to click • No rank change: Re-click is faster • Rank change: Re-click is slower • Changes interfere with re-finding ? 

  12. Old New

  13. “Pick a card, any card.”

  14. Abracadabra! Case 1Case 2Case 3Case 4Case 5Case 6

  15. Your Card is GONE!

  16. People Forget a Lot

  17. Change Blindness

  18. Change Blindness

  19. Old New

  20. We still need magic!

  21. Memorability Study • Participants issued self-selected query • After an hour, asked to fill out a survey • 129 people remembered something

  22. Memorability a Function of Rank

  23. Remembered Results Ranked High

  24. Old New

  25. Re:Search Engine Architecture Search engine result list query 1 query 2 … query n result list 1 result list 2 … result list n Index of past queries score 1 score 2 … score n Result cache Merge query result list User interaction cache Web browser User client

  26. Components of Re:Search Engine query 1 query 2 … query n query Index of past queries score 1 score 2 … score n • Index of Past Queries • Result Cache • User Interaction Cache • Merge Algorithm result list 1 result list 2 … result list n query 1 query 2 … query n Result cache User interaction cache result list result list 1 result list 2 … result list n Merge result list

  27. Index of Past Queries query 1 query 2 … query n query Index of past queries score 1 score 2 … score n • Studied how queries differ • Log analysis • Survey of how people remember queries • Unimportant: case, stop words, word order • Likelihood of re-finding deceases with time • Get the user to tell us if they are re-finding • Encourage recognition, not recall

  28. Merge Algorithm result list result list 1 result list 2 … result list n Merge result list • Benefit of New Information score • How likely new result is to be useful… • …In a particular rank • Memorabilityscore • How likely old result is to be remembered… • …In a particular rank • Chose list maximizes memorability and benefit of new information

  29. Benefit of New Information • Ideal: Use search engine score • Approximation: Use rank • Results that are ranked higher are more likely to be seen • Greatest benefit given to highly ranked results being ranked highly

  30. Memorability Score • How memorable is a result? • How likely is it to be remembered at a particular rank?

  31. Choose Best Possible List • Consider every combination • Include at least three old and three new • Min-cost network flow problem New b1 … b2 7 10 t b10 s 10 … m1 … 7 m2 … Slots m10 Old

  32. Old New

  33. Evaluation • Does merged list look unchanged? • List recognition study • Does merging make re-finding easier? • List interaction study • Is search experience improved overall? • Longitudinal study

  34. List Interaction Study • 42 participants • Two sessions a day apart • 12 tasks each session • Tasks based on queries • Queries selected based on log analysis • Session 1 • Session 2 • Re-finding • New-finding (“stomach flu”) (“Symptoms of stomach flu?”) (“Symptoms of stomach flu?”) (“What to expect at the ER?”)

  35. List Interaction Study

  36. Old New Experimental Conditions • Six re-finding tasks • Original result list • Dumb merging • Intelligent merging • Six new-finding tasks • New result list • Dumb merging • Intelligent merging New 1 New 2 New 3 New 4 New 5 New 6 Old 5 New 1 Old 1 Old 7 New 2 New 3 New 4 Old 4 New 5 New 6

  37. Old New Experimental Conditions • Six re-finding tasks • Original result list • Dumb merging • Intelligent merging • Six new-finding tasks • New result list • Dumb merging • Intelligent merging Old 1 Old 2 Old 4 New 1 New 2 New 3 New 4 New 5 New 6 Old 10 Old 1 Old 2 Old 4 Old 10

  38. Measures • Performance • Correct • Time • Subjective • Task difficulty • Result quality

  39. Experimental Conditions • Six re-finding tasks • Original result list • Dumb merging • Intelligent merging • Six new-finding tasks • New result list • Dumb merging • Intelligent merging Faster, fewer clicks, more correct answers, and easier! Similar to Session 1

  40. Results: Re-Finding 99% 88% 38.7 70.9 45.6

  41. Results: Re-Finding 1.79 1.53

  42. Results: Re-Finding Similarity 76% 60% 76% • Intelligent merging better than Dumb • Almost as good as the Original list

  43. Results: New-Finding 153.8 120.5

  44. Results: New-Finding 3.38 2.94

  45. Results: New-Finding Similarity 38% 50% 61% • Knowledge re-use can help • No difference between New and Intelligent

  46. Results: Summary • Re-finding • Intelligent merging better than Dumb • Almost as good as the Original list • New-finding • Knowledge re-use can help • No difference between New and Intelligent • Intelligent merging best of both worlds

  47. Conclusion • How people re-find • People repeat searches • Look for old and new • Finding and re-finding conflict • Result changes cause problems • Personalized finding and re-finding tool • Identify what is memorable • Merge in new information

More Related