Hao Wu Nov. 18 2014

1. Hao Wu Nov. 18 2014

2. Outline • Introduction • Related Work • Experiment • Methods • Results • Conclusions & Next Steps

3. Introduction • Every search engine looks alike.

4. Why eye tracking in information retrieval? • Understand how searchers evaluate online search results • Enhanced interface design • More accurate interpretation of implicit feedback (eg, clickthrough data) • More targeted metrics for evaluating retrieval performance

5. Background

6. Outline • Introduction • Related Work • Experiment • Methods • Results • Conclusions & Next Steps

7. Related works • Methods: search-engine log files; diary studies; eye tracking and detailed activity-logging • User Interface: “Faceted browsing” interfaces; dynamically categorize search results; dynamic filtering and visualization • Eye-tracking methodologies: develop different user models • Combine with clickthrough data • Scanning order in search result • Pattern of fixations(scanpaths) • Gender difference

8. Key research questions • Do people look at the same number of search results for different task types? • Do they attend to different components of search results for navigational and informational tasks? • Does the inclusion of more contextual information in search results help with informational tasks? Task Type * Snippet Length

9. Outline • Introduction • Related Work • Experiment • Methods • Results • Conclusions & Next Steps

10. Methods • Apparatus • MSN Search as search server • Tobiix50 eye-tracker • Participants • 18 participants have complete data • Age: 18 to 50, 11 male, 7 female • At least search web once per week • Experimental design and procedure • 12 search tasks (6 different tasks for each type) • 3 type of snippet length • Data collection • Gaze fixations >= 100 ms in AOIs and its sub elements • Non-gaze-related behavioral measures • Total time on task • Click accuracy

11. Examples

12. Outline • Introduction • Related Work • Experiment • Methods • Results • Conclusions & Next Steps

13. Overall searching behavior 1 Linear order • Attention vs. Ranking?

14. Overall searching behavior 2 • How many other items above and below the selected document did users look at?

15. Overall searching behavior 3 • Hub- spoke pattern • Does fixation time on each document change with subsequent visit to the first page?

16. Task Type & Snippet Length • Measures: • Repeated Measures Multivariate Analysis of Variance • 2 (Task Type) x 3 (Snippet Length) x 2 (Repetition) • Main effect test • Task type • Repetition • Snippet length • Interaction of Task Type & Snippet Length  significant  Not significant  significant

17. Mean time on task • How much time spend on each task when varied snippet length?

18. Click accuracy • How accurate are they when selecting ‘best result’ on first query page?

19. Total results fixated • Opposite pattern between navigational and informational task when varies length from medium to long.

20. Proportion of total fixation duration • How users distribute their attention to different elements?

21. Outline • Introduction • Related Work • Experiment • Methods • Results • Conclusions & Next Steps

22. Conclusions Problem: How varying the amount of information will affect user performance • Adding information to the contextual snippet • Increase in performance for informational tasks • Decrease in performance for navigational tasks • Snippet length increased • More attention to the snippet • Less attention to the URL Snippet length is a dilemma

23. Future direction • UI for information retrieval • Verify whether or not moving URL above the snippet? • Other types of meta data?