Tailoring Click Models to User Goals Fan Guo , Lei Li, Christos Faloutsos School of Computer Science Carnegie Mellon Un

1. LOGO Tailoring Click Models to User GoalsFan Guo, Lei Li, Christos FaloutsosSchool of Computer Science Carnegie Mellon University LOGO OVERVIEW RESULTS • Click Models • A principled way of understanding user interaction with web search results in a query session. • A statistical tool for leveraging search engine click logs to analyze and improve user experience • Input: click logs (query + document impression + clicks) Output: document relevance + user behavior parameters User Goals • Navigational: to find the link to an existing website, e.g., google. • Informational: more exploration, multiple clicks may arise, e.g., iron man. Fitting Multiple Click Models • Different user goals result in different browsing and click patterns. • The straightforward mixture-modeling approach is not practical. • Solution: (1) classify queries based on user goals; (2) fitting two models for navigational and informational queries. • Data Preparation • Query sessions with top 10 documents listed and at least one click. • Divided evenly according to time-stamp for training and test sets. • Query frequency has to be at least 3 in both sets. • A total of 121,179 distinct queries and 2.7 million query sessions. User Goal Classification • MeanClk/MedClk: mean and median of the click distribution. • AvgClk: the average number of clicks per query session. • Medtime: the median of time (in second) spent before the first click. Evaluation Criteria • LL (log-likelihood): the chance to exactly recover the test click sequence. • Perplexity: average prediction quality for each click independently. DEPENDENT CLICK MODEL SEARCH RELEVANCE SCORE ALGORITHMS CONCLUSION • We fit different click models for navigational queries and informational queries and obtain significant performance improvement. • We provide a quantitative comparison of user behavior using summary statistics derived from click models. • We propose the search relevance score to evaluate search engine performance at different granularity levels. ACKNOWLEDGEMENT • This study was inspired by an email communication with Nick Craswell. We would like to thank Chao Liu for the discussion that also helped to motivate this work.