Predicting Short-Term Interests Using Activity-Based Search Context

1. Predicting Short-Term Interests Using Activity-Based Search Context CIKM’10 Advisor: Jia Ling, Koh Speaker: Yu Cheng, Hsieh

2. Outline • Introduction • Modeling Search Activity • Study • Conclusions

3. Introduction • Satisfying searchers’ information needs involves a through understanding of their interests through: - search query - search engine result page (SERP) clicks - post-SERP browsing behavior • Construct interest models of the current query which including: - previous queries - previous clicks on SERP • Evaluate the predictive effectiveness of these models using future actions

4. Modeling Search Activity • Data - The data set contained browser logs with both searching and browsing episodes. - Log entries include a timestamp for each page view, and the URL of the Web page visited - Only in English-speaking United States locale - Search sessions on the Bing Web search engine were extracted

5. Modeling Search Activity • ODP Labeling - Represented context a distribution across categories in ODP topical hierarchy. - Provides a consistent topical representation of queries and page visits from which to build the models. - ODP category label can also reflect topical differences in the search results for a query or a user’s interests - Automatic classification skill to assign an ODP category labels to each page. - 219 categories at the top two levels of the ODP hierarchy were used ( called L) -

6. Modeling Search Activity • ODP Labeling - Strategy of labeling a page 1. Begin with URLs present in the ODP 2. Incrementally prunes non-present URLs until a match is found, or miss declared 3. Check for exact match with logistic regression classifier

7. Modeling Search Activity • Sources and Source Combinations - ODP labels automatically assigned to the following sources: 1. Query: the top 10 search results for the query 2. SERPClick: the search results clicked by the user during the search session 3. NavTrai: Web pages that the user visits from a SERP click

8. Modeling Search Activity • Model Definitions– Query Model(Q) - For each query, the category labels for the top 10 search results were obtained. - Probabilities are assigned to the categories in L by 1. normalized click frequencies for each top 10 results from search-engine click log data 2. the distribution across all ODP category labels - ODP categories in L that are not used to label are assigned the prior probabilities

9. Modeling Search Activity • Model Definitions– Context Model(X) - The context model is constructed based on actions which comprise previous data as follows: 1. Queries 2. Web pages visited through a SERP click 3. Web pages visited on the navigational trail following a SERP click

10. Modeling Search Activity

11. Modeling Search Activity • Model Definition – Intent Model(I)

12. Modeling Search Activity • Relevance Model or Ground Truth (R) - The relevance model contains actions that occur following the current query in the session

13. Modeling Search Activity

14. Study

15. Study

16. Study

17. Study

18. Study • Learning Optimal Context Weights Steps 1. Identify the optimal context weight (w) for each query on a held out training set 2. Create features for the query and the context that could be useful in predicting w

19. Study • Learning Optimal Context Weights - To create a training set, the query, context, and relevance models were used to compute the optimal context weight per query by minimizing the regularized cross-entropy for each query independently.

20. Study A regularizer that penalizes deviations from w=0.5

21. Study • Generating Features of Query and Context - Divide features into three classes: 1. Query class: capturing characteristics of the current query and the query model. 2. Context class: capturing aspects of the pre-query interaction behavior as well as features of the context model themselves. 3. QueryContext: capturing aspects of how the query model and context model compare. - These features were generated for each session in the set and used to train a predictive model

22. Study • Generating Features of Query and Context - Query class

23. Study • Generating Features of Query and Context - Context class

24. Study • Generating Features of Query and Context - QueryContext class

25. study

26. study • Predicting the Optimal Context Weight - 60% of those queries for training, 20%for validation, 20% for testing - 10-fold cross validation was performed to improve result reliability. - The folds were constructed by splitting session, so that all queries in a session are used for either training, validation, or testing

27. study

28. study • Predicting the Optimal Context Weight The most performant features related to the information divergence to the query models and the context model

29. study • Predicting the Optimal Context Weight

30. study

31. study • Varying Context and Relevance Information

32. Conclusions • A study of investigating the effectiveness of activity-based context in predicting user’s search interests. • Explored the value of modeling the current query, its context and their combination, and different sources. • Intent models developed from many sources perform best overall. • Developed techniques to learn the optimal combinations.