Intentional Query Suggestion: Making User Goals More Explicit During Search

1. Intentional Query Suggestion:Making User Goals More Explicit During Search Markus Strohmaier, Mark Kröll and Christian Körner WSCD‘09: Workshop on Web Search Click Data @ WSDM 2009 Barcelona, Spain Mark Kröll mkroell@tugraz.at Graz University of Technology, Austria

2. Related Work • on Query Suggestion • Query expansion using local and global documentanalysis, [Xu96]. • Generating query substitutions, [Jones06]. • Learning latent semantic relations from clickthrough data for query suggestions, [Ma08]. • on Search Intent • Understanding goals in web search, [Broder02, Rose/Levinson04] . • The Intention Behind Web Queries, [BaezaYates06]. • Understanding the Relationship between Searchers’ Queries and Information Goals, [Downey08]. Observation: There is little work that combines these two areas. • Goal Oriented Search Engine, [Liu02]. • Effects of Goal-Oriented Search Suggestions, [Mostert08].

3. Intentional Query Suggestion • Query Suggestion based on User Intent • suggesting queries that represent potential user intentions Definition of explicit intentional queries (suggested queries): • contain at least one verb and • describe a plausible state of affairs that the user may want to achieve or avoid (cf.) in • a recognizable way

4. Research Overview • How could query suggestion based on user intent be realized? • How would queries expanded by user intent influence • Click-Through? • Search results?

5. Algorithm for Intentional Query Suggestion • by creating a mapping between implicit intentional queries(short) and explicit intentional queries (long) • Output: list of suggestions • Example: • design your own poker chips • learn to play home poker • cheat at poker • buy poker chips • Input: • Example: • poker Potential Intentions

6. Text-based Intentional Query Suggestion • dataset containing explicit intentional queries • extracted from MSN log employing the algorithm from [Strohmaier08a] • implicit intentional queries are textually compared to all explicit intentional queries • used Jaccard Similarity Measure [BaezaYates99]

7. Neighborhood – based Intentional Query Suggestion • based on query log data we construct a bipartite graph, where • nodes of one type correspond to explicit intentional queries and • nodes of the second type correspond to implicitintentional queries not containing goals not containing goals containinggoals containing goals containing goals not containing goals not containing goals use neighboring queries to further describe and characterize explicit intentional queries

8. Parametric model Pd=3 • Distance Pd • size of neighborhood N Pd=3 qe,1 • T(qe,1) = {“weight”, “loss”, “supplement”, “upplements”} T(qe,1) Excerpt of the corresponding Bipartite Graph

9. Experimental Demonstrator [joint work with student Ferdinand Wörister]

10. Preliminary Evaluation of Algorithm • by conducting a human subject study • categorize the 10 top-ranked suggestions for 30 queries • Two relevance classes: • (i) potential user intention • (ii) clear misinterpretation Average Precision: 0.71 Average Interrater Agreement Kappa: 0.6

11. Step Back • presented one approach and experimental demonstrator to realize query suggestion based on user intent • reasonable precision of 71% concerning quality of suggested queries • How would queries expanded by user intent influence click through and search results?

12. Influence on Search Results(1) • Result Set Intersection between different Query Suggestion Mechanisms: • How many URLs intersect between URL result sets? 10 Traditional Suggestions Original Query 10 Intentional Suggestions 50 50 50 50 50 50 50 50 50 50 50 50 500 URL result sets (unique, top-level) 50 50 50 50 50 50 50 50

13. Influence on Search Results(2) • Result Set Intersection within the same Query Suggestion Mechanism: • How many URLs intersect between result sets that were retrieved by the same query suggestion mechanism regarding one original query? 10 Traditional Suggestions 10 Intentional Suggestions . . . . . . 50 50 50 50 50 50 . . . . . . • Results suggest that • queries that express a specific intention lead to more different results than traditional query suggestions

14. Influence on Click-Through • Do users click more frequently on suggested queries if they are explicit intentional? • Experimental Setup: • click-through events for different query lengths • created different bin sizes – 5000 queries randomly drawn from MSN log • corresponding click-through events were registered and counted • Results suggest that • queries that express a specific intention retrieve more relevant results than implicit intentional queries of the same length

15. Conclusions • combiningSearchIntent + Query Suggestion • usinghighlevelsearchintent [Mostert08] • employingsearchintent on a moredetailedlevelappearstobe a naturalnextstep • impact on searchresultsandbehavior (preliminaryexperiments) resultssuggesthigherclickthroughfor explicit intentional queries resultssuggestmore diverse resultsfor explicit intentional queries

16. References [BaezaYates99] Baeza-Yates R. and Ribeiro-Neto B. Modern Information Retrieval, AddisonWesley, 1999 [BaezaYates06] Baeza-Yates, R., Calderón-Benavides, L. and González-Caro, C. 2006. The Intention Behind Web Queries, in F. Crestani, P. Ferragina and M. Sanderson, ed.,Proceedings of String Processing and Information Retrieval (SPIRE ), Springer, 98-109. [Bendersky09] Bendersky, M. and Croft, W. B. , "Analysis of Long Queries in a Large Scale Search Log," Workshop on Web Search Click Data (WSCD 2009) Barcelona, Spain, February 9, 2009. [Broder02] Broder A. A taxonomy of web search. In ACM SIGIR Forum 36(2), pp. 3--10, 2002. [Downey08] Downey, D.; Dumais, S.; Liebling, D. & Horvitz, E. (2008), Understandingtherelationshipbetweensearchers' queries and informationgoals, in 'CIKM '08: Proceeding of the 17th ACM conference on Information and knowledgemining', ACM, New York, NY, USA, pp. 449--458. [Dumais98] Dumais S., Platt J., Heckerman D., and Sahami M. "Inductive learning algorithms and representations for text categorization". In: Proceedings International Conference on Information and Knowledge Management, New York, NY, USA, ACM Press, pp 148-155, 1998 [Jansen08] Jansen, B. J., Booth, D. L. and Spink, A. Determining the informational, navigational, and transactional intent of web queries. In Inf. Process. Manage. 44(3), pp. 1251--1266, 2008. [Jones06] Jones, R.; Rey, B.; Madani, O. & Greiner, W. (2006), Generatingquerysubstitutions, in 'WWW '06: Proceedings of the 15th international conference on World Wide Web', ACM, New York, NY, USA, pp. 387--396. [He07] K.Y. He and Y.S. Chang and W.H. Lu. Improving Identification of Latent User Goals through Search-Result Snippet Classification. WI '07: Proceedings of the 2007 IEEE/WIC/ACM International Conference on Web Intelligence, 683-686, IEEE Computer Society,2007. [Ma08] Ma, H.; Yang, H.; King, I. & Lyu, M. R. (2008), Learning latent semantic relations fromclickthroughdataforquerysuggestion, in 'CIKM '08: Proceeding of the 17th ACM conference on Information and knowledgemining', ACM, New York, NY, USA, pp. 709--718. [Mostert08] Mostert, J. & Hollink, V. (2008), Effects of Goal-OrientedSearchSuggestions, in 'Proceedings of the 20th Belgian-NetherlandsConference on ArtificialIntelligence'. [Liu and Lieberman02] Liu H. , Lieberman H. and Selker T.. GOOSE: A Goal-Oriented Search Engine with Commonsense. AH '02: Proceedings of the Second International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, 253--263, Springer-Verlag,London, UK,2002. [Rose04] Rose D. E. and Levinson D., Understanding user goals in web search. In Proc. of WWW 2004, May 17-22, 2004, New York, USA, 2004. [Strohmaier08a] Strohmaier, M., Prettenhofer, P. and Kröll, M. Acquiring explicit user goals from search query logs. In 'International Workshop on Agents and Data Mining Interaction ADMI‘ 08, in conjunction with WI '08', 2008. [Strohmaier08b] M. Strohmaier, P. Prettenhofer, M. Lux, Different Degrees of Explicitness in Intentional Artifacts - Studying User Goals in a Large Search Query Log, CSKGOI'08 International Workshop on Commonsense Knowledge and Goal Oriented Interfaces, in conjunction with IUI'08, Canary Islands, Spain, 2008. [Xu96] Xu, J. & Croft, W. B. (1996), Query expansionusinglocal and global documentanalysis, in 'SIGIR '96: Proceedings of the 19th annual international ACM SIGIR conference on Research and development in informationretrieval', ACM, New York, NY, USA, pp. 4--11.

17. End of Presentation Mark Kröll Graz University of Technology, Austria mkroell@tugraz.at Thanks for your attention!

18. Questions and Discussion