1 / 26

Web Research - Large-Scale Web Data Analysis

This research project focuses on large-scale web data analysis from 1997-2007, using transaction logs and search engine data. The goal is to gain insights, understand user behavior, and improve interface/system design and user training.

jbergman
Download Presentation

Web Research - Large-Scale Web Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Web Research - Large-Scale Web Data Analysis Amanda Spink Queensland University of Technology Jim Jansen The Pennsylvania State University

  2. Web Data Analysis 1997-2007 • Track Web search trends and characteristics • Web query transaction logs collected in 1997,1999, 2001, 2003, 2004, 2005 and 2006. • Combined dataset of 20 million+ Web searches

  3. Web Search Studies • Web search engines: - Alta Vista - Ask Jeeves - Excite - AlltheWeb - Vivisimo - Dogpile • Transaction log analysis studies • Focus on user search analysis for competitive advantage

  4. Web Data Sample

  5. Data Collection Methods • Various combinations of methods and approaches • Transaction log analysis • Videotaping and Audio-taping • Think aloud protocols • Usability – HCI techniques • Focus groups • Interviews • Survey • Experiments • Diaries

  6. Data Analysis Methods • Quantitative and statistical analysis • Qualitative analysis – grounded theory • Combination of both methods

  7. Key Issues – Search Studies What is the goal of the project? • Insights, understanding and develop theory • User modeling • Trends analysis • Interface/systems design • User training

  8. Key Issues – Search Studies • What variables to measure? • How much data is enough? • Methods used – single or multiple? • HCI approach – test interface/system features

  9. Transaction Log Analysis (TLA) • File or log of communications between user and system • File recorded on a server – side recordings • Log or file formats vary but there are fields common to most (e.g., IP address, cookie, time stamp, query, vertical, click thru)

  10. Why Collect and Analyze Log Data? • Gain understanding of user interaction with system and interface • Goal to improve system and interface design, and improve user training. • Transaction log analysis is extensively used in academia and industry

  11. TLA Process • Goals and objectives • Data collection • Log preparation • Data analysis • Making sense

  12. Data Collected • Process of collecting the interaction data for a given period in a transaction log • Collect data on the search episode • User identification • Date • Time • Search session content • Resources accessed (e.g., URL’s)

  13. Logging Software • Custom and commercial applications (the Wrapper - http://ist.psu.edu/faculty_pages/jjansen/academic/wrapper.htm) • WinWhatWhere spy software • Morea 1.1 software • Camtasia Studio

  14. Data Preparation • Process of cleaning and preparing the log data for analysis • Log data into a relational database • Cleaning the log – corrupted data • Parsing the log (e.g., removing Web sessions identified as agents) • Normalizing the log

  15. Log Analysis – Three Levels • Term • Query • Session

  16. Term occurrence Total terms High and low usage terms Term distribution Co-occurring terms Term Level Analysis

  17. Query Level Analysis • Initial query • Subsequent queries • Modified queries and query reformulation • Identical queries • Query complexity • Boolean use • Spelling • Types of queries • Query topics

  18. 1. People/Places 49.2% 2. Commerce, etc. 12.5% 3. Computers, etc. 12.4% 4. Health/sciences 7.4% 5. Education/Humanities 5% 6. Entertainment, etc. 4.5% 7. Sex/Pornography 3.2% 8. Society/Culture, etc. 3.1% 9. Government 1.5% 10. Performing/Fine Arts 0.6% 1. Commerce, etc. 21% 2. Indiscernible 19% 3. People/Places, etc. 15% 4. Computers/Internet 13% 5. Social/Culture 9% 6. Health/Sciences 6% 7. Education/Humanities 5% 8. Sex/Pornography 4% 9. Performing/Fine Arts 3% 10. Government 3% 11. Entertainment, etc. 2% Query Subjects – Alta Vista 2002 & Vivisimo 2004

  19. Web Search Session Level Analysis • Search duration • Search patterns • Successive and multitasking sessions • Page or resource viewing

  20. 56% less than 1 minute 72% sessions less than 5 minutes 81% sessions less than 15 minutes Mean: approx. 58 minutes and 2 seconds (see Jansen, B. J., Spink, A., and Koshman, S. 2007. Web searcher interactions with the Dogpile.com meta-search engine. Journal of the American Society for Information Science and Technology. 58(5), 744-755.) Web Session Duration (Minutes)

  21. Transaction Log Analysis (TLA) Methods • Quantitative and statistical analysis – requires software and expertise • Qualitative analysis – requires training • Creativity factor • Combination of quantitative and qualitative methods

  22. TLA Strengths • Data from a large user base • Reasonable and non-intrusive • Less time than other methods • Can be relatively inexpensive

  23. TLA Limitations • Transaction logs do not include user demographic and other data • Lacks data on search reasons and motivations • Incomplete data due to corrupted logging

  24. Conclusions • Search analysis is a complex process with many choices • TLA a powerful tool • Requires planning, training and expertise • Can be combined with other data collection and analysis techniques

  25. Further Reading Spink, A., & Jansen, B. J. (2004). Web Search: Public Searching of the Web. Springer. Jansen, B. J. (2006). Search log analysis: What is it; what's been done; how to do it. Library and Information Science Research, 28(3), 407-432 Jansen, B. J., Spink, A., & Taksa, I. (forthcoming). Handbook of Web Log Analysis. Idea Group Publishing.

  26. QUESTIONS? Thank You

More Related