EUBA : The E mory U ser B ehavior A nalysis System

1. EUBA: The Emory User Behavior Analysis System Eugene Agichtein, Qi Guo and Ryan Kelly Intelligent Information Access Lab http://ir.mathcs.emory.edu Math & Computer Science Department Arthur Murphy, Selden Deemer, Kyle Fenton Emory Libraries

2. Goals/Motivation • Evaluate effectiveness of search and discovery with automatic behavioral metrics • Perform aggregate and longitudinal studies • Develop tools for usability studies “in the wild” • Scale (hundreds/thousands of “participants”) • Realistic behavior and tasks • On-demand playback of “interesting” sessions • Unified analysis/query framework for internal and external resource access and usage statistics • Web-based query and statistics interface • Access auditing, privacy, anonymity enforced Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

3. Approach: Client-side instrumentation • Implemented on top of the Emory Installation of the LibX Toolbar: (http://www.libx.org) • Extended LibX to track UI events: JavaScript patch to sample the mouse movements and other events on pre-specified web search pages. Events are encoded into a string and buffered, and periodically sent to the server (on internal library network). Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

4. Events captured (v0.5, Aug. 2008) • Button/link clicks/Url changes • Name of the button, link, other meta-info • Mouse movements • (x,y) coordinates sampled ~every 10ms • Scrolling • Start, stop position, ~ every 10ms • Text entry, keypress (ctrl-c, ctrl-v) • Query text, options changes • Menu item events • Print, bookmark, save (all of them) • Hover over important elements • Mouse-in/out of browser Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

5. How it works • On login to Learning Commons, Firefox is started with • http://irlib.library.emory.edu/consent.cgi?user=USERID • If previously opted in (or out), goto homepage • Else show consent form • Store user choice in database; if opted in, also store salted hash string for user login • Can opted-in user behavior over “lifetime” • No way to recover login id by dictionary attack • Can be removed at any time by deleting mapping Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

6. How it works (2 of 3): Consent http://irlib.library.emory.edu/ Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

7. How it works (3 or 3): which URLs? • For all visited URLs LibX notifies the server; information varies by type of site: • White list (search sites): • Black list (known private sites): Only domain name is saved • All “https://” and “mail.*” URLs • White list (known search/discovery sites): • EUCLID, Primo, Google, Google Scholar, Yahoo and Live search engines, Wikipedia • All events captured • Gray list (search results and important public sites) • Mouse moves and clicks (no keypress/text) • The rest: • Only URL, button clicks, and menu items Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

8. Emory User Behavior Analysis System • Combines client side instrumentation, server-side caching, log management, querying, and analysis • Client-side instrumentation, data mining/machine learning (Qi Guo) • Log DB parsing, indexing, web-based interface for querying, playback, annotation (Ryan Kelly) • Plan: to release the system to research/library community (2009?) Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

9. EUBA Web-based analysis interface Prototype: http://ir.mathcs.emory.edu/library/private/index.pl user: test password: notsafe Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

10. Future Plans • Incorporate log data for ranking, discovery, query suggestion, collaborative filtering • Richer statistics and visualization • Streamline usability studies • Comments and suggestions welcome! Intelligent Information Access Labhttp://ir.mathcs.emory.edu/