1 / 23

Beyond Basic Search

Beyond Basic Search. Gary Marchionini University of North Carolina at Chapel Hill march@ils.unc.edu CS Colloquium Computer Science Department University of Illinois at Urbana-Champaign September 10, 2007. Outline. Information retrieval R&D has stimulated a multi-billion dollar industry

liedtke
Download Presentation

Beyond Basic Search

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Beyond Basic Search Gary Marchionini University of North Carolina at Chapel Hill march@ils.unc.edu CS Colloquium Computer Science Department University of Illinois at Urbana-Champaign September 10, 2007

  2. Outline • Information retrieval R&D has stimulated a multi-billion dollar industry • The challenges of information seeking and exploratory search that get searcher(s) more actively involved • Some early examples: faceted search, dynamic queries and agile views • Evaluation challenges

  3. Content-Centered Retrieval as Matching Document Representations to Query Representations Match Algorithm Surrogates Surrogates Query Form A Terms Document Space Query Space Sample Sample Vectors Query Form B Etc.. Etc.. A powerful paradigm that has driven IR R&D for half a century. Evaluation metric is effectiveness of the match. (e.g., recall and precision). A half duplex (at best) process that is strongly dependent on pre-processing

  4. Information Seeking Goals: Focus on Exploratory Search Explore Lookup Learn Investigate Accrete Analysis Exclude Synthesis Evaluation Discovery Plan/Forecast Transform Fact retrieval Known item search Navigation Verification Question answering Knowledge acquisition Comprehend/Interpret Compare Aggregate/Integrate Socialize

  5. Characteristics of the Exploratory Search Process • Multiple sessions • Multiple queries • Recall important • Collaborative • Substantial time spent in results • Coordinate with other tools • Relevance judging more difficult (subjective, domain dependent, nuanced) • Objects of interest tend toward abstract and complex

  6. What are we trying to support and evaluate? Active humans with information needs, information skills, powerful IR resources (that include other humans), and situated in global and local connected communities, all of which evolve over time

  7. Dynamic Queries • Direct manipulation (e.g., slider move, hover) defines and executes query with immediate feedback (see http://www.cs.umd.edu/hcil/spotfire/ for history) • An alternative to text query forms

  8. Faceted Search • Combine text search with category selection • Many E-commerce sites • Metadata critical (database backends loved) • Examples: • Flamenco http://flamenco.berkeley.edu/ • mSpace http://www.mspace.fm/ • Endeca http://endeca.com/ also see http://www.lib.ncsu.edu/endeca/ • Relation Browser http://idl.ils.unc.edu/rave/examples.html

  9. AgileViews • A view is a partition of an information space • There are many possible partitions for any space since many attributes may be used to ‘slice and dice’ the space • People should be able to effortlessly change views • Focus can change based on granularity • Focus can change based on attribute

  10. AgileViews Framework

  11. Relation Browser Example with all EIA pages [RB demo here]

  12. RB Goals • Facilitate exploration of the relationships between (among) different data facets • Display alternative partitions of the database with mouse actions • Support string search within partitions • Serve as an alternative to existing search and navigation tools

  13. Relation Browser Principles • Architectural Principle: Juxtapose facets • Two or more with 5-15 categories per facet • Topic is one important facet for most applications • Interaction Principle: Dynamic exploration of relationships among facets and categories • Database driven to promote flexible applications (requires systematic metadata)

  14. Key Challenges • Technical evolutions (Java, metadata to client side) • User expectations and preparations • Getting metadata and mapping to RB scheme • Given the cost and difficulty with hundreds of thousands of web pages, can we automate this process?

  15. Selected User Studies • ACM/IEEE JCDL 2004 • RB and Web UI for movie catalog in UNC Library • 15 participants, 3 kinds of task, 10 trials each UI (counterbalanced), time, accuracy, satisfaction measures • Results: no SRD for simple lookups, RB SRD better for data exploration and analysis; and satisfaction • JCDL 2007 • Relation Browser, Vanilla Facet, BLS website • 40 participants, Between and within subject designs with known item and exploratory tasks • Results: NSR differences • familiarity influences expectations—installed base syndrome • Automatically generated categorization comparable to carefully crafted website layouts

  16. Why Mixed Results? • Human factors (domain knowledge, motivation, system experience, physical and mental capabilities, etc.) • Task factors (precise mappings of types onto content, e.g., number of facets, fact versus concept, etc.) • Appropriate and/or subjective metrics (e.g., accuracy, errors, satisfaction, and time) • Facet and category generation (automatic metadata): machine learning, knowledge-based, manual, and combinations

  17. Open Video Examplewww.open-video.org • Open access digital library of digital video for education and research • ~5000 video segments: MPEG1, MPEG-2, MPEG-4, QuickTime • Multiple visual surrogates • Agile Views Design Framework • Facet partitions (collections, genre, length, etc.) • Multiple examples of views: Surrogates as previews (textual metadata, storyboard, except, fast forward, spoken descriptions/keywords) • Dynamic control mechanisms • Basic search (MySQL indexes)

  18. Alternative Previews for a Specific Video Segment [OV demo here]

  19. Video Surrogate Studies • A dozen studies over 6 years (CHI 07, JCDL 04, ASIST, AVI, MM 06, others) • Story boards, slide shows, fast forwards, excerpts, spoken keywords, spoken descriptions, combinations • Multiple tasks (gist, vist) • Multiple measures (accuracy, time, satisfaction, gaze) • Quant and qual: within, between, ethnographic • Results • Words matter • Visual adds value (conceptual and affective) • People able to infer from few cues, tolerate high rates • Coordination of multiple channels?

  20. User Study Framework For Surrogation: The Evaluation Challenges

  21. Evaluation Directions • Massive click through data with laboratory follow up (e.g., White et al SIGIR 2007) • Longitudinal user behavior (e.g., Kelly dissertation, 2003), new paradigms for behavioral streams via sensors or opt-in services • Laboratory studies of highly specified features • New metrics: use, understanding, adoption • Social behaviors (e.g., marketplace metrics such as ROI and adoption—why do groups adopt?

  22. Status and Preview • SIGIR, SIGCHI workshops (tools, theories, evaluation) • CACM, IP&M special issues • Search engine company R&D • NSF HCC program • Mashups and ideas/systems that integrate search into life activity (consider how successful and transparent autocomplete form fill has become, whereas explicit relevance feedback has not). e.g., UCAIR here at UIUC • Grand challenges of Cyberinfrastructure and Personal information services

  23. Thank You!Questions and Discussionmarch@ils.unc.eduhttp://ils.unc.edu/~march

More Related