Metasearching: The Problem, Promise, Principles, Possibilities & Perils

Metasearching: The Problem, Promise, Principles, Possibilities & Perils Roy Tennant California Digital Library

Purpose • To provide an overview, to set the stage • To help you begin thinking both about the possibilities, as well as current issues and problems • To hopefully leave you with the impression that despite the problems, it’s still very worth doing NISO Metasearching Meeting, October 2003

The Problem • Most users do not care where the information they need comes from, or who provides it…nor should they have to • But our systems presently require them to know: • How to select one or more databases • How to get to them • How to use the unique search options for each • How can we create systems that minimize what the user needs to know to get what they want? NISO Metasearching Meeting, October 2003

Local Catalog Vendor Dbs Local Web Site Remote Catalogs Remote Digital Content Local Digital Content Remote Web Sites

The Promise • The “Holy Grail” of resource discovery: one-stop shopping • The simplification of a formerly complex activity (put the complexity in the back end, not the front) • Allows the user to focus on evaluating results, not figuring out where to search NISO Metasearching Meeting, October 2003

Before we talk specifics… …let’s start with some principles… NISO Metasearching Meeting, October 2003

Principles Only librarians like to search, everyone else likes to find NISO Metasearching Meeting, October 2003

Principles All things being equal, one place to search is better than two or more. NISO Metasearching Meeting, October 2003

Principles “Good enough” is often just that. NISO Metasearching Meeting, October 2003

Principles Our ability to create effective one-stop searching is dependent on our ability to appropriately target user needs. NISO Metasearching Meeting, October 2003

Principles The size of the result set doesn’t matter as much as how the results are presented. NISO Metasearching Meeting, October 2003

Principles Services should be placed as close to the user as possible. NISO Metasearching Meeting, October 2003

Possibilities: Software Solutions • Special “metasearching” software optimized for: • Creating and managing target and resource profiles • Managing authorization and authentication • Parsing search strings and result sets • Merging, deduping, and ranking results • Stand-alone or integrated with an ILS • Many vendors (although a number are based on the same underlying software) NISO Metasearching Meeting, October 2003

Possibilities: Implementation Scenarios • One search box for all resources (only for small libraries w/few targets) • An interface that encourages/requires the user to select a subject or task group of DBs • Tailored portals for particular needs and user communities or purposes (e.g., “a few good things”) • Search boxes in other systems (e.g., courseware) NISO Metasearching Meeting, October 2003

Slide from Greg Van Essen, Endeavor

Perils • Software Provider Concerns • Database Provider Concerns • Library Concerns • User Concerns • Note: the following lists are not comprehensive, but illustrative NISO Metasearching Meeting, October 2003

Software Provider Concerns • Access management • Search mapping • Unreliability of targets • Systems that don’t support an API (that must be screen-scraped) • Inadequate result data for good: • Deduping • Ranking NISO Metasearching Meeting, October 2003

Database Provider Concerns • Access control (robust authentication and authorization) • Load • Inappropriate searches (searching databases that don’t apply) • Branding and “unfair” deduping NISO Metasearching Meeting, October 2003

Library Concerns • Selecting the right system • Cost (both upfront and ongoing) • System design and implementation • System maintenance • Ability to add new resources/targets • Ease of interface changes • Ease of upgrades NISO Metasearching Meeting, October 2003

User Concerns • What must I go through before hitting the search button? • How difficult is it to review results? • Are results ranked by relevance? (that will be my assumption) • Will I get buried? (too many sources, too many results?) • Do I have methods to easily focus in on what I want? • Once I find what I want, can I get to the full-text with a click? • Can I copy a citation and put it in my paper? NISO Metasearching Meeting, October 2003

Possible Futures • More standards for this type of interaction (i.e., support NISO’s efforts) • Effective results ranking • Learning from other systems (e.g., RedLightGreen.com’s citation displays) • Dynamic selection of sources based on user query NISO Metasearching Meeting, October 2003

Recap • “One-stop shopping” for resource discovery is the Holy Grail of librarianship • Metasearch services offer the best opportunity we’ve yet seen to attain this goal • It ain’t perfect, but it’s well worth pursuing • If we can present solid resources next to Google hits — and win back our users in the process — what’s not to like? NISO Metasearching Meeting, October 2003

Metasearching: The Problem, Promise, Principles, Possibilities & Perils