The Next Generation of Library Automation and Discovery: Key Issues and Trends

1. Marshall Breeding Independent Consult, Author, Founder and Publisher, Library Technology Guides http://www.librarytechnology.org/ http://twitter.com/mbreeding The Next Generation of Library Automation and Discovery: Key Issues and Trends WiLSWorldConference July 25, 2012

2. Summary • Libraries today face incredible challenges as they face challenges brought on by shifts in their collections to include ever increasing of electronic content, never-ending budget pressures, and rising expectations by their customers for instant access to information.  In response to these challenges, libraries demand more effective and efficient automation solutions with requirements for additional features and functionality aligned with these new realities that may not have been present in previous automation products.  In the past, libraries could gain adequate automation by choosing the best integrated library system that fit their technical requirements and budget.  Now, for better or worse, many choices now exist that represent quite different paths, including decisions regarding open source versus proprietary products, evolutionary ILS versus new-generation library services platforms, online catalogs versus discovery services, locally implemented versus cloud-based deployment.  Marshall Breeding will present an overview of the current library automation landscape, highlighting the advantages and concerns presented by this new slate of alternatives. 

3. Library Technology Guides www.librarytechnology.org

4. ILS Turnover Report

5. ILS Turnover Report -- Reverse

6. Mergers and Acquisitions http://www.librarytechnology.org/automationhistory.pl

7. Key Context: Libraries in Transition • Academic Shift from Print > Electronic • E-journal transition largely complete • Circulation of print collections slowing • E-books now in play (consultation > reading) • Public: Emphasis on Patron Engagement • Increased pressure on physical facilities • Increased circulation of print collections • Dramatic increase in interest in e-books • All libraries: • Need better tools for access to complex multi-format collections • Strong emphasis on digitizing local collections • Demands for enterprise integration and interoperability

8. Key Context: Technologies in transition • Client / Server > Web-based computing • Beyond Web 2.0 • Integration of social computing into core infrastructure • Local computing shifting to cloud platforms • Application Service Provider offerings standard • New expectations for multi-tenant software-as-a-service • Full spectrum of devices • full-scale / net book / tablet / mobile • Mobile the current focus, but is only one example of device and interface cycles

9. Key Text: Changed expectations in metadata management • Moving away from individual record-by-record creation • Life cycle of metadata • Metadata follows the supply chain, improved and enhanced along the way as needed • Manage metadata in bulk when possible • E-book collections • Highly shared metadata • E-journal knowledge bases, e.g. • Great interest in moving toward semantic web and open linked data • Very little progress in linked data for operational systems • AACR2 > RDA • MARC > RDF (Library of Congress bibliographic framework transition) http://www.loc.gov/marc/transition/

10. Each Library Type Distinctive • Academic – Public – School – Special • Academic: Emphasis on subscribed electronic resources • Public: Engaged in the management of print collections • Dramatic increase in interest in E-books • School: Age-appropriate resources (print and Web), textbook and media management • Special: Enterprise knowledge management (Corporate, Law, Medical, etc)

11. Cooperation and Resource sharing • Efforts on many fronts to cooperate and consolidate • Many regional consortia merging (Example: suburban Chicago systems) • State-wide or national implementations • Software-as-a-service or “cloud” based implementations • Many libraries share computing infrastructure and data resources

12. Each Library Type Distinctive • Academic – Public – School – Special • Academic: Emphasis on subscribed electronic resources • Public: Engaged in the management of print collections • Dramatic increase in interest in E-books • School: Age-appropriate resources (print and Web), textbook and media management • Special: Enterprise knowledge management (Corporate, Law, Medical, etc)

13. Cooperation and Resource sharing • Efforts on many fronts to cooperate and consolidate • Many regional consortia merging (Example: suburban Chicago systems) • State-wide or national implementations • Software-as-a-service or “cloud” based implementations • Many libraries share computing infrastructure and data resources

14. Status Quo Sustainable? • ILS for management of (mostly) print • Duplicative financial systems between library and campus • Electronic Resource Management (non-integrated with ILS) • OpenURL Link Resolver w/ knowledge base for access to full-text electronic articles • Digital Collections Management platforms (CONTENTdm, DigiTool, etc.) • Institutional Repositories (DSpace, Fedora, etc.) • Discovery-layer services for broader access to library collections • No effective integration services / interoperability among disconnected systems, non-aligned metadata schemes

15. Academic Library Issues • Greater concern with electronic resources • Management: Need for consolidated approach that balances print, digital, and electronic workflows • Access: discovery interfaces that maximize the value of investments in electronic content

16. Major trend in Information Technology Few organizations have core competence in large-scale computer infrastructure management Essentially outsourcing of server housing and management Usually based on a consumption-based business model Most new automation products delivered through some flavor of cloud computing Many flavors to suit business needs: public, private, hybrid Cloud Computing

17. Multi Tennant SaaS is the modern approach One copy of the code base serves multiple sites Software functionality delivered entirely through Web interfaces No workstation clients Upgrades and fixes deployed universally Usually in small increments Software as a Service

18. SaaS provides opportunity for highly shared data models WorldCat: one globally shared copy that serves all libraries Primo Central: central index of articles maintained by Ex Libris shared by all libraries implementing Primo / Primo Central KnowledgeWorks database of of e-journal holdings shared among all customers of Serials Solutions products General opportunity to move away from library-by-library metadata management to globally shared workflows Data as a service

19. Open Systems • Achieving openness has risen as the key driver behind library technology strategies • Libraries need to do more with their data • Ability to improve customer experience and operational efficiencies • Demand for Interoperability • Open source – full access to internal program of the application • Open API’s – expose programmatic interfaces to data and functionality

20. Mobile Computing

21. Challenge: Disjointed approach to information and service delivery • Library Web sites offer a menu of unconnected silos: • Books: Library OPAC (ILS online catalog module) • Articles: Aggregated content products, e-journal collections • OpenURL linking services • E-journal finding aids (Often managed by link resolver) • Subject guides (e.g. SpringshareLibGuides) • Local digital collections • ETDs, photos, rich media collections • Metasearch engines • Discovery Services – often just another choice among many • All searched separately

22. ILS Data Online Catalog Search: Scope of Search • Books, Journals, and Media at the Title Level • Not in scope: • Articles • Book Chapters • Digital objects • Web site content • Etc. Search Results

23. Next-gen Catalogs or Discovery Interface (2002-2009) • Single search box • Query tools • Did you mean • Type-ahead • Relevance ranked results (for some content sources) • Faceted navigation • Enhanced visual displays • Cover art • Summaries, reviews, • Recommendation services

24. Discovery Interface search model ILS Data Digital Collections Search: Local Index ProQuest Search Results EBSCOhost MetaSearch Engine … MLA Bibliography ABC-CLIO Real-time query and responses

25. Discovery Products http://www.librarytechnology.org/discovery.pl

26. Differentiation in Discovery • Products increasingly specialized between public and academic libraries • Public libraries: emphasis on engagement with physical collection • Academic libraries: concern for discovery of heterogeneous material types, especially books + articles + digital objects

27. Discovery from Local to Web-scale • Initial products focused on technology • AquaBrowser, Endeca,Primo, Encore, VuFind, • LIBERO Uno, Civica Sorcer, Axiell Arena • Mostly locally-installed software • Current phase is focused on pre-populated indexes that aim to deliver Web-scale discovery • Primo Central (Ex Libris) • Summon (Serials Solutions) • WorldCat Local (OCLC) • EBSCO Discovery Service (EBSCO) • Encore with Article Integration (no index, though)

28. Web-scale Index-based Discovery ILS Data (2009- present) Digital Collections Search: Web Site Content Institutional Repositories Aggregated Content packages Search Results Consolidated Index … E-Journals Reference Sources Pre-built harvesting and indexing

29. Web-scale Search Problem ILS Data Digital Collections Search: Web Site Content Institutional Repositories Consolidated Index Aggregated Content packages Search Results … E-Journals Pre-built harvesting and indexing ??? Non Participating Content Sources Problem in how to deal with resources not provided to ingest into consolidated index

30. Encore Synergy ILS Data Digital Collections Search: Local Index ProQuest Local Index Results … EBSCOhost Remote Search Results … MLA Bibliography Web Services Local Index Results ABC-CLIO

31. New Library Management Model Unified Presentation Layer Search: Self-Check /Automated Return Library Services Platform ` Digital Coll Search Engine Consolidated index Discovery Service ProQuest API Layer StockManagement EBSCO … Enterprise ResourcePlanning Smart Cad / Payment systems JSTOR LearningManagement AuthenticationService Other Resources

32. Next-gen catalogs or discovery services have been around since 2002 Many mature products Continuing to evolve and expand Online catalog components of ILS products have taken on many of the characteristics of discovery layers Examples: LS2 PAC, Polaris PowerPAC Adoption of Discovery Services

33. Discovery Service Installations

34. EBSCO Discovery Service

35. Global Primo Installations

36. Summon Global Adoption

37. Expanding the Depth of Discovery

38. Citations / Metadata > Full Text • Citations or structured metadata provide key data to power search & retrieval and faceted navigation • Indexing Full-text of content amplifies access • Important to understand depth indexing • Currency, dates covered, full-text or citation • Many other factors

39. Full-text Book indexing • HathiTrust: 11 million volumes, 5.3 million titles, 263,000 serial titles, 3.5 billion pages • HathiTrust in Discovery Indexes • Primo Central (Jan 20, 2012) [previously indexed only metadata] • EBSCO Discovery Service (Sept 8 2011) • WorldCat Local (Sept 7, 2011) • Summon (Mar 28, 2011)

40. Challenge for Relevancy • Technically feasible to index hundreds of millions or billions of records through Lucene or SOLR • Difficult to order records in ways that make sense • Many fairly equivalent candidates returned for any given query • Must rely on use-based and social factors to improve relevancy rankings

41. Quest for Improved Relevancy • Example: Ex Libris Primo ScholarRank • Relevancy tuned for scholarly content • Uses bX data to assign score that reflects scholarly importance • Able to weight by disciplines and filter by other factors for signed-in users • Now available in Primo Version 4

42. Challenges for Collection Coverage • To work effectively, discovery services need to cover comprehensively the body of content represented in library collections • What about publishers that do not participate? • Is content indexed at the citation or full-text level? • What are the restrictions for non-authenticated users? • How can libraries understand the differences in coverage among competing services?

43. Evaluating the Coverage of Index-based Discovery Services • Intense competition: how well the index covers the body of scholarly content stands as a key differentiator • Difficult to evaluate based on numbers of items indexed alone. • Important to ascertain now your library’s content packages are represented by the discovery service. • Important to know what items are indexed by citation and which are full text • Important to know whether the discovery service favors the content of any given publisher

44. Open Discovery Initiative • NISO Work Group to Develop Standards and Recommended Practices for Library Discovery Services Based on Indexed Search • Informal meeting called at ALA Annual 2011 • Co-Chaired by Marshall Breeding and Jenny Walker • Term: Dec 2011 – May 2013

45. Balance of Constituents Marshall Breeding, Vanderbilt UniversityJamene Brooks-Kieffer, Kansas State University Laura Morse, Harvard University Ken Varnum, University of Michigan Anya Arnold, Orbis Cascade AllianceSara Brownmiller, University of Oregon Lucy Harrison, College Center for Library Automation (D2D liaison/observer) Lettie Conrad, SAGE PublicationsBeth LaPensee, ITHAKA/JSTOR/PorticoJeff Lang, Thomson Reuters Linda Beebe, American Psychological AssocAaron Wood, Alexander Street Press Jenny Walker, Ex Libris GroupJohn Law, Serials SolutionsMichael Gorrell, EBSCO Information Services David Lindahl, University of Rochester (XC) Jeff Penka, OCLC (D2D liaison/observer)

46. Timeline

47. ODI Project Goals: • Identify … needs and requirements of the three stakeholder groups in this area of work. • Create recommendations and tools to streamline the process by which information providers, discovery service providers, and librarians work together to better serve libraries and their users. • Provide effective means for librarians to assess the level of participation by information providers in discovery services, to evaluate the breadth and depth of content indexed and the degree to which this content is made available to the user.

48. The rise of e-books • Academic libraries: e-books included in aggregated content packages • E-books used primarily for research and consultation, not long reading • Public Libraries: Subscriptions to e-book services that provide an outsourced collection of loanable e-books • K-12 Schools, Colleges, Universities: interest in electronic textbooks

49. Integrating e-Books into Library Automation Infrastructure • Current approach involves mostly outsourced arrangements • Collections licensed wholesale from single provider • Hand-off to DRM and delivery systems of providers • Loading of MARC records into local catalog with linking mechanisms • No ability to see availability status of e-books from the library’s online catalog or discovery interface

50. Technology Issues • Access to materials controlled through Digital Rights Management • Closed ecosystems that control content through identity management and rights policies • Imposes significant overhead on the user experience: • Download an install DRM components • Establish user credentials in site trusted by DRM • Works only with devices that comply with DRM restrictions