Cross Collection Discovery in the Yale Digital Commons

1. Cross Collection Discoveryin the Yale Digital Commons Youn Noh November 19, 2010

2. Outline • Introduction • Project background • Related work • Project context • Office of Digital Assets and Infrastructure • Yale Digital Commons • Current phase • Goals and deliverables • Campus partners and collections • Architecture and metadata • Demo • Future work

3. Outline • Introduction • Project background • Related work • Project context • Office of Digital Assets and Infrastructure • Yale Digital Commons • Current phase • Goals and deliverables • Campus partners and collections • Architecture and metadata • Demo • Future work

4. Digital Collections at Yale

5. Digital Collections at Yale • Yale faculty and students access Yale’s extensive collections for teaching and research. • Some Yale faculty would also like to archive their personal collections. • Yale’s Information Technology Services supports web site development for particular classes. Web sites are typically developed as one offs. • Yale’s Office of Digital Dissemination, in the Office of the Secretary, promotes the internationalization of Yale and the dissemination of Yale’s collections to the world.

6. The Problem of Silos

7. The Problem of Silos • Thematically related content is separated. • User interfaces have to be built for each collection. • Users may not know where to look. • The information architecture for Yale’s search environment is still largely based on organizational structure. • There is no easy way to drill down to content based on interests or information need. • Resources may not be organized or described in a consistent manner across collections.

8. Outline • Introduction • Project background • Related work • Project context • Office of Digital Assets and Infrastructure • Yale Digital Commons • Current phase • Goals and deliverables • Campus partners and collections • Architecture and metadata • Demo • Future work

9. Cross Collection Search (2007) • Mellon-funded Collections Collaborative re-grant project to enhance discovery, search, and access to Yale’s special collections. • Partnership led by Yale University Library and Yale’s Information Technology Services. • Proof-of-concept metadata aggregation using OAI-PMH. • Challenges and lessons learned • Reusable infrastructure requires upfront investment. • Payoffs are not immediate. • Sustainability is always an issue.

10. Outline • Introduction • Project background • Related work • Project context • Office of Digital Assets and Infrastructure • Yale Digital Commons • Current phase • Goals and deliverables • Campus partners and collections • Architecture and metadata • Demo • Future work

11. Single Search for Library, Archive and Museum Collections • Project sponsored by OCLC to create guidelines for the implementation of single search for local aggregations of LAM collections. • Working Group • Getty Research Institute • Minnesota Historical Society • Smithsonian Institution • Wellcome Trust • UC Berkeley • University of Calgary • Victoria and Albert Museum • Yale Center for British Art • Yale University • Final deliverable will be a white paper based on an internal survey that addresses issues identified by the Working Group.

12. ARTstor Shared Shelf • Project to develop a cataloging and image management system that integrates with the ARTstor Digital Library. • Target audience • Library visual resources collections • Instructional technology (and faculty) • Fills a gap. • No single image cataloging system has market dominance. • Leverages strengths. • The ARTstor Digital Library has a broad user base. • Cataloging interface is being developed iteratively based on requirements gathering and user testing at partner institutions. • Business model is being developed in consultation with the Shared Shelf Steering Committee, which includes Cornell and Yale.

13. Outline • Introduction • Project background • Related work • Project context • Office of Digital Assets and Infrastructure • Yale Digital Commons • Current phase • Goals and deliverables • Campus partners and collections • Architecture and metadata • Demo • Future work

14. Office of Digital Assets and Infrastructure • Provides strategic and operational leadership for the development of Yale’s digital assets and infrastructure. • Leads and coordinates collaboration among campus units. • Galleries, libraries, archives, and museums • Office of Digital Dissemination • Office of Public Affairs and Communications • Yale University Press • Identifies overlaps and gaps in infrastructure for teaching and research. • Arts Area Advisory Committee • Collections and Educational Technology • Provost’s Committee on Scholarly Publishing • Mass Storage Working Group

15. Yale Digital Commons • Provides a collaborative framework for developing services to support Yale’s digital assets throughout their lifecycle. • Supports digital production, collaboration, dissemination, and stewardship functions. • Improves sustainability of programs through larger-scale adoptions. • Services • Digital asset management • Digital preservation • Persistent linking • Cross collection discovery

16. M M M M M Content Export Aggregate M M M M M Metadata Messaging Yale Digital Commons Components iTunesU YouTube DAM Drupal Web Persistent Linking Digital Preservation Metadata • CollectionManagement • Systems • - Orbis • - TMS • eMU Kaltura CDN Data Warehouse/ Reporting Cross Collection Discovery Search OAI Isilon Mass Storage C1 C2

17. Outline • Introduction • Project background • Related work • Project context • Office of Digital Assets and Infrastructure • Yale Digital Commons • Current phase • Goals and deliverables • Campus partners and collections • Architecture and metadata • Demo • Future work

18. Cross Collection DiscoveryGoals and Deliverables • Goals • Develop shared practices and infrastructure. • Provide broader access to Yale’s collections. • Deliverables • Metadata aggregation service (built on OAICat) • Central OAI service provider harvests metadata from campus partners. • File transfer option for partners that do not implement providers. • Central OAI data provider provides aggregated metadata to external harvesters. • User interface and search service (built on VuFind) • Customized record displays based on metadata format. • Crosswalk for indexing, advanced search, and facets. • Normalized local controlled vocabularies for key fields. • Programmatic access provided via APIs.

19. Cross Collection DiscoveryCampus Partners and Collections • Yale Center for British Art • Paintings and Sculpture • Prints and Drawings • Rare Books and Manuscripts • Yale Peabody Museum • All departments • Yale University Art Gallery • All departments • Yale University Library • Map Collection • Lewis Walpole Library Prints and Drawings • Office of Digital Dissemination • Yale University on iTunes U

20. Cross Collection DiscoveryArchitecture

21. Cross Collection DiscoveryMetadata • Crosswalks and mappings • Local database schemas • eMU • TMS • Yale NetCast tool • Standard metadata formats • CDWA-Lite • Darwin Core • Dublin Core • MARC • VuFind / Solr index fields • Based roughly on MARC. • New fields added as needed. • XSL used for transformations.

22. Cross Collection DiscoveryMetadata • Facets and local controlled vocabularies • Access • Metadata must be in the public domain. • Assets may be restricted. • Important distinction to make in user interface for non-Yale users. • Local controlled vocabulary of integer values (0 for public domain and 1 for restricted access) used to designate type of access. • Providers host assets and handle user authentication. • Institution • Important for campus partners. • Collection • Corresponds to museum departments, library collections, and categories in iTunes U. • Provided as OAI sets. • Means of bringing together similar resources held by different units.

23. Cross Collection DiscoveryMetadata • Facets and controlled vocabularies • Creator (1XX) • For specimens? Scientific name author. • Type (LDR/06) • Museums use normalized local controlled vocabulary for classification developed for digital asset management system object models. • Topic (6XX) • Topical or iconographic description is important for access. • Museums are exploring social tagging to broaden access. • Genre (655) • Region (651, 650z, 690z) • Museums use culture. • Era (648, 650y, 690y) • For specimens? Periods, epochs, ages, groups, and formations.

24. Cross Collection DiscoveryDemo • Search • Item record display • Resource dissemination • Refine search

25. Outline • Introduction • Project background • Related work • Project context • Office of Digital Assets and Infrastructure • Yale Digital Commons • Current phase • Goals and deliverables • Campus partners and collections • Architecture and metadata • Demo • Future work

26. Cross Collection DiscoveryFuture Work • Usability • Stakeholder survey • User testing • Search analytics • Controlled vocabulary services • Use ARTstor vocabulary services. • Search optimization • Tweak Lucene / Solr to boost fields and records in search. • Topic modeling • Apply probabilistic text mining technique for learning topics across collections.