Promoting Digital Preservation Partnerships at the U.S. Library of Congress April 2004

1. Promoting Digital Preservation Partnerships at the U.S. Library of CongressApril 2004

2. Presentation Overview • Discuss Library of Congress approach to collaborating on digital content issues • Talk about LC internal activities as well as new program to build a network of preservation partners • Provide details about specific projects Digital Partnerships at LC

3. Challenge is Broad • All institutions that collect and keep information must confront digital preservation • All must devise ways to manage digital objects across their life cycle • Collaboration is vital to identify and leverage solutions • Cultural heritage is at risk Digital Partnerships at LC

4. Digital Preservation Issues/Questions • Issues include: • Rapid technological change • Content volume, volatility, and heterogeneity • Content authenticity • Interoperability among repositories • Questions include: • What is the universe of content? • How to appraise/select? • How to preserve objects? • How to manage intellectual property? Digital Partnerships at LC

5. Addressing Digital Challenges at LC • Internal Activities • External Activities Digital Partnerships at LC

6. American Memory • Gateway to primary sources relating to the history and culture of the United States • Includes over 7 million items: photographs, manuscripts, rare books, maps, recorded sound, and moving pictures • Developed pioneering practices for managing digital collections Digital Partnerships at LC

7. American Memory: Collaboration • Collaboration key for funding, developing American Memory collections • Ameritech competition: Funding for libraries, museums, and archives to create digital collections • 23 collections produced • 33 institutions involved Digital Partnerships at LC

8. Global Gateway • Provides web access to primary source materials of interest to other nations • Current projects with Netherlands, Brazil, Russia, and Spain • Staff from each country work with LC to select content; collections are bilingual Digital Partnerships at LC

9. International Work on Web Preservation • LC part of a new international Internet consortium consisting of 12 national libraries • Working to: • Develop technical specifications for open source tools to harvest and preserve web content • Share experience and best practices Digital Partnerships at LC

10. Current LC Digital Holdings • Over 20 Terabytes collected to date • Examples include: • Web pages documenting significant events, including Iraq conflict, U.S. national elections • George Mason University 9/11 Archives • American Memory and other digitized collections Digital Partnerships at LC

11. Expanding LC Digital Content Collection • Identify digital content documenting important topics and arrange for LC to get and sustain in partnership with others • Initial interest is public policy web content • Other subject areas are under study • LC, other institutions must build new infrastructure to meet much larger needs Digital Partnerships at LC

12. Addressing Digital Challenges at LC • Internal Activities • External Activities Digital Partnerships at LC

13. NDIIPP Legislation and Funding • Created by federal legislation (PL 106-554) in December 2000 • Up to $175 million potentially available • Preserving Our Digital Heritage plan approved December 2002 Digital Partnerships at LC

14. NDIIPP Goals • Develop a national digital collection and preservation strategy • Work with industry, federal agencies, libraries, research institutions, and not-for-profit entities • Help identify and preserve at-risk digital content • Support development of improved tools, models, and methods for digital preservation Digital Partnerships at LC

15. NDIIPP Focus Areas • Network of preservation partners • Preservation architecture • Digital preservation research Digital Partnerships at LC

16. Network of Preservation Partners • Volume and complexity of digital content calls for a distributed approach • LC is providing resources and leadership to construct a network of preservation partners • Three primary aspects will be considered: • Content identification and preservation • Business models • Standards and best practices Digital Partnerships at LC

17. Content Partnerships • First NDIIPP partnership solicitation closed in Nov. 2003; focus is on content • Have received many good proposals covering an excellent cross-section of digital content • Expect to announce awards by Spring 2004 • Important first step in building a national preservation network Digital Partnerships at LC

18. NDIIPP Focus Areas • Network of preservation partners • Preservation architecture • Digital preservation research Digital Partnerships at LC

19. Preservation Architecture • Architecture is a blueprint to guide development of national preservation network • Design principles: • Support institutional relationships • Separate preservation and access • Construct modularly • Assemble over time, not all at once • Upgrade parts without disruption of the whole • Use broadly adoptable standards and protocols Digital Partnerships at LC

20. NDIIPP Preservation Architecture People Above Institutions In Between Bits Below 10101010010101010100 Digital Partnerships at LC

21. Digital Partnerships at LC

22. Archive Ingest and Handling Test • Identify and document issues associated with ingest of heterogeneous set of digital content • Testers are cultural heritage institutions with digital preservation capabilities • Each to work with same test data set: GMU 9/11 Archive • 12gb with 57,000 objects & associated metadata • HTML, graphics, video, sound, PDF, Word, ? Digital Partnerships at LC

23. AIHT (cont.) • Over 1 year, testers to: • Measure and document all ingest processes • Report on steps taken to preserve data integrity/document logical integrity • Detail complete backup and restore • Describe efforts needed to export test set to another institution for ingest • Aim is to refine architecture, identify best practices Digital Partnerships at LC

24. NDIIPP Focus Areas • Network of preservation partners • Preservation architecture • Digital preservation research Digital Partnerships at LC

25. Research Program • Goal is a digital preservation research grants program administered by NSF • Agenda shaped by It’s About Time report • Looking to fund cutting edge research to address major needs • Expect call for proposals within the next several months Digital Partnerships at LC

26. Parallel Approaches to Digital Preservation • Global • Develop community-accepted standards and best practices for broad use • Build distributed tools and solutions across organizational boundaries • Local • Establish organizational policies for creating, maintaining, and selecting/disposing digital objects • Implement specific approaches to formats, metadata, storage, and long-term accessibility Digital Partnerships at LC

27. Global Examples • Open Archival Information System (OAIS) • http://ssdoo.gsfc.nasa.gov/nost/isoas/overview.html • UK Digital Preservation Coalition • http://www.dpconline.org • Metadata Encoding and Transfer Standard (METS) • http://www.loc.gov/standards/mets • PDF/A • http://www.aiim.org/standards.asp?ID=25013 • LOCKSS • http://lockss.stanford.edu/ Digital Partnerships at LC

28. Local Examples • OCLC Digital Archive • http://www.oclc.org/digitalarchive • National Library of the Netherlands e-Depot • http://www.kb.nl/kb • UK Public Records Office • http://www.pro.gov.uk/about/preservation/digital/default.htm • Victorian Electronic Records Strategy • http://www.prov.vic.gov.au/vers • NARA Electronic Records Archives • http://www.archives.gov/electronic_records_archives Digital Partnerships at LC

29. Recap: Collaboration is Key • Issues relating too are too big, too complex for one institution to handle alone • A network of partners working together to manage and preserve content is essential • Applies to LC internal, external activities Digital Partnerships at LC

30. For More Information • memory.loc.gov/ammem/techdocs • Technical info on building digital collections • www.DigitalPreservation.gov • Current Information about NDIIPP • Publications, including Preserving Our Digital Heritage and It’s About Time • William G. LeFurgy • wlef@loc.gov • (202) 707-8618 Digital Partnerships at LC