Digital Preservation: Lessons learned through national action Digital Preservation

1. Digital Preservation: Lessons learned through national action Digital Preservation Interoperability Framework Workshop April 2010

2. Chartered by the US Congress in 2000, the National Digital Information Infrastructure and Preservation Program (NDIIPP) began with a strategy of collaboration and iteration. 2

3. Mission To ensure access over time to a rich body of digital content through the establishment of a national network of partners committed to selecting, collecting and preserving at-risk digital information

4. The Program is engaged in forming a network of people and an architecture for preservation.

5. The NDIIPP motto is “Learn by doing” Some of the lessons we have learned thus far are….

6. We can think of digital preservation as being like a race with relay teams proceeding in parallel lanes at different speeds towards the finish line. Enabling access across time and organizational and technical change is the equivalent of a finish line.

7. Relay as metaphor for digital preservation • We take on preservation a generation at a time preparing to do the best we can in order to hand it off to the next generation. • Throughout the lifecycle of a digital object, there are numerous hand-offs across organizations and technical environments. • It is at the handoffs that the community needs to pay attention to standards and best practices.

8. Community Environment Organization Content Consider these aspects to digital preservation—the content, the organizations taking responsibility, the technical environment, and the community.

9. Content

10. Audio Visual Geospatial Web Sites Images and Text Content The NDIIPP partner projects have been working with these categories of digital content. The work has been driven primarily by content type.

11. This is a map of the diverse content domains represented by NDIIPP work.

12. What we learned about content standards • Format standards are key to understanding how to manage digital content through changes in software and environments. • Packaging standards are essential to moving content reliably between creator and stewardship entities. We learned this during our first project, Archive Ingest and Handling Test, and through transferring over 100 Tb of content across partner organizations. • Metadata standards are important to content creators. We learned this from our partnerships with photography associations.

13. Organization

14. NDIIPP Partner Organization Types 2010

15. What we learned about organizations • Business processes are not interoperable. Sometimes it took as much work to get the agreement and funding in place as to do the project. • Organizations need a local benefit to engage in larger collaborations. • Organizations have interests beyond their own domain. Example: Photographers associations are interested in collaborating with museum imaging standards efforts.

16. Environment

17. This initial view of architecture reflects the NDIIPP approach to infrastructure from a shared tools and services perspective, acknowledging that local resources would steer adoptions of storage and software at stewardship institutions.

18. Tools and services were developed around the 3 aspects of the architecture. Access and Views Understanding, Interpreting, Caring For Ingest, Acquisition Export Storage, Data Assurance Indiana Digital Preservation Summit 3rd International Digital Curation Conference 18 18

19. Tools and Services from the NDIIPP projects NGDA Tools NutchWAX Wayback LOCKSS Recollection Access and Views Conspectus Database DataVerse GIS Archiving Tool Heritrix Hub & Spoke JHOVE LoDN LOCKSS NGDA Tool Suite PAWN SRB Web Archives Workbench Web Archiving Service BagIt Memento UDFR Understanding, Interpreting, Caring For Ingest, Acquisition Export ACE DataVerse FACIT iRODS JHOVE LOCKSS L-Store DuraSpace Storage, Data Assurance Indiana Digital Preservation Summit 3rd International Digital Curation Conference 19 19

20. What we learned about environment • There is a high interest in sharing tools and services for preservation but local repository adoptions depend upon organizational resources and policies. • The simplest and most transparent tools are most often adopted. The broad acceptance of LOCKSS attests to this. • Hardware vendors need to be engaged to understand preservation requirements. NDIIPP sponsors an annual storage meeting with vendors.

21. Communities

22. What we learned about communities • Shared expertise can reduce time and effort to establish local preservation environments. Ex. The MetaArchive Cooperative • Working together builds trust. • Digital preservation is a task too challenging for a single institution. It requires multiple domain expertise, about content and technology. • Communities are essential to sustain digital preservation efforts for the long-term. • Standards need to be considered in the context of diverse communities.

23. NDIIPP efforts engaged with standards

24. Standards and best practices are needed at the intersections of preservation regimes where technology and organizations hand-off content.

25. For more information: • Consult the NDIIPP website: www.digitalpreservation.gov • Or email Martha Anderson mande@loc.gov