Core Issues in Digital Preservation

1. Core Issues in Digital Preservation Jacob Nadal, Preservation OfficerUCLA Library

2. Is there a general preservation framework that applies to all records? How does it differ in application between artifactual and digital preservation? PERSPECTIVE

3. preserv- -ation -ed Preservation consists in sustainable efforts, optimized over time

4. preserv- -ation -able -ed -ing Preservation consists in sustainable efforts, optimized over time

5. Framework • Materials: tangible substance that carries media • Media: materials that record information • Transport: means(s) for perceiving media • Language: system for interpretation of media

6. LINEAR B: Digital Preservation Analog Photo: British Museumhttp://www.britishmuseum.org/

7. Linear B • Bronze Age Cretan script: c. 1450 to 1375 B.C. • No cribs, such as the Rosetta Stone, an almost entirely logical decipherment • This is the essential problem that digital preservation tries to avert or mitigate • Show all four parts of our preservation framework • Discovered by Sir Arthur Evans, in spring of 1900 on numerous inscribed (media) clay (material) tablets.

8. Linear B Tablet and Transcribed Glyphs Photo: Dennis Jarvishttp://www.flickr.com/photos/archer10/

9. First successes • Counting system was easy to determine • Analog to Digital: Some formats & encodings are favored because they’re easy to identify • 90 distinct characters, indicative of a syllabic system, with a writing direction from left to right • Debate over relation to Greek or Cypriot. Most felt it was a unique Cretan language. • Analog to Digital: Encoding and File Format

10. Alice Kober: Pattern Recognition • 1940 - Alice Kober identifies word triplets • Same word stem with different endings, presumably for case (e.g. accusative, or nominative) • Kober separated symbols into modifiers and word stems • Analog to Digital: Metadata, Headers, Content blocks, Structured data

11. Michael Ventris: Patterns to Prose • Consonant-vowel patterns established • Problem of missing vowels and leading vowels: e.g. di-vi-si-b(i)-le or i-n(i)-di-vi-si-b(i)-le • Analog to Digital: The problem of compression • Developed refinements of Kober’s chart to manage these relationships

13. A few good guesses • Refinement of relationships gave Ventris enough confidence to take a guess at three words, the towns of Anisos, Knossos, and Tulissos • Assigning consonant values opened up more words • Greek philologist John Chadwick partnered to carry forward the decoding of a Greek dialect from the time of the Trojan War.

14. In effect, those are the issues in digital preservation: • Began with identification of parts… • Digital Forensics and Analysis • … associated with possible informational content … • Metadata and Contextual Information • … then instantiated by a subject expert and translated into a known, contemporary language. • Digital Curation and Migration

15. Defining Digital Preservation Photo: John Keogh http://www.flickr.com/people/jvk/

16. Short Definition of Digital Preservation • Digital preservation combines policies, strategiesandactions that ensure access to digital content over time. • Remove digital and it’s a generic definition of preservation • The medium definition adds some strongly digital concepts, that do not bear heavily on artifactual preservation.

17. Medium Definition of DIgital Preservation Digital preservation combines policies, strategies and actions to ensure access to reformattedand born digital content regardless of the challenges of media failureand technological change. The goal of digital preservation is the accurate renderingof authenticated content over time.

18. Long Definition Core • Digital preservation combines policies, strategies and actions to ensure the accurate rendering of authenticated content over time, regardless of the challenges of media failure and technological change. Digital preservation applies to both born digital and reformatted content. • Digital preservation policies document an organization’s commitment to preserve digital content for future use; specify file formats to be preserved and the level of preservation to be provided; and ensure compliance with standards and best practices for responsible stewardship of digital information. • Digital preservation strategies and actions address content creation, integrity and maintenance.

19. Long Form Details:Content Creation • Content creation includes: • Clear and complete technical specifications • Production of reliable master files • Sufficient descriptive, administrative and structural metadata to ensure future access • Detailed quality control of processes

20. Long Form Details: Content Integrity • Content integrity includes: • Documentation of all policies, strategies and procedures • Use of persistent identifiers • Recorded provenance and change history for all objects • Verification mechanisms • Attention to security requirements • Routine audits

21. Long Form Details: Content Maintenance • Content maintenance includes: • A robust computing and networking infrastructure • Storage and synchronization of files at multiple sites • Continuous monitoring and management of files • Programs for refreshing, migration and emulation • Creation and testing of disaster prevention and recovery plans • Periodic review and updating of policies and procedures

22. Long Form (Core) Long Form (Detail) • Content creation includes: • Clear and complete technical specifications • Production of reliable master files • Sufficient descriptive, administrative and structural metadata to ensure future access • Detailed quality control of processes • Content integrity includes: • Documentation of all policies, strategies and procedures • Use of persistent identifiers • Recorded provenance and change history for all objects • Verification mechanisms • Attention to security requirements • Routine audits • Content maintenance includes: • A robust computing and networking infrastructure • Storage and synchronization of files at multiple sites • Continuous monitoring and management of files • Programs for refreshing, migration and emulation • Creation and testing of disaster prevention and recovery plans • Periodic review and updating of policies and procedures • Digital preservation combines policies, strategies and actions to ensure the accurate rendering of authenticated content over time, regardless of the challenges of media failure and technological change. Digital preservation applies to both born digital and reformatted content. • Digital preservation policies document an organization’s commitment to preserve digital content for future use; specify file formats to be preserved and the level of preservation to be provided; and ensure compliance with standards and best practices for responsible stewardship of digital information. • Digital preservation strategies and actions address content creation, integrity and maintenance.

23. Long Form (Core) Long Form (Detail) Create Good Files • Content creation includes: • Clear and complete technical specifications • Production of reliable master files • Sufficient descriptive, administrative and structural metadata to ensure future access • Detailed quality control of processes • Content integrity includes: • Documentation of all policies, strategies and procedures • Use of persistent identifiers • Recorded provenance and change history for all objects • Verification mechanisms • Attention to security requirements • Routine audits • Content maintenance includes: • A robust computing and networking infrastructure • Storage and synchronization of files at multiple sites • Continuous monitoring and management of files • Programs for refreshing, migration and emulation • Creation and testing of disaster prevention and recovery plans • Periodic review and updating of policies and procedures Don’t get lost in the fine print • Digital preservation combines policies, strategies and actions to ensure the accurate rendering of authenticated content over time, regardless of the challenges of media failure and technological change. Digital preservation applies to both born digital and reformatted content. • Digital preservation policies document an organization’s commitment to preserve digital content for future use; specify file formats to be preserved and the level of preservation to be provided; and ensure compliance with standards and best practices for responsible stewardship of digital information. • Digital preservation strategies and actions address content creation, integrity and maintenance. Keep an Eye on Them Store Them Safely

24. Text

25. Text • UTF-8, a way of representing Unicode, is standard • Digital text is purely character data • No font or layout information is stored in a pure text file • Critical for searching and manipulation • XML is a UTF-8 text format

26. Images

27. Images • TIFF standard preservation format; JPEG2000 emerging as a new alternative • Must be uncompressed image data (TIFF and JP2K can both store compressed data) • At least 300 pixels per inch (ppi/dpi), 24-bit color • More pixels allows more magnification without pixelation • Color should be calibrated and profiled with an ICC color profile.

28. Audio

29. Audio • Broadcast WAV (BWAV) – Wave file with a metadata header • WAV audio is Pulse Code Modulation (PCM), the universal format for uncompressed audio • Resolution of at least 44.1 kHz (CD quality), preferably 96 kHz • Bit Depth of at least 16-bit (CD quality), pref. 24-bit

30. Video & Moving Image

31. Video & Moving Image • Standards and practices developing • Uncompressed desirable, but high storage costs • Compression is normal in video, but may cause preservation problems • Uncompressed .AVI is the current safe bet • Motion JP2K & MPEG21 may be options • H.264 becoming the standard for service copies • Pick one, but plan on a migration

32. Data and Interactivity

33. Data and Interactivity • Need to decide if fixed points in time are required: Are you storing an instance of data? • Need to decide if active system is required: Are you maintaining and experience or immersive environment? • Or, are you doing both? • ICPSR: www.icpsr.umich.edu/icpsrweb • CDL: www.cdlib.org/services/uc3/datamanagement • Variable Media Network: variablemedia.net

34. Metadata

35. PREMIS: PREservation Metadata: Implementation Strategies

36. Storage and Maintenance

37. Storage and Maintenance • Lots of options • LOCKSS Networks • Digital archives (OCLC digital archive, DuraSpace) • DIY systems, from a couple removable hard drives, to cloud storage, to building your own data center.

38. RLG: Trusted Digital Repositories • OAIS compliance • Administrative responsibility • Organizational viability • Financial sustainability • Technological and procedural suitability • System security • Procedural accountability

39. The OAIS Reference Model

40. Digital Preservation: Reasonable Expectations • Digital preservation has strong points and weak points; so does artifactual preservation. • With digital preservation, we should expect • High day to day reliability • Low incidence of acid decay, mold, or biohazards • Some preservation problems in the future; stick to standards and the impact will be mitigated

41. Methods of Preservation • Digital Archaeology: Recovery and forensic analysis of data from damaged media • Conservation: Maintaining original equipment for access • Bit preservation: Storage, transfer and refresh of data • Migration: Transformation of data into new formats to allow for continued access • Emulation: Recreation of original operating environment for continued access

42. Methods of Preservation • Digital Archaeology: Recovery and forensic analysis of data from damaged media • Conservation: Maintaining original equipment for access • Bit preservation: Storage, transfer and refresh of data • Migration: Transformation of data into new formats to allow for continued access • Emulation: Recreation of original operating environment for continued access Now, but also never. Step one (and 2… n) This is what’s next.

43. Questions & Comments: jacobnadal.com/247 THANK YOU!