Design Principles for Digital Preservation Systems

1. Design Principles forDigital Preservation Systems Systems? Systems Stephen Abrams University of California Curation Center California Digital Library www.cdlib.org/uc3

2. Design Principles forDigital Preservation Systems Programs Stephen Abrams University of California Curation Center California Digital Library www.cdlib.org/uc3

3. What’s the problem we’re trying to solve? • Connecting people with digital content in meaningful ways across barriers of space and time • Preservation is the end, systems are just the means Robbi Verte, Gesture of hand holding a flash disk, www.123rf.com/ Pietro Izzo, Open hand, www.flickr.com/photos/pietroizzo/482812880

4. Think outside the (system) box « Integration » • Worry about designing your overall preservation program before considering the systems that will implement parts of it Luxmart, Working together team puzzle concept, http://commons.wikimedia.org/wiki/File:Working_Together_Teamwork_Puzzle_Concept.jpg

5. Think outside the (system) box « Integration » • Worry about designing your overall preservation program before considering the systems that will implement parts of it • A preservation program should provide effective control over managed content in several key areas • Technical control • Managing bits, descriptions of bits, relationships between bits, etc. • Analysis, planning, monitoring, intervention, etc. • Intellectual (or curatorial) control • Creation, selection/acquisition, arrangement, cataloging, etc. Cf. Kenney & McGovern (2003), “Five organizational stages of digital preservation,” Digital Libraries: A Vision for the 21st Century (Ann Arbor: MPublishing,),hdl:2027/spo.bbv9812.0001.001

6. Good design is good design • Any principles for preservation system design should be informed by general principles for system design and engineering design • Beginning with a clearly defined need • Leading to a creative design in response to the need • Resulting in a system fully meeting the need Cf. Royal Academy of Engineering (1999), Principles of Engineering Design www.raeng.org.uk/education/vps/principles/pdf/armstrong_keynote.pdf Jason DeRusha, Crying baby shot, www.flickr.com/photos/derusha/1465953800 Izumi Mitatami, The hamburger, www.flickr.com/photos/marvin_izumi/3881467402 Sean Dreilinger, Big hamburger, little kid, www.flickr.com/photos/seandreilinger/3002176844

7. Design principles for preservation systems • Integration  Least surprise • Definition  Elegance • Generality  Community • Parsimony • Modularity • Granularity • Orthogonality • Emergence • Redundancy • Evolution • Transience Yes, some of these may sound, or even be, somewhat inconsistent or contradictory

8. Know what you (really) need « Definition » • What you need, not just what you want • Tie requirements to specific use cases • Actual, expected, anticipated, or probable • Hypothetical? • Acceptance criteria • Did you get what you asked for? Guardian, Five reasons why waiters don’t write down your order, www.guardian.co.uk/lifeandstyle/2011/aug/14/waiters-dont-write-orders-down

9. Know what you don’t need « Parsimony » • Necessity vs. sufficiency vs. superfluity • “A scientific theory should be as simple as possible, but no simpler” – Einstein • “It is futile to do with more things that which can be done with fewer” – William of Ockham • “Not too big, not too small, just right” – Goldilocks Wikimedia Commons, Goldilocks 1912, commons.wikimedia.org/wiki/File:Goldilocks_1912.jpg

10. Inclusive applicability « Generality » • Solve a general problem today, to avoid having to solve a specific problem tomorrow • Support for re-configuration or self-configuration • Facilitate (re)use, potentially in novel ways Cf. Yourdon and Constantine (1979), Structure Design: Fundamentals of a Discipline of Computer Programming (Prentice-Hall), www.win.tue.nl/ ~wstomv/ quotes/structured-design.html#19 Brian Snelson, Lovely new metric spanners and torque wrench, www.flickr.com/photos/exfordy/353771860 LoggerHead Tools, Bionic wrench, loggerheadtools.com

11. The Unix philosophy « Modularity / Granularity / Orthogonality / Emergence » • Make each program do one thing well • To do a new job, build afresh rather than complicate old programs by adding new features • Expect the output of every program to become the input to another, as yet unknown, program • Don't hesitate to throw away the clumsy parts and rebuild them Cf. McIllroy et al. (1978), “Unix time-sharing system: Forward,” Bell Systems Technical Journal 57(6): 1899–1904, www3.alcatel-lucent.com/bstj/vol57-1978/articles/bstj57-6-1899.pdf WindellOskay, Inside-out Lego brick, www.flickr.com/photos/oskay/265899811

12. Just in case, just in case « Redundancy » • Plan for failure • Replication to avoid single points of failure • Decorrelation to avoid cascade failure Cf. Rosenthal (2010), “LOCKSS: lots of copies keeps stuff safe,” US Workshop on Roadmap for Digital Preservation Interoperability Framework, NIST, Gaithersburg, MD lockss.org/locksswiki/files/NIST2010.pdf Energy Press, PWC: Το δάνειο στο ΛΑΓΗΕ δεν αρκεί για να μην καταρρεύσει η αγορά www.energypress.gr/news/lianikh-reymatos/PWC:-To-daneio-sto-LAGHE-den-arkei-gia-na-mhn-katarreysei-h-agora NCinDC, Life is one big balancing act, www.flickr.com/photos/ncindc/3229050640

13. First make it work, then make it work better « Evolution » • Configuration • Customization • Iterative enhancement Cf. May and Zimmer (1996), “Evolutional development model for software,” HP Journal (August): 39-45, www.hpl.hp.com/hpjournal/96aug/aug96a4.pdf Wikimedia Commons, commons.wikimedia.org/wiki/File:Human_evolution_scheme.svg

14. Easy come, easy go « Transience » • Preservation systems are inherently ephemeral and expendable; the content managed in them is not • Avoid system lock-in • Standardized content representation • Standardized APIs • Smooth migration paths • Your (aging) system’s DIP should be a replacement system’s SIP • Preferably, change at a time and place of your choosing Cf. Janée (2009), “Relay-supporting archives: Requirements and progress,” International Journal of Digital Information 4(1), www.ijdc.net/index.php/ijdc/article/view/102

15. Keep the customer satisfied « Least surprise » • Default system behaviors should conform to implicit user expectations • Know the communities you are seeking to serve • Consistency • Treat like things alike Cf. Raymond (2003), “Applying the rule of least surprise,” Art of Unix Programming (Addison-Wesley), http://www.faqs.org/docs/artu/ch11s01.html Narufag, Naruto in the scream, narufag.deviantart.com/art/naruto-in-the-scream-267479366

16. Commodity, firmness, and delight « Elegance » • “Well building hath three conditions: firmness, commodity, and delight” – Vitruvius, De architectura[trans. Wotten, 1694] • The analogous conditions for computer, rather than structural, architecture are… • Utility • Resilience • Elegance Cf. Madni (2012), “Elegant systems design: Creative fusion of simplicity and power,” Systems Engineering 15(3): 347-54, doi:10.1002/sys.21209 Dominic 2007, Pantheon Dome, www.flickr.com/photos/9556741@N03/3157684854

17. Beg, borrow, or steal « Community » • Learn from the solutions and experience of the community • Support the community and contribute back Cf. Anderson (2011), “National Digital Stewardship Alliance: Community, content, commitment,” CENDI Principals and Alternatives, Washington, DC www.cendi.gov/presentations/03_06_11_Anderson_Martha_NDSA.pdf Duncan Hall, Attention aux PickPockets, www.flickr.com/photos/dullhunk/4575707721 Enrique Martinez Bermejo, Community-manager, www.flickr.com/photos/kikemb/5428414543

18. Principles in action: micro-services • Decomposition of infrastructure function into a granular set of independent, but highly interoperable services Cf. Abrams, Cruse, Kunze, and Minor (2011), “Curation micro-services: A pipeline metaphor for repositories,” Journal of Digital Information 12(2), journals.tdl.org/jodi/article/view/1605 “Archivematica implements a micro-service approach to digital preservation” http://www.archivematica.org/ “Curation Architecture Prototype Services (CAPS), is built on the micro-services approach to digital curation” http://www.libraries.psu.edu/ “The micro-services approach … seemed similar to the SDR 2.0 principle of making services more modular” http://library.stanford.edu/ “[SDB] provides a viable solution to the challenges of long term digital preservation by delivering a flexible, extensible set of micro-services” http://www.tessella.com/ “The University of North Texas (UNT) has implemented a robust architecture for digital library initiatives utilizing the Curation Micro Services methodology for building repository infrastructure” http://www.library.unt.edu/

19. Principles in action: Merritt repository UI/API DataONE coord’ing node LDAP LDAP LDAP RDBMS RDBMS RDBMS RDBMS No-SQL RDBMS RDBMS User agent Load balancer UI/API DataONE member node UI/API DAS Ingest Inventory Storage node SAN Load balancer Ingest Message queue Storage broker Storage node Ingest Cloud Fixity Storage node IDF DataCite EZID http://www.cdlib.org/uc3/merritt http://merritt.cdlib.org/

20. Design principles for preservation systems • Integration  Least surprise • Definition  Elegance • Generality  Community • Parsimony • Modularity • Granularity • Orthogonality • Emergence • Redundancy • Evolution • Transience Note that these are principles, not rules Their applicability will depend on local needs, conditions, expertise, resources, etc. — Rely on your intuition and experience www.slideshare.net/UC3/pasig-2013abramsdesignprinciplesforpreservationsystems

21. UC Curation Center • www.cdlib.org/uc3 uc3@ucop.edu Stephen Abrams David Loy Patricia Cruse Mark Reyes Shirin Faenza Joan Starr Scott Fisher Carly Strasser Erik Hetzner Marisa Strong Joshua Hubbard Adrian Turner Greg Janée Bhavitavya Vedula John Kunze Kenneth Weiss Rosalie Lack Perry Willet www.slideshare.net/UC3/pasig-2013abramsdesignprinciplesforpreservationsystems