1 / 12

Curation Micro-Services “It’s a Series of Tubes”

Curation Micro-Services “It’s a Series of Tubes”. Curation Micro-Services “It’s a Series of Tubes”. The Unix philosophy. “Make each program do one thing well” “To do a new job, build afresh rather than complicate old programs by adding new features”

farren
Download Presentation

Curation Micro-Services “It’s a Series of Tubes”

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Curation Micro-Services “It’s a Series of Tubes” Curation Micro-Services“It’s a Series of Tubes”

  2. The Unix philosophy “Make each program do one thing well” “To do a new job, build afresh rather than complicate old programs by adding new features” “Expect the output of every program to become the input to another, as yet unknown, program” “Design and build software … to be tried early” “Don't hesitate to throw away the clumsy parts and rebuild them” — D. L. McIlroy et al., “Unix time-sharing system forward,” Bell System Technical Journal 57:6, part 2 (1978): 1902

  3. The micro-services “philosophy” CC http://www.flickr.com/photos/oskay/265899811/ http://www.flickr.com/photos/elsie/8229790/ CC

  4. Curation micro-services

  5. Curation micro-services

  6. Design goals Principle of least surprise Multiple interface modalities • RESTful HTTP • Command line • Procedural (Java, Perl, Ruby, …) Linked data Stable URL references The file system is the database State or content Storage node Object Version File Storage service http://example-store/ default/ 1234/ 3/ xyz state/

  7. “You say micro, I say macro…” Service Tool Convention Access ANVL ARK bagit.pl BagIt CAN checkm.pl Checkm Dflat ERC EZID GhOST Ingest Inventory LockIt N2T namaste.pl Namaste Noid Pairtree ReDD RUU Storage Access ANVL ARK BagIt CAN Checkm Dflat ERC EZID GhOST Ingest Inventory LockIt N2T Namaste Noid Pairtree ReDD RUU Storage

  8. Development roadmap

  9. Ingest process flow Create identifier Identity Identifier Submit Submitting user agent Ingest Node Addversion Notification Version metadata Getversion metadata Addversion Storage Node Notification Version metadata Getversion metadata Inventory Node Version metadata

  10. Ingest implementation Submitting user agent Ingest notification HTML form Submission notification Batch or single object Job metadata Submitter Queue Consumer Ingester Storage Servlet Implicitly multi-threaded Zookeeper dæmon Dæmon Explicitly multi-threaded Servlet Implicitly multi-threaded Job payload

  11. Questions? silverpipes.jog / firstpresmacomb.org

  12. More information UC Curation Center (UC3) http://www.cdlib.org/uc3 Micro-service specifications https://confluence.ucop.edu/display/Curation Digital curation group http://groups.google.com/group/digital-curation UC3 Stephen Abrams Erik Hetzner Margaret Low Mark Reyes Perry Willett Patricia Cruse Greg Janée David Loy Tracy Seneca Scott Fisher John Kunze Isaac Rabinovitch Marisa Strong

More Related