AutoArchive ?:The ADS and the SWORDARM project

1. AutoArchive?:The ADS and the SWORDARM project Catherine Hardman - Archaeology Data Service University of York White Rose/RoaDMap 24th May 2012 http://archaeologydataservice.ac.uk

2. Archiving Digital Data After 15+ years experience of digital archiving within archaeology, we are seeing a move toward some organisations submitting digital archives to the ADS as a norm. Southampton's Designated Archaeology Collections Programme Wessex Archaeology Image Archive

3. The origins of the SWORDARM project These, and other, projects have highlighted the need for an automated system for file upload and metadata deposition if the process of a scalable and sustainable digital archive is to become a reality. … a semi-automated, project-based system, that allows for the deposition of archaeological data and metadata through the use of a SWORD-style protocol. It will develop a costing module that will make the charging process more transparent for the community. It will streamline the ADS infrastructure making its procedures and, perhaps more significantly, make the future of the digital repository more sustainable in the long-term.

4. What will the automated ingest system do? Allow users to upload files directly to the ADS repository Building on existing structure it will link files to existing metadata from the online recording form for fieldwork projects (OASIS) Allow users to add additional and new metadata for projects Semi-automate the accessioning of files and associated metadata within the ADS Collections Management System, streamlining the archiving process Allow users to set up and manage ‘accounts’ that will allow for the creation of sample costing’s, will allow them to make informed decisions over the selection and retention strategies of the archive and provide for the semi-automated billing of charges.

5. Managing deposits: current workflow • Problems • Protracted • Physical movement of data and metadata is not ideal • Archive or metadata often incomplete = negotiation • Inefficient • Duplication of effort • Inflexible for user Dissemination Collection Management System Digital Archivist Archiving Manual input of metadata and data conversion

6. Managing deposits: SWORDARM workflow ‘Clearing House’ for data and metadata Sword-Arm System Dissemination Collection Management System Sword-Arm PCMS • Account Management • Upload data • Management of the deposit • Creation of collection and file level metadata • Costing module • Licensing • Virus check • Checksum • Creation of file level metadata (DROID) • Allocation of DOI • Simple automated conversions Once submitted the archive is passed to the archivist for validation Archiving

7. Why SWORD ? • SWORD, or Simple Web-service Offering Repository Deposit, Protocol • is a light-weight Atom publishing protocol, developed by JISC, that has been designed to facilitate the deposition of resources into remote scholarly repositories; this interoperable standard has consequently been used to alleviate the problem for digital archives in develop such web-based clients • developed out of an awareness of the trend within the digital community to standardise the outputs of digital repositories through OAI-PMH, DC metadata etc. BUT little or no standards for getting data into these repositories • Is currently utilised by DSpace, EPrints, Fedora, and Intralibraryrepositories • http://swordapp.org/ • http://www.ariadne.ac.uk/issue54/allinson-et-al/

8. SWORD Protocol SWORD Protocol But…its initial development stemmed from a requirement for the deposit of scholarly communications outputs into repositories – these typically being small text-based items the problem…much archaeological data is neither simple or text-based as a result…while the SWORD protocol can be used for many deposits… e.g. Grey Literature, and other text-based files…for more complex and large datasets a SWORD-style protocol will be implemented

9. Metadata generation

10. Cost Module Cost Module Unregistered Registered Sand Box Allowing users to ‘test’ various costing options SWORD ARM Formal ‘quote’ for archiving based on a per file basis • Cost control • Depositor control of archive allowing • involvement in selection and retention • of files • Manage multiple depositions • Pricing based upon ‘economies • of scale’ • Financial encouragement for use, • making it ‘cheaper’

11. Cost Control … we have tried to keep things simple with a start up fee covering elements of management and administration for each project and then a range of prices usually per file; the more complex the file type the more expensive to deposit. So we’d charge £1 for a .csv file and up to £6 for an .au audio file that takes a lot more time to check, document and preserve. Experience has taught that the only circumstance where we can really benefit from economies of scale is when dealing with image files and that too is taken into consideration within the charging model. ADS Blog March 21st 2012 http://archaeologydataservice.ac.uk/blog/sword-arm/

12. Outcomes • Extend the use of the SWORD protocol • Allow the ADS to enhance its data management structures, semi-automating metadata capture, file upload, and account management • Provide an exemplar of the integration of a charging module within data management infrastructures • Help researchers to deposit their research data • Provide case studies from four UK HE institutions for deposit in a repository using the SWORD client • Embed the outputs within a preservation infrastructure and facilitate the policy objectives of research councils and non HE public bodies to enable project-based charging and deposit. • Workshops: • August 2012 – Testing of demo version – University of York • March 2013 – Testing of full version by institutional managers

13. SomeUseful Links SWORD-ARM Information on theSWORD-ARM project http://archaeologydataservice.ac.uk/research/swordarm Keep up to date with SWORD-ARM through the project blog http://archaeologydataservice.ac.uk/blog/sword-arm/ The SWORD website For more information on the SWORD protocol http://swordapp.org/