1 / 19

Principles for Sustainable Data Curation;

Principles for Sustainable Data Curation;. Steven Worley Computational and Information Systems Laboratory NCAR. Can Research Library Repositories Benefit from the Federal Lab Experience?. Topics. My perspective – Research Data Archive @ NCAR Principles for Sustainable Data Curation

eytan
Download Presentation

Principles for Sustainable Data Curation;

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Principles for Sustainable Data Curation; Steven Worley Computational and Information Systems Laboratory NCAR

  2. Can Research Library Repositories Benefit from the Federal Lab Experience?

  3. Topics • My perspective – Research Data Archive @ NCAR • Principles for Sustainable Data Curation • Stable Funding • Knowledgeable Staff • Robust Digital Storage • Protection from Loss • Data and Metadata Format • Partnerships • Data Management Evolution ARL, Leadership Fellows

  4. My perspective – Research Data Archive @ NCAR Core Data Categories • Operational and Reanalysis Model Outputs Meteorological and Oceanographic Observations Remote Sensing Observations • Topography, Bathymetry, Vegetation, and Land Use ARL, Leadership Fellows

  5. My perspective – Research Data Archive @ NCAR • Purposes • Support climate & weather research at NCAR and UCAR Universities • Extend data service worldwide • Basic Metrics • Established in 1960s • 600+ datasets, +4M files • +70 datasets growing daily - monthly ARL, Leadership Fellows

  6. My perspective – Research Data Archive @ NCAR ARL, Leadership Fellows

  7. Archiving • Metadata • Data Integrity • Preservation • Management • Supervision • Guidance • Integrity • Access • Archiving • Metadata • Data Integrity • Preservation • Curation • Steward-ship • Requests • and • Needs • Users • Data • Assistance • Feedback • US • International ARL, Leadership Fellows

  8. Sustainable Curation - Stable Funding Permits: • Flexibility • Evolution of data management to meet expectations • Holistic approach – not driven by narrowly defined projects • Take advantage of unplanned opportunities • Necessary to keep collection viable for long-term ARL, Leadership Fellows

  9. Sustainable Curation - Knowledgeable Staff Data domain knowledge enables: • Understand data and do integrity checks • Choose data organization to fit science discipline • Design appropriate access systems and do consulting Consistent staffing levels nurtures: • Professionals dedicated to best practices • Human-based knowledge cannot be under estimated ARL, Leadership Fellows

  10. Sustainable Curation – Robust Digital Storage Keep pace with digital media evolution: • Expect data migration every 2-5 years • Tape, disk capacity, etc. • Plan, test, and implement migration carefully • Mistakes are irrecoverable! • Use knowledgeable staff heavily Why evolve? • Users expect more data with faster access • Media will eventually fail ARL, Leadership Fellows

  11. Sustainable Curation – Protection from Loss Create backup data and test disaster recovery Why? • Physical failures • Environmental: Power outage, Fire, Flood, ….. • Hardware: Disk system failure, Tape degradation • Poor curation practices • Metadata loss • Accidental data over-writes and deletions • Solutions • Store backup at separate physical location • Treat metadata and data as equals - couple together ARL, Leadership Fellows

  12. Sustainable Curation – Protection from Loss ARL, Leadership Fellows

  13. Sustainable Curation – Protection from Loss RDA : 40% ARL, Leadership Fellows

  14. Sustainable Curation – Data and Metadata Format Formats are a serious consideration because: • Must maintain data access for long-term • How? • Insist that data and metadata are in standard formats • Avoid computer OS dependent formats • Worry about application driven formats • E.G.: .xls, .xlsx, .doc, .docx, .ppt, .pptx, etc. • Challenge; Scientist are reluctant to help • Curators nightmare; never ending data and metadata format diversity ARL, Leadership Fellows

  15. Sustainable Curation – Partnerships Science productivity is enhanced by partnerships • Open sharing of data and metadata • Relies heavily on standards • No one archive or repository can do it all • BUT, users need/want it all • Cost saving by sharing ARL, Leadership Fellows

  16. Data Management Evolution – Person-centric 1960s to 1990s ARL, Leadership Fellows

  17. Data Management Evolution – Metadata-centric 1990s – 2010s ARL, Leadership Fellows

  18. Summary: For Research Library Repositories Sustainable Data Curation Robust Digital Storage Knowledgeable Staff Stable Funding Protection from Loss Data/Metadata Format Partnerships ARL, Leadership Fellows

  19. Research Data Archive @ NCARhttp://dss.ucar.edu/worley@ucar.edu ARL, Leadership Fellows

More Related