1 / 15

PRACE-2IP WP10 - iRODS workshop iRODS projects @ CINES

PRACE-2IP WP10 - iRODS workshop iRODS projects @ CINES Gerard GIL (CINES) – gil@cines.fr (Linkoping 26-28 September 2012) 2012). CINES presentation CINES projects using iRODS Adonis Archive Replication ISAAC. 2/2/2010. 2.

yovela
Download Presentation

PRACE-2IP WP10 - iRODS workshop iRODS projects @ CINES

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PRACE-2IP WP10 - iRODS workshop iRODS projects @ CINES Gerard GIL (CINES) – gil@cines.fr (Linkoping 26-28 September 2012) 2012)

  2. CINES presentation • CINES projects using iRODS • Adonis • Archive Replication • ISAAC 2/2/2010 2 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  3. CINES is supervised and funded by the Ministry of • Higher Education and Research. • CINES islocated in Montpellier (South of France) • 30 yearsserving the National AcademicResearch • CINES provides the french public researchcommunitywithcomputingresources and services. • 50 persons : (technicians, engineers and administratives) 2 missions : • Digital preservation • High Performance Computing 2/2/2010 3 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  4. Digital preservation • In 2004 the CINES was given the mandate to provide long-term preservation capabilities for digital objects related to scientific and technical information. • Electronic PhD theses • Digitized publications • Multimedia pedagogics • Scientific datasets • National agreement : (Dec 2010) CINES has received the agreement by the French Institution of Archives for his Digital Archive Repository (PAC platform) • International certification process: CINES is one of the four pilot sites (UKDA, DNb, DANS) to test European Certification Framework for long term preservation (supported by European Commission) ISO certification • Member of EUDAT 4 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  5. High Performance Computing National HPC Center since 1980 • Scalar, vector and parallel processing • French T1 system : JADE • SGI altix ICE 8200 • 23040 intel core, 267 Tflops • 600 TB Lustre Filesystem • Accelators : GPU, CELL, FPGA, ... • PRACE member on behalf of GENCI 5 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  6. iRODS in CINES projects • ADONIS • Archives Replication • ISAAC 6 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  7. ADONIS ADONIS TGE (Very Large Infrastructure) TGE is in charge of building a digital infrastructure for: Unified Access to Digital Document and Data produced by « Human and Social Sciences » CRDO (Resource Center for Oral Data) Pilot project selected to validate options for « Oral data » (2010) 7 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  8. Transfer ArchivePlatform ADONIS deposit Synchronization iRODS ? Formats Conversions iRODS replication/ Synchronization Dissemination System • Faulttolerence • Integrity control • Resource management rules • High speed data transfers 8 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  9. ARCHIVES REPLICATION To complete its national agreement for Digital Archive, CINES has to provide a distant copy of the archives it manages. The Archives Replication Project has been defined to reach this goal. • Several solutions studied : Arcsys, iRODS, iSCSI, … • iRODS was selected : • Open solution • Metadata description • Resource management rules • Fault tolerence • Integrity control • Authorization management • … 9 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  10. ARCHIVES REPLICATION (cross-replication) CINES Archive system CINES distant storage resource CINES iRODS server other iRODS server CINES ZONE replication other Archive system other ZONE Other distant storage resource replication 10 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  11. ISAAC Information Scientifique Archivée Au CINES • Mid-term preservation of Scientific Data • 3 to 4 years preservation/archive; • Objectives : • preserve/archive for 3 to 4 years • Give additional time for the researcher to appraise the relevance / importance of the information • Put in place processes for scientific data valorization / preservation • This goes well beyond a simple storage or backup. • At the end of this 3-4 years period, two options : • Migration onto the long-term preservation platform (PAC) • Restitution to the producer/owner. 11 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  12. ISAAC TRANSFER INGEST • Document audit • Format validation • Metadata input • Unique persistent indetifier • Additional checks • Rights management ACCESS • Search on metadata • Files catalog • Rights management • File download DATA USER • Producer • Authorized user • Communities DATA PRODUCER Preservation context definition and service level agreeement. - Metadata; - File formats; • Knowledge base; • Etc. iRODS • Distributed data • Resource management rules • Metadata description • Versatile management for Big Data • Authorization management • Faulttolerence • Integritychecks • High speed data transfer • … STORAGE • Fixity checks • Replication • Event logging THEMATIC COMMITTEE EXPERTS GROUP 12 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  13. ISAAC DATA USER • Producer • Authorized user • Communities STORAGE Storage abstract Layer Ingest API Format validation Metadata management Integritychecks Rights management ………. STORAGE STORAGE WEB INTERFACE 13 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  14. ISAAC Information Scientifique Archivée Au CINES • A generic platform with a national/european scope • A prototype is being put in place for the PRECCINSTA datasets • Prediction and Control of Combustion Instabilities for industrial gas turbines • potentially 2TB of data, (up to 10 TB June 2012) • HDF5, Netcdf, XDMF • Developments based on standard, open technologies : iRODS, Java, PostgreSQL, OpenLDAP, 14 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

  15. Questions ? Informations : • http://www.cines.fr • rouchon@cines.fr(Digital Archive Dept. leader) 15 PRACE-2IP WP10 - iRODS workshop (Linkoping 26-28 September 2012)

More Related