1 / 21

e-Infrastructure to enable EO and Climate Science

e-Infrastructure to enable EO and Climate Science. Dr Victoria Bennett Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk. What is CEDA. The Centre for Environmental Data Archival

chakra
Download Presentation

e-Infrastructure to enable EO and Climate Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. e-Infrastructure to enable EO and Climate Science Dr Victoria Bennett Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk

  2. What is CEDA • The Centre for Environmental Data Archival • Serves the environmental science community through 4 data centres and involvement in a host of projects • www.ceda.ac.uk

  3. Centre for Environmental Data Archival CEDA Data

  4. Centre for Environmental Data Archival CEDA Users

  5. Centre for Environmental Data Archival CEDA Users

  6. e-Infrastructure e-Infrastructure Investment JASMIN CEMS

  7. JASMIN-CEMS Headlines • 4.6 Petabytes of “fast” disk – with excellent connectivity • A compute platform for running Virtual Machines • A small HPC compute cluster (known as “LOTUS”) • Connected to • CEMS infrastructure in ISIC for commercial applications • JASMIN nodes at remote sites • Dedicated network connections to specific sites

  8. CEMS– what is it • A joint academic-industrial facility for climate and environmental data services • Will provide: • Step change in EO/climate data storage, processing and analysis • A scalable model for developing services and applications : hosted in a cloud-based infrastructure • Data quality and integrity tools • Information on data accuracy and provenance • To give users confidence in the data, services and products

  9. CEMS • More in these presentations: • Sam Almond, 16:20 today, TOPSIG session, “The Role of the ISIC CEMS facility in the Development of Quality Assured Datasets and Downstream Services from EO Data” • Victoria Bennett, 16:30 tomorrow, Data Facilities session, “The Facility for Cimate and Environmental Monitoring from Space (CEMS)”

  10. JASMIN/CEMS Data 1.0 Pb 4.6 Pb

  11. JASMIN and CEMS functions

  12. JASMIN-CEMS Science Use cases • Processing large volume EO datasets to produce: • Essential Climate Variables • Long term global climate-quality datasets • EO data validation & intercomparisons • Evaluation of models relying on the required datasets (EO datasets & in situ ) and simulations) being in the same place

  13. JASMIN-CEMS Science Use cases • User access to 5th Coupled Model Intercomparison Project (CMIP5) • Large volumes of data from best climate models • Greater throughput required • Large model analysis facility • Workspaces for scientific users. Climate modellers need 100s of Tb of disk space, with high-speed connectivity • UPSCALE project • Shipping ~5 Tb/dayto JASMIN from HERMIT (Germany), expecting 250 Tb in total • 2 VMs built and available to analyse the data • Large cache on fast disk available for post-processing results

  14. JASMIN/CEMS kit

  15. JASMIN locations JASMIN-North University of Leeds 150 Tb JASMIN-Core STFC RAL 3.5 Pb + compute JASMIN-West University of Bristol 150 Tb JASMIN-South University of Reading 500 Tb + compute

  16. JASMIN links

  17. JASMIN-CEMS Latest Status • 5th Sept 2012: • 62 Virtual Machines created (40 JASMIN, 22 CEMS) • Approx 375 Tb (of ~1.2PB) data migrated to Panasas storage • First users on the system : trial data processing, large volume data downloads (>100TB UPSCALE Data), group workspaces

  18. Thank you!

  19. JASMIN kit • JASMIN/CEMS Facts and figures • JASMIN: • 3.5 PetabytesPanasas Storage • 12 x Dell R610 (12 core, 3.0GHz, 96G RAM)Servers • 1 x Dell R815 (48 core, 2.2GHz, 128G RAM)Servers • 1 x Dell Equalogic R6510E (48 TB iSCSI VMware VM image store) • VMWarevSphereCenter • 8 x Dell R610 (12 core, 3.5GHz, 48G RAM) Servers • 1 x Force10 S4810P 10GbE Storage Aggregation Switch

  20. JASMIN kit • JASMIN/CEMS Facts and figures • CEMS: • 1.1 PetabytesPanasas Storage • 10 x Dell R610 (12 core 96G RAM) Servers • 1 x Dell Equalogic R6510E (48 TB iSCSI VMware VM image store) • VMWarevSphereCenter + vCloud Director

  21. JASMIN kit • JASMIN/CEMS Facts and figures • Complete 4.5 PB (usable - 6.6PB raw) Panasas storage managed as one store, consisting of: • 103 4U “Shelves” of 11 “Storage Blades” • 1,133 (-29) “Storage Blades” with 2x 3TB drives each • 2,266 3.5" Disc Drives (3TB Each) • 103 * 11 * 1 -29 = 1,104 CPUs (Celeron 1.33GHz CPU w. 4GB RAM) • 29 “Director Blades” with Dual Core Xeon 1.73GHz w.8GB RAM) • 15 kW Power in / heat out per rack = 180 kW (10-20 houses worth) • 600kg per rack = 7.2 Tonnes • 1.03 Tb/s total storage bandwidth = Copying 1500 DVDs per minute • 4.6PB Useable == 920,000 DVD's = a 1.47 km high tower of DVDs • 4.6PB Useable == 7,077,000 CDs = a 11.3 km high tower of CDs

More Related