1 / 15

The Ocean, the Atmosphere, and the Grid A UK perspective

The Ocean, the Atmosphere, and the Grid A UK perspective. David Webb Southampton Oceanography Centre. Natural Environment Research Council. San Francisco 4/5th August 2001. ECMWF Winds. Ocean Models. Satellite Observations. Alace floats. Moorings. Ship observations.

mariko
Download Presentation

The Ocean, the Atmosphere, and the Grid A UK perspective

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Ocean, the Atmosphere, and the GridA UK perspective David Webb Southampton Oceanography Centre NaturalEnvironmentResearchCouncil San Francisco 4/5th August 2001

  2. ECMWF Winds Ocean Models Satellite Observations

  3. Alace floats Moorings Ship observations Autonomous vehicles Bugs

  4. The Oceanographers View ...

  5. The user wants ... Seamless movement from data search to data extraction to visualisation to comparison with other data to analysis to … and back again.

  6. Some of the problems ... • Very large individual data files - up to 8 Gb • Large numbers of files - i.e. one every six hours • Very diverse data We need: - Quick look data - Sub-samples in space and time - Track local and cache copies - Map conversions We want to: - Inter-compare 2-D and 3-D fields - Overlay station data on 2-D and 3-D fields - Simplify access -Use browsers -Move logic to the user.

  7. Outline Data Grid Middleware + Agents Users Data Centres Browser C/Fortran WWW Metadata SOC HPC GIS Cache BODC 3D System Farm Cache BADC Data Source

  8. Organisation Data Centres Middleware + Agents User interface • Metadata • Data • Cached data • Model Grids • Algorithms • Formats + conventions • HDF • netCDF/ferret • Retrieval • Speed • Delays • Pre-processing • Sub-sampling • Data compression • Extreme events • (Do you want 200 GB?) • Simple data transforms • Change grid • Sub-sample • Interpolate • Formats • Complex transforms • Density/vorticity/etc • Extreme events • Heat fluxes • Transform metadata • Logical operations • Hunt for related data • Data quality tests • Handle caching/delays • Browser (Netscape) and anonymous users • Fortran/C program • Matlab /GIS • Handle caching/delays • Data Sources …and W3C compliant

  9. The Atmospheric Data Centre View ...

  10. Computation Graphics 1 Database Client 2 BADC Data & Catalogue Computation Graphics Database Database Client 3 BADC- Migrating from “downloads” to E-Science At BADC At USER Institution In the beginning, users downloaded files, we are about to provide tools at the data centre, which will eventually be “griddable”.

  11. Accessing Atmospheric Data at the BADC via Grid Technologies New Concept: The Catalogue Interface Resource Broker (more than just the SRB) • The CIRB would provide access to data both within the BADC and in other locations. It would be “cache aware”. • No one data centre would be the unique CIRB, rather each would be a peer in a network of CIRBs fronting each data source. • The CRIB will provide access to the data lying in the other data stores. Security and resource issues will be dealt with by “e-science” software agents. • Users would be able to access data via a GUI on their own machine, on our systems, or even via user written APIs.

  12. Possible US Links: NCAR - Atmospheric Data NOAA - Ocean and Atmospheric Data Los Alamos - Ocean Model Data LLNL - Climate Change Model Data Sets

  13. Summary • The data grid needs: • Dataset sub-volumes • Dataset sub-sampling • Data cache and cache metadata • Conversion and compression • User software to seamlessly integrate • Data discovery from many data centres • Plotting/comparison of data • As GIS, Browser and subroutines • The compute grid needs: • link to compute portals • access and security • libraries • scheduling for • - ensemble experiments • - loose coupled models • - fully coupled models • automatic cataloguing • globally accessible databases • code maintenance San Francisco 4/5th August 2001

More Related