1 / 13

SRB and UK e-science

SRB and UK e-science. L. Blanshard, R. Downing, G. Drinkwater, D. Hanlon, K. Kleese van Dam, L. Roberts, R. Tyer CCLRC e-science Daresbury Lab P. Berrisford, G. Brown, K. Haines, C. Moreton-Smith, A. Hasan CCLRC e-science Rutherford Lab. UK e-science.

danyl
Download Presentation

SRB and UK e-science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SRB and UK e-science L. Blanshard, R. Downing, G. Drinkwater, D. Hanlon, K. Kleese van Dam, L. Roberts, R. Tyer CCLRC e-science Daresbury Lab P. Berrisford, G. Brown, K. Haines, C. Moreton-Smith, A. Hasan CCLRC e-science Rutherford Lab

  2. UK e-science • Tasked with providing IT infrastructure and tools to allow UK scientist opportunity to maximally exploit data. • Data Management Group essential in this effort. • SRB provides underlying management of data for practically all data management systems.

  3. e-minerals/e-materials • GSI-enabled mini-grid for Earth scientists and Chemists. • Primarily simulations on Condor pool and PBS clusters. • SRB holds simulated data, logical names meaningful to scientists. • CCLRC-developed data portal used to locate data and submit jobs.

  4. e-minerals/e-materials

  5. e-minerals/e-materials • Deployed late 2003. • Recently successfully migrate from SRB 3.2 to 3.3.1. • Stored more than 200,000 files. • Total of more than 160GBs data.

  6. ISIS and Diamond • ISIS - neutron and muon facility at Rutherford Lab. • Used by Physicists, Geologists, Biologists, engineers. • Data stored on NT disk farm, backed up to RAL tape store. • SRB used to archive data to tape store. • More than 5600 files in 700 containers stored. • Currently testing 3.4, stored more than 2TB in test system.

  7. ISIS and Diamond • Diamond - a synchrotron light source at RAL. • Used by scientists from many fields. • Expected online in 2007. • Planning to use SRB to manage data from data acquisition through to data archival. • Expect approx 1TB/day recorded by experiments. Mainly small files. • Anticipate federation with ISIS data.

  8. ISIS and Diamond

  9. NGS and IB • National Grid Service (NGS): • In production since 2004. • Provides computing and storage for many UK activities. • Uses GSI-enabled SRB for data management. • More than 800,000 files stored in SRB amounting to 800GB of data.

  10. NGS and IB • Integrative Biology (IB) project: • IT infrastructure to model biological processes. • Requires access controlled secure data management. • Uses NGS GSI-enabled SRB for storage of simulated data. • More than 300GB data stored in SRB.

  11. NGS and IB

  12. Experience • Some projects use Scommands and some Jargon. • All use basic commands (or APIs) to build apps tailored to the projects needs. • Many projects require additional metadata catalogue. Make use of the CCLRC data portal to access both catalogue and SRB. • Lots of small files, bulk operations important. • As more systems go to production migration from one release to another has to be smooth (req backwards compatibility).

  13. Summary • UK e-science is making good use of SRB to provide data management for a variety of projects. • Production experience from many of the projects is fed-back to SRB developers making for better and better SRB releases.

More Related