1 / 13

QCDgrid status report

QCDgrid status report. UKQCD data grid Chris Maynard. QCDgrid outline . QCDgrid is primarily a data grid Store data online (disk) Security through redundancy Multiple copies of files in geographically distinct locations

marge
Download Presentation

QCDgrid status report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. QCDgrid status report UKQCD data grid Chris Maynard QCDgrid

  2. QCDgrid outline • QCDgrid is primarily a data grid • Store data online (disk) • Security through redundancy • Multiple copies of files in geographically distinct locations • Replica catalogue service (RC) links copies to logical file name (LFN) • Data access through metadata catalogue service (MC) • Configurations marked up in QCDml1.1 QCDgrid

  3. Machines – LHC terminology • Tier 1 system • 50TByte SAN system • QCDOC front end directly connected • Hold one copy of all UKQCD data • Machine exists, not yet part of grid • Tier 2 systems • Edinburgh, Liverpool, Southampton, Swansea • Linux server + RAID array disk • Now: 5 TByte. Upgrade to12.5 TByte • 50 TByte capacity + 50 TByte replicated QCDgrid

  4. Software • Middleware: Globus 2.4 • RC: Globus RC • MC: Native XML database eXist • Bespoke Control Thread (CT) • Command line client tools • GUI client tools • Metadata browser • Based on OGSA-DIA browser • Job submission tool QCDgrid

  5. Control Thread QCDgrid

  6. Command line tools • For grid administrator • Add/remove machine • Disable/enable machine • Add/remove users (see later) • Remove Data • For user • Add, retrieve, store locally, data • Job submission post-production on grid data • Remote machine not required to be QCDgrid QCDgrid

  7. QCDgrid GUI • Same functionality as command line • Metadata browser • Search and read metadata • Based on OGSA-DIA browser • Freely available from qcdgrid.forge.nesc.ac.uk • Undergoing modification to take account of split schema in QCDml • Demonstration • Can read MC without authentificaton QCDgrid

  8. MC Browser and QCDml • QCDml1.1 • Two schemata • Configuration XML • Ensemble XML • MD catalogue and browser being modified • QCDml split is logical split • Nearly always search on Ensembles not • Native XML database has few ensemble files • Relational DB good, Hierarchical DB Bad • Quick searching! QCDgrid

  9. Ensemble • Look for actions • Couplings • Volumes • Can then examine individual config XML QCDgrid

  10. Config XML • Can search and display for an individual config XML QCDgrid

  11. Download data • Can download data using GUI • Or extract markovChainLFN • Download data later from script for interactive job QCDgrid

  12. Authenticated access • QCDgrid data access • X509 certificate from trusted CA • Currently only UK e-Science CA • We can trust your CA! • QCDgrid is not closed • Not anonymous, but authenticated access. • ILDG data sharing • Technically could move to anonymous access • Require data sharing policy from ILDG board QCDgrid

  13. QCDgrid and ILDG • ILDG RC services are SRM • QCDgrid RC catalogue published as SRM • This is not yet implemented • QCDgrid will appear as single ILDG node • ILDG MD services • UKQCD configurations marked up in QCDml1.1 • MC catalogue public anonymous access • Aggregate to obtain ILDG MC QCDgrid

More Related