1 / 15

GridPP: status report

GridPP: status report. Andy Parker (Taken from GridPP report at OSC by Tony Doyle). What was GridPP1?. A team that built a working prototype grid of significant scale > 2,000 (9,000) CPUs > 1,000 (5,000) TB of available storage > 1,000 (6,000) simultaneous jobs

xanthe
Download Presentation

GridPP: status report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GridPP: status report Andy Parker(Taken from GridPP report at OSC by Tony Doyle) Oversight Committee

  2. What was GridPP1? • A team that built a working prototype grid of significant scale > 2,000 (9,000) CPUs > 1,000 (5,000) TB of available storage > 1,000 (6,000) simultaneous jobs • A complex project where 88% of the milestones were completed and all metrics were within specification A Success “The achievement of something desired, planned, or attempted” Oversight Committee

  3. What is GridPP2? Structures agreed and in place (except LCG phase-2) • 253 Milestones, 112 Monitoring Metrics at present. • Must deliver a “Production Grid”: robust, reliable, resilient, secure, stable service delivered to end-user applications. • The Collaboration aims to develop, deploy and operate a very large Production Grid in the UK for use by the worldwide particle physics community. Oversight Committee

  4. LCG Overview • By 2007: • 100,000 CPUs • - More than 100 institutes worldwide • building on complex middleware being developed in advanced Grid technology projects, both in Europe (Glite) and in the USA (VDT) • prototype went live in September 2003 in 12 countries • Extensively tested by the LHC experiments during this summer Oversight Committee

  5. LHCb Data Challenge 186 M Produced Events Phase 1 Completed 3-5 106/day LCG restarted LCG paused LCG in action 1.8 106/day DIRAC alone 424 CPU years (4,000 kSI2k months), 186M events • UK’s input significant (>1/4 total) • LCG(UK) resource: • Tier-1 7.7% • Tier-2 sites: • London 3.9% • South 2.3% • North 1.4% • DIRAC: • Imperial 2.0% • L'pool 3.1% • Oxford 0.1% • ScotGrid 5.1% Entering Grid Production Phase.. Oversight Committee

  6. Paradigm ShiftTransition to Grid… 424 CPU · Years May: 89%:11% 11% of DC’04 Jun: 80%:20% 25% of DC’04 Jul: 77%:23% 22% of DC’04 Aug: 27%:73% 42% of DC’04 Oversight Committee

  7. Issues “LCG-2 MIDDLEWARE PROBLEMS AND REQUIREMENTS FOR LHC EXPERIMENT DATA CHALLENGES” First large-scale Grid production problems being addressed… at all levels https://edms.cern.ch/file/495809/2.2/LCG2-Limitations_and_Requirements.pdf Oversight Committee

  8. UK Tier-1/A Centre Rutherford Appleton Laboratory Grid Resource Discovery Time = 8 Hours • High quality data services • National and international role • UK focus for international Grid development 1000 CPU 200 TB Disk 60 TB Tape (Capacity 1PB) 2004 Disk Use 2004 CPU Utilisation Oversight Committee

  9. UK Tier-2 Centres ScotGrid Durham, Edinburgh, Glasgow NorthGrid Daresbury,Lancaster, Liverpool, Manchester, Sheffield SouthGrid Birmingham, Bristol, Cambridge, Oxford, RAL PPD,Warwick LondonGrid Brunel, Imperial, QMUL, RHUL, UCL Oversight Committee

  10. * Leeds Manchester * * DL * Oxford RAL * • In future will include services to facilitate collaborative (grid) computing • Authentication (PKI X509) • Job submission/batch service • Resource brokering • Authorisation • Virtual Organisation management • Certificate management • Information service • Data access/integration • (SRB/OGSA-DAI/DQPS) • National Registry (of registry’s) • Data replication • Data caching • Grid monitoring • Accounting Level-2 Grid Oversight Committee

  11. Application Development ATLAS LHCb CMS SAMGrid (FermiLab) BaBar (SLAC) QCDGrid PhenoGrid Oversight Committee

  12. More Applications • ZEUS uses LCG • needs the Grid to respond to increasing demand for MC production • 5 million Geant events on Grid since August 2004 • QCDGrid • For UKQCD • Currently a 4-site data grid • Key technologies used • - Globus Toolkit 2.4 • - European DataGrid • eXist XML database • managing a few hundred gigabytes of data Oversight Committee

  13. Disseminationmuch has happened.. more people are reading about it.. LHCb-UK members get up to speed with the Grid Wed 5 Jan 2005GridPP in Pittsburgh Thu 9 Dec 2004GridPP website busier than ever Mon 6 Dec 2004Optorsim 2.0 released Wed 24 Nov 2004ZEUS produces 5 million Grid events Mon 15 Nov 2004CERN 50th anniversary reception Tue 26 Oct 2004GridPP at CHEP'04 Mon 18 Oct 2004LHCb data challenge first phase a success for LCG and UK Mon 4 Oct 2004Networking in Nottingham - GLIF launch meeting Mon 4 Oct 2004GridPP going for Gold - website award at AHM Mon 6 Sep 2004GridPP at the All Hands Meeting Wed 1 Sep 2004R-GMA included in latest LCG release Wed 18 Aug 2004LCG2 administrators learn tips and tricks in Oxford Tue 27 Jul 2004Take me to your (project) leader Fri 2 Jul 2004ScotGrid's 2nd birthday: ScotGrid clocks up 1 million CPU hours Fri 25 Jun 2004Meet your production manager Fri 18 Jun 2004GridPP10 report and photographs Wed 9 Jun 2004CERN recognizes UK's outstanding contribution to Grid computing Wed 2 Jun 2004UK particle physics Grid takes shape Wed 19 May 2004A new monitoring map for GridPP Mon 10 May 2004Press reaction to EGEE launch Tue 4 May 2004GridPP at the EGEE launch conference Tue 27 Apr 2004LCG2 released Thu 8 Apr 2004University of Warwick joins GridPP Thu 8 Apr 2004 Grid computing steps up a gear: the start of EGEE Thu 1 Apr 2004EDG gets glowing final review Mon 22 Mar 2004Grids and Web Services meeting, 23 April, London Tue 16 Mar 2004EU DataGrid Software License approved by OSI Fri 27 Feb 2004GridPP Middleware workshop, March 4-5 2004, UCL Fri 20 Feb 2004Version 1.0 of the Optorsim grid simulation tool released by EU DataGrid Tue 17 Feb 2004Summary and photographs of the 9th GridPP Collaboration Meetin Thu 12 Feb 2004 138,976 hits in December Oversight Committee

  14. Grid and e-Science Support in 2008 I. Experiment Layer II. Application Middleware III. Grid Middleware IV. Facilities and Fabrics What areas require support? IV Running the Tier-1 Data Centre IV Hardware annual upgrade IV Contribution to Tier-2 Sysman effort  (non-PPARC) hardware IV Frontend Tier-2 hardware IV Contribution to Tier-0 support III One M/S/N expert in each of 6 areas III Production manager and four Tier-2 coordinators II Application/Grid experts (UK support) I ATLAS Computing MoU commitments and support I CMS Computing MoU commitments and support I LHCb Core Tasks and Computing Support I ALICE Computing support I Future experiments adopt e-Infrastructure methods • No GridPP management: (assume production mode established + devolved management to Institutes) Oversight Committee

  15. OSC actions • Plan for support of physics analysis community in 2007 • Better understanding of capacity limits at Tier1 and strategic priorities • £1M bid for LCG2 to go to PPARC Science Committee • Plan to raise job completion efficiency • Statement on gLite • OSC views project as excellently managed and very successful Oversight Committee

More Related