1 / 15

Tony Doyle

GridPP: the UK's contribution to the international collaboration building a worldwide Grid, the LHC Computing Grid GridPP – is the system usable?. Tony Doyle. Summary. GridPP runs a major part of the EGEE/LCG Grid, which supports ~3000 users

jewellvasquez
Download Presentation

Tony Doyle

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GridPP: the UK's contribution to the international collaboration building a worldwide Grid, the LHC Computing Grid GridPP – is the system usable? Tony Doyle

  2. Summary • GridPP runs a major part of the EGEE/LCG Grid, which supports ~3000 users • The Grid is not (yet) as transparent as end-users want it to be • The underlying overall failure rate is ~10% • User (interface)s, middleware and operational procedures (need to) adapt • (see talks by Dave Britton and Stephen Burke for more info. on performance and operations [now]) • Procedures to manage the underlying problems such that system is usable are highlighted Usable Systems

  3. “Active” User requires thousands of CPU hours EGEE CPU hours(1 April 2006 to 31 July 2006 ) 5 million hours Usable Systems

  4. Virtual Organisations • Users are grouped into Virtual Organisations • Users/VO varies from 1 to 806 members (and growing..) • Broadly four classes of VO • LHC experiments • EGEE supported • Worldwide (mainly non-LHC particle physics) • Local/regional e.g. UK PhenoGrid • Sites can choose which VOs to support, subject to MOU/funding commitments • Most GridPP sites support ~20 VOs • GridPP nominally allocates 1% of resources to EGEE non-HEP VOs • GridPP currently contributes 30% of the EGEE CPU resources Usable Systems

  5. User View? • Perspective matters • This talk is not • a usability survey • unbiased • representative • Straw poll • users overcame initial registration hurdles within ~two weeks • users adapt to Grid in (un-)coordinated ways • The Grid was sufficiently flexible for many analysis applications Usable Systems

  6. Analysis Object Data Analysis Object Data Analysis Object Data AOD Physics Analysis ESD: Data or Monte Carlo Event Tags Collaboration -wide Tasks Event Selection Calibration Data Analysis, Skims INCREASINGDATAFLOW Raw Data Analysis Groups Physics Objects Physics Objects Physics Objects Individual Physicists Physics Analysis Usable Systems

  7. User evolution Number of UK Grid users (exc. Deployment Team) Quarter: 05Q4 06Q2 06Q3 Value: 1342 1831 2777 Many EGEE VOs supported c.f. 3000 EGEE target Number of active users (> 10 jobs per month) Quarter: 05Q4 06Q1 06Q2 Value: 83 166 201 Fraction: 6.2% 11.0% Viewpoint: growing fairly rapidly, but not as active as they could be? depends on the “active” definition Usable Systems

  8. Know your users? UK-enabled VOs 806 atlas 763 dzero 577 cms 566 dteam 150 lhcb 131 alice 75 bio65 dteamsgm41 esr 31 ilc 27 atlassgm 27 alicesgm 21 cmsprg 18 atlasprg 17 fusn 15 zeus 13 dteamprg 13 cmssgm 11 hone 9 pheno 9 geant 7 babar 6 aliceprg 5 lhcbsgm 5 biosgm 3 babarsgm 2 zeussgm 2 t2k 2 geantsgm 2 cedar 1 phenosgm 1 minossgm 1 lhcbprg 1 ilcsgm 1 honesgm 1 cdf Usable Systems

  9. Scriptor Job details Logical Folders Job Monitoring Job builder Log window User Interface • The GUI is relatively low-level (jobs, file collections) • Dynamic panels for higher level functions Dockable windows Screenshot of the Ganga GUI Usable Systems

  10. ATLAS • GANGA software framework (jointly with LHCb) • data challenges • producing Monte Carlo data • 10 million CPU hours • per year • LHCb • DIRAC software to submit analysis jobs using Grid • 2006 analysis job completion efficiency improved to 91% • CMS • Monte Carlo production, data transfer, job submission • CMS transfers top a petabyte a month for the last three months Complex Applications Usable Systems

  11. Particle physicists collaborate, play roles and delegate e.g. “prg” production group “sgm” software group managers Underpinned by Memoranda of Understanding Current MoU signatories: China France Germany Italy India Japan Netherlands Pakistan Portugal Romania Taiwan UK USA Pending signatures: Australia Belgium Canada Czech Republic Nordic Poland Russia Spain Switzerland Ukraine Negotiation w.r.t. resource and service level WLCG MoU Usable Systems

  12. Resource allocation • Need to assign quotas and priorities to VOs and measure delivery • VOMS provides group/role information in the proxy • Tools to control quotas and priorities in site services being developed • So far only at whole-VO level • Maui batch scheduler is flexible, easy to map to groups/roles • Sites set the target shares • Can publish VO/group-specific values in GLUE schema, hence the RB can use them for scheduling • Accounting tool (APEL) measures CPU use at global level (UK task) • Storage accounting currently being added • GridPP monitors storage across UK • Privacy issues around user-level accounting, being solved by encryption Usable Systems

  13. User Support • Becoming vital as the number of users grows • But modest effort available in the various projects • Global Grid User Support (GGUS) portal at Karlsruhe provides a central ticket interface • Problems are categorised • Tickets are classified by an on-duty Ticket Process Manager, and assigned to an appropriate support unit • UK (GridPP) contributes support effort • GGUS has a web-service interface to ticketing systems at each ROC • Other support units are local mailing lists • Mostly best-effort support, working hours only • Currently ~tens of tickets/week • Manageable, but may not scale much further • Some tickets slip through the net Usable Systems

  14. Documentation & Training • Need documentation and training for both system managers and users • Mostly expert users up to now, but user community is expanding • Induction of new VOs is a particular problem – no peer support • EGEE is running User Fora for users to share experience • Next in Manchester in May ’07 (with OGF) • EGEE has a dedicated training activity run by NeSC/Edinburgh • Documentation is often a low priority, little dedicated effort • The rapid pace of change means that material requires constant review • Effort on documentation is now increasing • GridPP has appointed a documentation officer • GridPP web site, wiki • Installation manual for admins is good • There is also a wiki for admins to share experience • Focus is now on user documentation • New EGEE web site – coming soon Usable Systems

  15. Alternative view? • The number of users in the Grid School for the Gifted is ~manageable now • The system may be too complex, requiring too much work by the “average user”? • Or the (virtual) help desk may not be enough? • Or the documentation may be misleading? • Or.. • Having smart users helps (the current ones are) Usable Systems

More Related