1 / 26

Grid Computing at DESY

Grid Computing at DESY. Andreas Gellrich * DESY IT Group 1 st EGEE Course at DESY 24 March 2005 *http://www.desy.de/~gellrich/. Contents. Introduction Grid Projects at DESY Grid Infrastructure at DESY How to start Conclusions Aim of this talk is to give you a starting point at DESY!.

gzavala
Download Presentation

Grid Computing at DESY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Computing at DESY Andreas Gellrich* DESY IT Group 1st EGEE Course at DESY 24 March 2005 *http://www.desy.de/~gellrich/

  2. Contents • Introduction • Grid Projects at DESY • Grid Infrastructure at DESY • How to start • Conclusions Aim of this talk is to give you a starting point at DESY!

  3. Internet, Networks The Grid Dream Mobile Access Supercomputer, PC-Cluster G R I D M I D D L E W A R E Desktop Data Storage, Sensors, Experiments Visualizing

  4. Introduction • DESY is one of the world leading centers for research with particle accelerators and synchrotron light • DESY operates the electron-hadron storage ring HERA-II with three running experiments (H1, HERMES, ZEUS) • DESY is a Tier-0/1 centre for the HERA experiments • DESY has traditionally not been involved in CERN experiments and is therefore not automatically part of LCG; future? • DESY supports the Physics Institutes of the University of Hamburg in their CMS Grid activities • DESY has identified Grid as a strategic technology for the future

  5. Grid Projects at DESY

  6. Grid @ DESY • Grid activities were started mid 2003 by H1 and IT • A first Grid Testbed was based on EDG 1.4 and SuSE 7.2 / 8.2 • Open initiative to study the usage of Grids at DESY • Basic Grid functionalities were implemented and tested • Monte Carlo Production was identified as primer candidate • Since spring 2004 an LCG-2 Grid infrastructure in operation • ZEUS fostered MC production on the Grid for HERA-II • Most HEP sites are in LCG and promote their resources there • ZEUS acquired tens of Grid sites supporting the VO ‘zeus’ • The aim is to federate resources by opportunistically using them within DESY

  7. applications EGEE network EGEE @ DESY • Enabling Grids for E-SciencE • EU 6th Framework Programme (FP 6) • EGEE-1 started on April 1st, 2004 • 70 partners in 27 countries • federated in 10 regional Grids • Headquarter: CERN • DESY is in DE/CH federation • DESY, DKRZ, FhG-SCAI, FZK, GSI,CSCS • DESY is in SA1 (operating a Grid infrastructure) • DESY is funded with ~ 2FTEyears • EGEE-2 will start for 3 years at April 1st, 2006 • http://www.eu-egee.org/

  8. D-GRID @ DESY • R&D programme for a national German e-science infrastructure • DESY is founding member of the HGF institutes • Organization in 6 communities • Anticipated programme start is mid/end 2005, for 3 year • Funding volume of 15 MEUR • Currently pre-project phase • A handful of community specific projects and 1 integration project • DESY leads HEP Community Project (CP) • DESY participates in the Integration project (IP), led by GridKa • DESY brings in know-how and experiences in data management

  9. ILDG @ DESY • In Lattice QCD (LQCD) costly simulations so-called configurations, performed on High Performance Clusters are the basis for research activities • The International Lattice DataGrid (ILDG) was started with the aim of making gauge field configurations available to an international group of scientists using Grid technologies • Each configuration submitted to ILDG will consist of a set of meta-data and a set of binary files • In the context of the German Lattice Forum (LATFOR) DESY is setting up a Data Grid testbed, exploiting the common Grid infrastructure, in particular the catalogue services • R&D efforts at DESY Zeuthen / NIC

  10. LCG @ DESY • Some physics institutes of the University of Hamburg are located on the DESY site • DESY supports the institutes by providing computing infrastructure • The Institute for Experimental Physics is partner of the CMS collaboration at LHC • In a joint effort between DESY and the institute a Grid infrastructure to enable LCG on site will be installed • Their Grid hardware is incorporated in the common DESY Grid infrastructure • In cooperation with the University of Aachen a CMS Tier-2 centre is planned in Germany; ATLAS?

  11. dCache@DESY … • dCache provides data management software for DESY • dCache is Grid-aware; it supports Grid protocols such as srm and GridFTP • DESY’s entire data space is managed via dCache • dCache provides a Storage Element (SE) for the Grid • This SE brings DESY’s data to the Grid!

  12. … dCache@DESY … Write Pools Read Only Pools SRM Resilient Manager File system Engine Pool Manager Pnfs PM gsiFtp gsidCap http(sg) other plug-in

  13. SRM Client … dCache@DESY Pre-staging, Space Mgmt, Pinning Transfer Protocol Negotiation Store/Retrieve Data Transfer: gsiFTP, gsidcap Virtual Storage Layer SRM SRM SRM SRM Cache dCache C J a A s S m i T HSM n O OSM TSM Enstore Hpss e R

  14. Grid Infrastructure at DESY

  15. Grid Infrastructure at DESY … • DESY installed and operates a complete and independent Grid infrastructure which provides generic (non experiment specific) Grid services to all experiments and groups a DESY • The DESY Production Grid is based on LCG_2_2_0 and includes: • Resource Broker (RB), Information Index (BDII), Proxy (PXY) • Local Replica Catalog (LRC), Meta Data Catalog (MDC) • 26 nodes, incl. 18 WNs (36 CPUs) • dCache-based SE with access to the entire DESY data space of 0.5 PB • Certification services for DESY users in cooperation with GridKa • VO management for the HERA experiments (‘hone’, ‘herab’, ‘hermes’, ‘zeus’), LQCD (‘ildg’), ILC (‘ilc’, ‘calice’), Astro-particle Physics (‘baikal’, ‘icecube’)

  16. grid-cat grid-ui grid-rb LRC/MDC ssh JDL output grid-vo grid-ce srm-dcache world VO PBS RB CE SE WN WN WN WN grid100-111 … Grid Infrastructure at DESY … CAT UI RB BDII certs $HOME/.globus/ CE dCache- SE GIIS ldap://grid-vo.desy.de/ GRIS GRIS /etc/grid-security/grid-mapfile dcap disk tape Site A Site B Site C

  17. … Grid Infrastructure at DESY … • rack-mounted 1U servers • dual Intel P4 XEON 2.8 GHz • 2 GB DDRAM • GigaBit Ethernet • 80 GB (E)IDE system disk • 200 GB (E)IDE data disk • 10 Gbit/s DESY back-bone • 1 Gbit/s WAN (G-WIN)

  18. … Grid Infrastructure at DESY … grid-ce.desy.de

  19. … Grid Infrastructure at DESY

  20. ZEUS MC Production … Funnel Sites ZEUS Integrated Production System KEK Grid Sites Rutherford Montreal RAL Tel Aviv Bonn ZEUS Grid Gateway Scotgrid ... DESY …

  21. … ZEUS MC Production … • The Grid Sites are currently being integrated in the ZEUS Integrated MC Production system (ZIMP) in addition to funnel • ZIMP provides user request portals, book-keeping, statistics, and monitoring and plans for automated production services • Besides DESY, INFN, RAL, ScotGrid, U Dortmund, and U Karlsruhe currently support the VO ‘zeus’ • 130 M events have been produced on the Grid so far • 2/3 produced outside DESY • Data Quality Management (DQM) has been successfully applied RAL 21 Sep 2004

  22. … ZEUS MC Production

  23. How to Start

  24. Getting started • Become a Grid user with a certificate (authentication) • Become a member of a VO (authorization) • Remember what you learned in the EGEE Course • http://grid.desy.de/ • http://grid.desy.de/cert/ • http://grid.desy.de/user/

  25. Conclusions • DESY deploys a Grid Infrastructure, the DESY Production Grid • The DESY Production Grid includes all generic Grid services which are needed to make up a complete independent Grid • The Grid middleware LCG-2 is deployed • One common Grid infrastructure for all Grid activities at DESY • ZEUS has proven the usability of the Grid for HEP • For HERA experiments and major groups VOs exist • For DESY, the Grid has become a strategic technology since we believe future experiments will expect the Grid as a tool

  26. Grid @ Web • Grid Computing: • http://www.globus.org/ • http://www.eu-datagrid.org/ • http://cern.ch/lcg/ • http://www.eu-egee.org/ • http://d-grid.de/ • DESY Grid Web Site: • http://grid.desy.de/ • http://www.dcache.org/ • DESY Grid User Meeting (GUM): • http://grid.desy.de/gum/ (bi-weekly, Fri 15.00h, SmRm 2k) • mailto:grid-users@desy.de (Mailing List)

More Related