1 / 33

Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004

Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004. Il progetto EGEE Luciano Gaido. Il contesto. 01/01 – 03/04. 12/01 – 03/04. 2002-2005 + 2006-2008. Grid.it (FIRB). 01/03 – 12/05. 06/00 -. Risultati dei progetti precedenti. Release di middleware EDG 2.x

nickolas-macris
Download Presentation

Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004 Il progetto EGEE Luciano Gaido

  2. Il contesto 01/01 – 03/04 12/01 – 03/04 2002-2005 + 2006-2008 Grid.it (FIRB) 01/03 – 12/05 06/00 - ...... Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  3. Risultati dei progetti precedenti • Release di middleware • EDG 2.x • LCG 2.0 (basata su EDG 2.0) • INFN-GRID 2.0 (basata su LCG 2.0) • EGEE-0 (aka LCG 2.0) • Infrastrutture • Testbed EDG (development, application) • Testbed INFN-GRID • Testbed DataTAG • Grid di produzione italiana (INFN-GRID, GRID-IT) • Infrastruttura di produzione LCG Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  4. Le infrastrutture Mar 2004 Jan 2001 TB2 TB1 TB3 TB0 EDG 1.4 EDG 2 EDG 1 Globus 1 Assessment R&D 2001 2004 2002 2003 LCG-1 LCG-2 INFNGRID-1 INFNGRID-2 Figura presa dalla pres. di M. Parsons a Cork Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  5. Il progetto EGEE • Obiettivo: “EGEE aims to integrate current national, regional and thematic Grid efforts in order to create a seamless European Grid Infrastructure. This infrastructure will build on the underlying EU Research Network GEANT and exploit Grid expertise that has been generated by projects such as the European Union DataGrid project, other EU supported Grid projects and the national Grid initiatives such as UK e-Science, InfnGrid and NorduGrid.” • Iniziato il 1 aprile 2004, con durata biennale • Previsto un rinnovo per un secondo biennio • 70 partner, 32 MEuro • Kick-off meeting a Cork (19-22 aprile 2004) Applications Grid infrastr. Geant network Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  6. Organizzazione di EGEE • I 70 partner sono stati divisi in 12 federazioni: • CERN • Central Europe including Austria, Czech Republic, Hungary, • Poland, Slovakia and Slovenia • France • Germany and Switzerland • Ireland and the United Kingdom • Italy • Northern Europe including Belgium, Denmark, Finland, • the Netherlands, Norway and Sweden • Russia • South-East Europe including Bulgaria, Cyprus, Greece, Israel and • Romania • South-West Europe including Portugal and Spain • NRENS (National Research and Education Networks) • United States Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  7. Struttura di EGEE Tre filoni di attività: • Joint Research Activities: • JRA1 – Middleware Engineering and Integration • JRA2 – Quality Assurance • JRA3 – Security • JRA4 - Network Services Development • Service Activities: • SA1 – Grid Operation and Support • SA2 – Network Resource Provision • Networking Activities: • NA1 – Management of I3 (Integrated Infrastructure Initiative) • NA2 – Dissemination and Outreach • NA3 – User Training and Induction • NA4 – Application Identification and Support • NA5 – Internal Cooperation Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  8. Dalla pres. di Bob Jones a Cork EGEE Implementation • From day 1 (1st April 2004) Production grid service based on the LCG infrastructure running LCG-2 grid middleware (SA) LCG-2 will be maintained until the new generation has proven itself (fallback solution) • In parallel develop a “next generation” grid facility (JRA) Produce a new set of grid services according to evolving standards (Web Services) Run a development service providing early access for evaluation purposes Will replace LCG-2 on production facility in 2005 VDT EDG . . . LCG-1 LCG-2 EGEE-1 EGEE-2 AliEn LCG . . . Globus 2 based Web services based EGEE Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  9. Dalla pres. di Bob Jones a Cork EGEE and LCG EGEE builds on the work of LCG to establish a grid operations service • LCG: a worldwide collaboration of • The LHC experiments • The Regional Computing Centres • Physics institutes • Mission: • Prepare and deploy the computing environment that will be used by the experiments to analyse the LHC data • Strategy: • Integrate thousands of computers at dozens of participating institutes worldwide into a global computing resource • Rely on software being developed in advanced grid technology projects, both in Europe and in the USA • Status: • LCG service up and running with LCG-2 mware – successfully being used for LHC data challenges Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  10. JRA1: Middleware Engineering and Integration L’attività di sviluppo è distribuita su vari cluster: • UK • IT/CZ (F. Prelz, M. Sgaravatto) • CERN • Nordic Ogni cluster ha un suo development testbed distribuito L’integrazione e testing del m/w e lo sviluppo di tools è responsabilità del CERN. Del cluster italo/ceco fa parte anche Datamat Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  11. JRA1: Middleware Engineering and Integration Punto di partenza per la reingegnerizzazione del m/w: • Design e componenti grid basati su quanto deciso dall’RTAG* ARDA (A Realization of Distributed Analysis) • Design Team costituito da S. Fisher (UK), F. Prelz (IT/CZ), D. Groep (Nordic), M. Livny (VDT), P. Buncic (CERN), P. Kunszt (CERN), F. Hemmer (CERN) e E. Laure (CERN) • Responsabilità del CE (IT/CZ) • Esperienza e componenti da AliEn, VDT e EDG • Web Services * Requirements and Technical Assessment Group (LCG) Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  12. JRA1: Middleware Engineering and Integration • Key deliverables/milestones • Jun’04 Tools for mware engineering and integration deployed (MJRA1.1) • Jun’04Architecture and Planning Document - Rel 1 (DJRA1.1) • Jun’04 SW cluster develop. and testing infrastructure available (MJRA1.2) • Aug’04 Integration and testing infrastructure in place with test plans – Rel 1 (MJRA1.3) • Aug’04Design Document for grid services - Rel 1 (DJRA1.2) Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  13. Dalla pres. di R. Barbera a Cork NA4 Work Breakdown Structure http://alipc1.ct.infn.it/grid/egee/na4 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  14. Dalla pres. di R. Barbera a Cork NA4 Generic Applications goals • Activity NA4 focuses on the identification and support of early-user and established applications for use on the EGEE infrastructure. It has the following objectives: • To identify through the dissemination partners and a well defined integration process a portfolio of early user applications from a broad range of application sectors from academia, industry and commerce. • To support development and production use of all of these applications on the EGEE infrastructure and thereby establish a strong user base on which to build a broad EGEE user community. • To initially focus on two well-defined application areas – Particle Physics and Life sciences. • The expected outcome of the activity will be the establishment of a broad portfolio of applications across a wide range of sectors suited to execution on the EGEE infrastructure meeting the needs of a broad collection of user groups from many sectors across Europe as illustrated in Figure (note the timeline for the introduction of each application domain is purely illustrative). Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  15. Dalla pres. di R. Barbera a Cork Conclusions and outlook for GILDA • GILDA is a complete suite of grid elements (test-bed, CA, VO, monitoring system, web portal) and applications fully dedicated to dissemination purposes. • GILDA runs and will run the last production (stable) version of the grid middleware (currently Grid.it 2.0.0 based on LCG 2.0.0). • GILDA is the dissemination tool which will be used by NA3 during induction courses and tutorials (first in Edinburgh next week!). • Next future (2-3 weeks): • adding new sites: NeSC, InterQuanta (private company in Spain), any other interested; • finalize and advertise the “24x7” public “grid demonstrator”: a customized version of the full GENIUS web portal from where all users belonging to the GILDA VO can submit a pre-defined set of applications to the GILDA Testbed; • publish all the documentation on the GILDA web site. Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  16. Dalla pres. di R. Barbera a Cork The GILDA Tutorials (1/2) • Edinburgh, 7 April 2004 • Tunis, 22-23 April 2004 • Edinburgh, 26-28 April 2004, registration and program • CERN, 17-19 May 2004 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  17. SA1: obiettivi • Core Infrastructure services: • Operate essential grid services • Grid monitoring and control: • Proactively monitor the operational state and performance, • Initiate corrective action • Middleware deployment and resource induction: • Validate and deploy middleware releases • Set up operational procedures for new resources • Resource provider and user support: • Coordinate the resolution of problems from both Resource Centres and users • Filter and aggregate problems, providing or obtaining solutions • Grid management: • Coordinate Regional Operations Centres (ROC) and Core Infrastructure Centres (CIC) • Manage the relationships with resource providers via service-level agreements. • International collaboration: • Drive collaboration with peer organisations in the U.S. and in Asia-Pacific • Ensure interoperability of grid infrastructures and services for cross-domain VO’s • Participate in liaison and standards bodies in wider grid community Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  18. SA1: strutture (1) Numeri: RC (ad libitum), OMC (1), ROC (9), CIC (4) RC (Resource Centres): • Provide computing and storage resources for the grid OMC (Operation Management Center): • Manage the operations of the entire EGEE grid from a single centralised location (CERN) CIC (Core Infrastructure Center): • Provide the basic service infrastructure of the Grid • Operate the key services which connect users with resources • Support the Regional Operation Centers Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  19. SA1: strutture (2) ROC (Regional Operation Center): • Customize and certificate the middleware releases • Provide the middleware release and up-to-date documentation to a set of Resource Centers (RC) • Validate the set of grid services and components to form a functioning Grid to serve user communities • Validate installation/upgrade procedures and documentation • Develop/Run test suites for components and Grid services to certify a site installation • Refer and escalate middleware problems to developers • Collaborate with CIC to have monitoring systems and performance evaluator running • Define the tools to measure the service level provided by resource centers • Manage the access policies and sharing of the resources • Provide 24x7 support service for users and local administrators Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  20. SA1: distribuzione geografica delle strutture 10 federazioni: • CERN OMC + CIC • Francia ROC + CIC • Italia ROC + CIC • Germania e Svizzera ROC • UK e Irlanda ROC + CIC • Northern Europe ROC • South West Europe ROC • South East Europe ROC • Central Europe ROC • Russia ROC Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  21. SA1 in Italia: da dove partiamo La grid in italia (InfnGrid) esiste dal 2001: • E’ diventata una infrastruttura di produzione (Grid-It) nel 2003 • 22 siti operativi (alcuni fanno parte anche dell’infrastruttura LCG2) • Usata per le attività degli esperimenti (Data Challenge e produzioni): • CMS preproduction (INFN-GRID_1.0 aka LCG_1.0, winter 2003) • ALICE DC prima fase (febbraio-maggio 2004) • ATLAS DC e produzioni di VIRGO e BABAR in partenza • Prove di produzione di LHCb in corso • Release: si usa INFN-GRID 2.0.0 (basata su LCG_2.0.0, 100% compatibile) Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  22. http://grid-it.cnaf.infn.it Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  23. Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  24. Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  25. Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  26. Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  27. SA1 in Italia Chi siamo: • INFN-GRID • Firb (Grid.it) tramite INFN • ENEA • UniLE* • UniNA* • UniCal* * SPACI (SouthernPartnershipforAdvancedComputationalInfrastructures) Responsabili ROC italiano: C. Vistoli, L. Gaido Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  28. SA1-IT: da dove partiamo The existing italian Grid Operation Center is the first step towards the EGEE ROC + CIC. It will evolve according to the EGEE plans. • Existing teams: • Release and Documentation Team - “builds” the INFN-GRID release, starting from the LCG release and (up to now) - provides the LCFGng profiles - provides documentation and a sw repository • Central Management Team - Takes care of the deployment of the release, certificates the sites and manages the grid services Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  29. SA1-IT: da dove partiamo (cont.) • Existing teams (cont.): • Monitoring Team GridICE development and support Provides a customized view for • VO, • Sites, • Grid Operation Center (+ services), • Users • Support Team Provides support through a web portal (interim solution: OneorZero with a ticketing system and a knowledge base system) Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  30. SA1: dove andiamo Per costruire una grid europea (EGEE), risultato dell’interconnessione delle grid nazionali, le strutture di operation, management e supporto esistenti nelle differenti regioni/federazioni devono armonizzarsi ed integrarsi evolvendo in una struttura comune. Percorso: • definire un modello generale per ROC e CIC • identificare le differenti esigenze dei ROC (e quindi dei Rersouce Center) • Trovare il minimo comune denominatore (dei requirement) per definire il contenuto della release EGEE (in collaborazione con JRA1) • Organizzarsi perche’ il resto (quello che non farà parte delle release ufficiale) venga prodotto e supportato da SA1 e/o da uno o più ROC  fondamentale evitare divergenze Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  31. SA1: cosa stiamo facendo? I primi passi: • Continuiamo a gestire le infrastrutture nazionali esistenti • Creato gruppo ROC managers: • Riunione telefonica settimanale • Primo meeting ‘de visu’: milano 10-11 maggio • Questionario sulle piattaforme hw e sistemi operativi • Molte altre cose da definire..... • In via di definizione i ruoli di CIC e ROC • Definizione Execution Plan • Definizione dei diversi testbed necessari (vedi figura) Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

  32. EGEE Certification, Testing and Release Cycle JRA1 SA1 CERTIFICATION TESTING APP INTEGR DEPLOY SERVICES Integrate HEP EXPTS Basic Functionality Tests BIO-MED DEVELOPMENT & INTEGRATION UNIT & FUNCTIONAL TESTING Run Certification Matrix OTHER TBD DEPLOYMENT PREPARATION PRE-PRODUCTION PRODUCTION Run tests C&T suites Site suites APPS SW Installation Release candidate tag Certified release tag Deployment release tag Production tag Dev Tag

  33. Domande? Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

More Related