1 / 34

Griding the Nordic Supercomputer Infrastructure The importance of national and regional Grids

Griding the Nordic Supercomputer Infrastructure The importance of national and regional Grids. Conference xxx - August 2003. Anders Ynnerman Director Swedish National Infrastructure for Computing. Research Infrastructures – e-Infrastructure: Grid initiatives 27 May 2005.

yale
Download Presentation

Griding the Nordic Supercomputer Infrastructure The importance of national and regional Grids

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Griding the Nordic Supercomputer InfrastructureThe importance of national and regional Grids Conference xxx - August 2003 Anders Ynnerman Director Swedish National Infrastructure for Computing Research Infrastructures – e-Infrastructure: Grid initiatives 27 May 2005

  2. Outline of presentation • The Nordic HPC landscape • SweGrid testbed for production • Benefits of a national Grid effort • NorduGrid - ARC • Nordic DataGrid Facility INFO Event, May 26th 2005 - 2

  3. HPC program initiated 2001 Danish Center for Scientific Computing (DCSC) Distributed program Location of facilities based on scientific evaluation of participating groups not on computer centers or infrastructure Århus Odense DTU Copenhagen University Budget 2.1 MEuros (2001) 1.6 MEuros (2002) 1.5 MEuros (2003) 1.4 MEuros (2004) Copenhagen University is a partner in NorduGrid Danish Center for Grid Computing (DCGC) Virtual center located at 5 sites Main location at the Niels Bohr Institute, Copenhagen University 0.75 MEuros/year Operations and support of a Danish Grid Research agenda with 10 senior researchers and 5 PhD students Strong ties to NorduGrid and Nordic DataGrid Faicility Participation in EGEE through NBI HPC-infrastructure GRID-initiatives DENMARK INFO Event, May 26th 2005 - 3

  4. NOTUR – The Norwegian High-Performance Computational Infrastructure Trondheim Oslo Bergen Tromsö Budget 168 MEuros (2000-2003) 50% from reserach councils Statoil, DNMI In kind contributions New program similar is being initiated and placed under the NREN (UNINETT) Metacenter goal – To establish a uniform and as seamless as possible, access to HPC resources Parallab is contributing to EGEE security team Bergen and Oslo are partners in NorduGrid NORGRID project has been initiated under the umbrella of NOTUR HPC-infrastructure GRID-initiatives NORWAY INFO Event, May 26th 2005 - 4

  5. Center for Scientific Computing (CSC) is the (main) center in Finland The largest HPC center in Scandinavia Runs services for academic users all over Finland and FMI Budget Directly from the ministry of education Smaller university HPC-systems (clusters) are beginning to appear CSC is contributing resources to NorduGrid CSC is a partner in DEISA Helsinki Institute of Physics is running several GRID projects and is contributing to the EGEE security team A Finnish Grid dedicated to Condensed Matter Physics – example of topical Grids HPC-infrastructure GRID-initiatives FINLAND INFO Event, May 26th 2005 - 5

  6. Swedish National Infrastructure for Computing (SNIC) formed during 2003 6 participating centers Umeå Uppsala Stockholm Linköping Göteborg Lund National resource allocation (SNAC) Budget 4.5 MEuros/year (Government) 4.0 MEuros/year private foundations Lund and Uppsala are partners in NorduGrid Large resource contributions to NorduGrid are SNIC clusters at Umeå and Linköping SweGrid production Grid (600 CPUs) over 6 sites 120 Tb storage SNIC is hosting the EGEE Nordic Regional Operations Center Stockholm (PDC) is coordinating the EGEE security activity PDC is a partner in the European Grid Support Center HPC-infrastructure GRID-initiatives SWEDEN INFO Event, May 26th 2005 - 6

  7. Grids in the context of HPC centers Parallel Vector Processors Large Shared Memory Systems Price/Performance Clusters with High Speed Interconnect Clusters with Ethernet Loosely coupled workstations Throughput computing on Grids 1/ No of Users INFO Event, May 26th 2005 - 7

  8. SweGrid production testbed • The first step towards HPC center Gridification • Initiative from • All HPC-centers in Sweden • IT-researchers wanting to research Grid technology • Users • Life Science • Earth Sciences • Space & Astro Physics • High energy physics • PC-clusters with large storage capacity • Build for GRID production • Participation in international collaborations • LCG • EGEE • NorduGrid • … INFO Event, May 26th 2005 - 8

  9. SweGrid subprojects 2.5 MEuro 6 PC-clusters 600 CPUs for throughput computing 0.25 MEuro/year 6 Technicians Forming the core team for the Northern EGEE ROC • 0.25 MEuro/year • Portals • Databases • Security • Globus Alliance • EGEE - security INFO Event, May 26th 2005 - 9

  10. SweGrid production test bed • Total budget 3.6 MEuro • 6 GRID nodes • 600 CPUs • IA-32, 1 processor/server • 875P with 800 MHz FSB and dual memory busses • 2.8 GHz Intel P4 • 2 Gbyte • Gigabit Ethernet • 12 TByte temporary storage • FibreChannel for bandwidth • 14 x 146 GByte 10000 rpm • 200 TByte nearline storage • 140 TByte disk • 270 TByte tape • 1 Gigabit direct connection to SUNET (10 Gbps) INFO Event, May 26th 2005 - 10

  11. 2.5 Gbit/s 10 Gbit/s Univ. LAN SweGrid 1 Gbps Dedicated SUNET connectivity GigaSunet 10 Gbit/s Typical POP at Univ. INFO Event, May 26th 2005 - 11

  12. Persistent storage on SweGrid? 2 1 3 Bandwidth Availability Size Administration INFO Event, May 26th 2005 - 12

  13. SweGrid status • All nodes installed during January 2004 • Extensive use of the resources already • Local batch queues • GRID queues through the NorduGrid middlware – ARC • Some nodes also available on LCG • 60 national users • 1/3 of SweGrid is dedicated to HEP (200 CPUs) • Contributing to Atlas Data Challenge 2 • As a partner in NorduGrid • Also supporting LCG (gLite) • Investigating compatibility between ARC and LCG • Forms the core of the Northern EGEE ROC • Accounting is being introduced - SGAS INFO Event, May 26th 2005 - 13

  14. The first users of SweGrid INFO Event, May 26th 2005 - 14

  15. How have you found porting your applications to SweGrid to be? INFO Event, May 26th 2005 - 15

  16. What is your overall impression of use of SweGrid resources? INFO Event, May 26th 2005 - 16

  17. Do you think all supercomputing resources should be available on a Grid? INFO Event, May 26th 2005 - 17

  18. SweGrid results • Unified the Swedish Supercomputing Community • Raised additional private funding for computing infrastructure • Spawned national Grid R&D projects • Created early user Grid awareness • Created an interface to EU-projects • EGEE • Regional operations center co-operated with SweGrid • Co-ordination of EGEE security • DEISA participation through PDC at KTH • Co-ordination of Baltic Grid Initiative INFO Event, May 26th 2005 - 19

  19. National projects and services • Grid resource brokering, Umeå • Grid accounting, SweGrid Accounting System (SGAS), Stockholm, Uppsala, Umeå • Semi-automatic Grid interface-generation for numerical software libraries, Umeå • Grid security research – focused on trust models, Stockholm, Umeå and Uppsala • Parallel Object Query System for Expensive Computations (POQSEC) on Grids, Uppsala • National Helpdesk, all SweGrid centers • National storage solutions, Linköping, Stockholm, Umeå INFO Event, May 26th 2005 - 20

  20. North European EGEE ROC (SA1) • SweGrid forms the core with three sites • HPC2N • 100 CPU cluster • LCG2.4, SL3 CE, Debian WN • SE: 10 TByte • Worker nodes are shared between EGEE and SweGrid (ARC) • PDC • Migration of SweGrid 100 CPU system to EGEE • Eventually new 884 CPU Xeon EM64T cluster • NSC • 32 CPU cluster on pre-production testbed • SL3 • LCG23 (Upgrade week after 3rd EGEE Conference) • SGAS (Grid bank), (Module for export of accounting information from SGAS to GOC accounting has been developed. Tests and implementation late April early May. (Specifically for accounting of Atlas VO jobs) • Preparation for RC’s in Finland and Norway has started INFO Event, May 26th 2005 - 21

  21. EGEE security effort (JRA3) • 12 FTE from the Northern Federation • KTH (lead), FOM, UvA, UiB and UH-HIP • Existence of a national Grid and high security competence at PDC was key in accepting the co-ordination role • Responsible for overall EGEE Security Architecture • partner in the Middleware design • Responsible for EGEE Security software development • close collaboration with Middleware development INFO Event, May 26th 2005 - 22

  22. BalticGrid • Extending the European eInfrastructure to the Baltic region • Partners: • 10 partners from five countries in the Baltic region and Switzerland (CERN) • Budget: • 4.4M€ over 30 months • Coordinator: • KTH PDC, Stockholm • Resources: • 17 resource centres from start • Major part of the work to be performed by the Estonian, Latvian and Lithuanian partners INFO Event, May 26th 2005 - 23

  23. SweGrid Future • Long term vision is to have all HPC resources (computers & storage) in Sweden available through SweGrid • Accounting will be/is available soon (SGAS) • Clusters will be connected to SweGrid • New loosely coupled clusters for throughput computing • Existing and new clusters with high speed interconnect • SweGrid should grow 10x to provide a national throughput Grid that can be a resource in international projects • Provide a platform for Swedish participation in new EU-projects INFO Event, May 26th 2005 - 24

  24. The NorduGrid project • Started in January 2001 & funded by NorduNet-2 • Initial goal: to deploy DataGrid middleware to run “ATLAS Data Challenge” • NorduGrid essentials • Built on GT-2 • Replaces some Globus core services and introduces some new services • Grid-manager, Gridftp, User interface & Broker, information model, Monitoring • Middleware named ARC • Track record • Used in the ATLAS DC tests in May 2002 • Contributed 30% of the total resources to ATLAS DC II • Continuation • Could be included in the framework of the ”Nordic Data Grid Facility” • Co-operation with EGEE/LCG INFO Event, May 26th 2005 - 25

  25. Resources running ARC • Currently available resources: • 10 countries, 40+ sites, ~4000 CPUs, ~30 TB storage • 4 dedicated test clusters (3-4 CPUs) • SweGrid • Few university production-class facilities (20 to 60 CPUs) • Three world-class clusters in Sweden and Denmark, listed in Top500 • Other resources come and go • Canada, Japan – test set-ups • CERN, Russia – clients • Australia • Estonia • Anybody can join or part • People: • the “core” team grew to 7 persons • local sys admins are called up when users need an upgrade INFO Event, May 26th 2005 - 26

  26. INFO Event, May 26th 2005 - 27

  27. Reflections on NorduGrid • Bottom up project driven by an application motivated group of talented people • Middleware adaptation and development has followed a flexible and minimally invasive approach • Nordic HPC centers have “connected” large general purpose resources since it is good PR for the centers • As soon as “NorduGrid” usage of these resources increases they will be disconnected. There is no such thing as free cycles! • Motivation of resource allocations is missing • NorduGrid lacks an approved procedure for resource allocation to VOs and individual user groups based on scientific reviews of proposals • Major obstacle for utilization of resources on Grids in general! INFO Event, May 26th 2005 - 28

  28. Challenges Users Country 4 Users Country 3 Users Country 2 Users Country 1 Current HPC setup Funding agency country 1 HPC-center HPC-center HPC-center HPC-center Funding agency country 2 Funding agency country 3 Funding agency country 4 INFO Event, May 26th 2005 - 29

  29. HPC-center HPC-center HPC-center HPC-center Funding agency Country 1 Funding agency Country 1 Funding agency Country 1 Funding agency Country 1 Other GRIDs Users VO 3 Proposals VO 1 GRID management VO 2 Accounting Authorization Middleware MoU SLAs INFO Event, May 26th 2005 - 30

  30. eIRG • Policy framework • Authentication • Authorization • Accounting • Resource allocations • SLAs • MoUs • Grid Economies • Strong Nordic interest and representation in eIRG • Collaboration between Nordic national Grids needs eIRG policy INFO Event, May 26th 2005 - 31

  31. Nordic Data Grid Facility - Vision To establish and operate a Nordic computing infrastructure providing seamless access to computers, storage and scientific instruments for researchers across the Nordic countries. Taken from proposal to NOS-N INFO Event, May 26th 2005 - 32

  32. Nordic DataGrid Facility - Mission • operate a Nordic production Grid building on national production Grids • operate a core facility focusing on Nordic storage resources for collaborative projects • develop and enact the policy framework needed to create the Nordic research arena for computational science • co-ordinate and host Nordic-level development projects in high performance and Grid computing. • create a forum for high performance computing and Grid users in the Nordic Countries • be the interface to international large scale projects for the Nordic high performance computing and Grid community INFO Event, May 26th 2005 - 33

  33. Conclusions • Nordic countries are early adopters of GRID technology • There are several national GRIDs • These Grids have enabled participation in EU infrastructure projects • Nordic representation in GGF, eIRG, etc. • Several advanced partners for future Grid related EU projects available in the Nordic region • A Nordic GRID is currently being planned • Nordic policy framework • Nordic interface to other projects INFO Event, May 26th 2005 - 34

  34. INFO Event, May 26th 2005 - 35

More Related