1 / 15

Large-Scale Distributed Computing in the Netherlands

Large-Scale Distributed Computing in the Netherlands. an overview David Groep, NIKHEF. What is DutchGrid?. collaboration of 12 organisations established in 2000 functions as a platform , getting manpower & hardware from partner orgs major contributors

kiona
Download Presentation

Large-Scale Distributed Computing in the Netherlands

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Large-Scale Distributed Computing in the Netherlands an overview David Groep, NIKHEF

  2. What is DutchGrid? • collaboration of 12 organisations • established in 2000 • functions as a platform, getting manpower & hardware from partner orgs • major contributors • WTCWVirtual Laboratory Project (ICES/KIS-II) • ASCI (5 combined Dutch CS Faculties) • Foundation NCF (NWO – natl research council) • NIKHEF (EDG)

  3. WCW Amsterdam ASTRON, JIVE NIKHEF SARA UvA AMOLF CWI Amsterdam Telin Leiden KNMI Utrecht Delft Nijmegen WCW VU DutchGrid site map 20021104 David Groep, NIKHEF DutchGrid Sites

  4. Resources • ASCI DAS-2 (VU, UvA, Leiden, TUDelft, Utrecht) • 200 dual P-III 1GHz CPUs • homogeneous clusters, 5 locations • NIKHEF DataGrid clusters • 75 dual P-III ~ 1GHz • 1Gb/s IPv4 + 1Gb/s IPv6 • NCF Gridnational computer facilities foundation from NWO • 66 node dual AMD-K7 Fabric Research Cluster (NIKHEF) • 32 node duals “production quality” cluster (SARA)* • 10Gb/s optical “lambda” test bed† • … • BioASP– various smaller O(10 node) clusters

  5. Resources (cont.) SARA – National HPC Centre • Processing • SGI 1024 processor MPP • Mass storage • StorageTek NearLine tape robot • currently: 500 TByte • will grow “indefinitely” (with funds from NCF) • User expertise centre In TotalOperations/deployment effort ~ 8 FTE availableResearch effort O (40 FTE) over 12 organisations

  6. Resource Usage

  7. 2xGbE CERN,CH CERNRTR 622Mb “rest”Global Xing 2x2.5Gb ONS 2xGbE AMSTERDAM AMSTERDAM IGRID6509 18xGbE UvA NIKHEF “NCSA” “ANL” CHICAGO, IL OC48c lambda ONS 2xGbE GbE’s 7xGbE GbE StarLight6509 SP01 ONS 15454 2.5Gb lambda OC48cSR GbE StarLightRTR GbE BR1CHI BR1AMS BR1AMS BR1CHI BR1AMS 622Mb L2 VPN 622Mb L2 VPN AR5 AR5 622Mb L2 VPN AR5 2.5GPOS 2.5GPOS 10GPOS 10GPOS 2.5GPOS 10GPOS STARTAP 622Mb to “rest” 10GbE StarLightT640 BR4 CR1-1 CR1-1 CR1-2 CR1-2 CR1-1 CR1-2 10G POS Level3 ARGONNE IL 10G POS Tyco BR2 BR2 CR2-1 CR2-1 CR2-2 CR2-2 BR2 CR2-1 CR2-2 SingARENTANet2 … ABILENE NY 622M PO Prot.Tyco/IEEAF 622M PO Prot.Tyco/IEEAF 622M PO Prot.Tyco/IEEAF NY NY NY Networks

  8. NIKHEF network load Monthly graph NIKHEF-SURFnet5 IPv4 interconnect October/November 2002

  9. New Initiatives inside NL NCFGrid strengthening Grid Infrastructure in NL • 4 partners (NIKHEF, ASCI, SARA, ASTRON) • 600 kEUR funding • hardware (3 clusters, 10Gb/s networking) and people (1.5 FTE over 3 years) • fabric research + production-quality grids • new apps: bio informatics, astronomy • approved August 2002 First cluster assembled and being tested…

  10. D.I.Sc. FoodInf BioInf LargeEquip Bio diversity … Test Bed Scaling & Validation IntVisual. Inf. Mngt Intg ExpEnvs UIs HPC SecAAA Networks New Initiatives inside NL (2) in planning stageO (30-50 MEUR) Virtual Laboratory for e-ScienceDutch “knowledge infrastructure” project • Higher-level services to efficiently use global resources • Both “bottom-up” and “top-down” in same project • 14 partners, 6 application domains • middleware research programme

  11. What is happening where in NL?

  12. DutchGrid Op Services • Authentication Services (Certification Authority) • EDG CACG “proper” CA • Student and educational CA • EDG “Tutorial” CA • Regular coordination and info meetings • Installation support • Have been distributing EDG 1.2 till now • User support

  13. Principal Apps • HEP: DØ, Atlas, LHCb, AliceMonte-Carlo and Analysis • OPERA– ozone monitoring • WTCW Virtual Laboratoryvisual analysis environment for (small-scale) experimental science • Chemo-physical surface analysis (PIXE,FT-IR,SIMS) • BioASP – correlating “omics” databases • Virtual Surgery (planning by-passes, artery repairs) • EcoGridcorrelating bio-diversity information databases • Radio AstronomyLOFAR, AVO, VLBI – semi-online and offline processing

  14. Research Areas in NL • High-bandwidth networkingSURFnet, UvA, SARA, NIKHEF • Generic AAA and `security’UvA, NIKHEF • Cluster, Storage Management & MonitoringNIKHEF, SARA • Adaptive programming environmentsFree University Amsterdam • Memory-processor co-allocation TU Delft • Federated Information & Content ManagementUvA • Interactive visualisationCWI, UvA

  15. Summary • DutchGrid Platform started informally in 2000 • de-facto focus for Grid computing in NL • ops (CA, web, …) supported by partners • mix of people from all ‘layers’networks, fabrics, security&AAA, prog. environments, HPC visualisation, semantic DBs, applications

More Related