1 / 26

Advancing Computer Science Research Through DAS Systems Collaboration

Explore the impact of Distributed ASCI Supercomputer systems (DAS) on research, collaborations with VL-e, SURFnet, and Grid’5000, new user integration, system configurations, usage by ~200 users, workshop insights, and international grid experiments. Discover the significance of DAS infrastructure for computer science research experimentation.

jamesryan
Download Presentation

Advancing Computer Science Research Through DAS Systems Collaboration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DAS-1 DAS-2 DAS-3 Henri Bal Vrije Universiteit Amsterdam Faculty of Sciences

  2. Outline • Introduction • DAS-1, DAS-2, DAS-3 systems • Impact of DAS • Collaborations with • VL-e • SURFnet • Grid’5000 • New users

  3. Distributed ASCI Supercomputer • Joint infrastructure of ASCI research school • Clusters integrated in a singledistributed testbed • Long history (10 years)and continuity

  4. DAS is a Computer Science grid • Motivation: CS needs its own infrastructure for • Systems research and experimentation • Distributed experiments • Doing many small, interactive experiments • DAS is simpler and more homogeneous than other grids • Single operating system • “A simple grid that works’’

  5. Typical heterogeneous Grid

  6. DAS-1 (1997-2002) Configuration 200 MHz Pentium Pro 64-128 MB memory 2.5 GB local disk Myrinet interconnect BSDI => Redhat Linux VU (128) Amsterdam (24) 6 Mb/s ATM Leiden (24) Delft (24)

  7. VU (72) Amsterdam (32) SURFnet1 Gb/s Leiden (32) Delft (32) Utrecht (32) DAS-2 (2002-2007) Configuration two 1 GHz Pentium-3s >= 1 GB memory 20-80 GB disk Myrinet interconnect Redhat Enterprise Linux Globus 3.2 PBS => Sun Grid Engine

  8. DAS-3 Procedure • NWO proposal (ASCI, VL-e, MultimediaN), Sep’04 • Funding from NWO/NCF, Apr’05 • Steering group + implementation group • European tender (with Stratix and TUD/GIS) • Selected ClusterVision, Apr’06 • Optical wide-area network bySURFnet (GigaPort-NG project)

  9. DAS-3 • dual AMD Opterons4 GB memory • 250-1500 GB disk • More heterogeneous: • 2.2-2.6 GHz Single/dual core nodes • Myrinet-10G (exc. Delft) • Gigabit Ethernet • Scientific Linux 4 • Globus, SGE UvA/MultimediaN (46) VU (85) SURFnet6 UvA/VL-e (40) 10 Gb/s lambdas TU Delft (68) Leiden (32)

  10. Performance

  11. Outline • Introduction • DAS-1, DAS-2, DAS-3 systems • Impact of DAS • Collaborations with • VL-e • SURFnet • Grid’5000 • New users

  12. Usage of DAS • ~200 users, 34 Ph.D. theses • Clear shift of interest: Cluster computing Distributed computing Grids & peer-to-peer computing Virtual laboratories for e-Science

  13. Opening workshops • DAS-1 opening (1998): • only 1 talk (VU) about distributed computing • DAS-2 opening (2002): • only 1 talk (VU) about local computing • DAS-3 opening (NOW): • talks about grids, e-Science, optical networks, international embedding, distributed multimedia computing, P2P/gossiping, co-allocating Grid Scheduler

  14. Grid’5000 Impact of DAS • Major incentive for VL-e  20 M€ BSIK funding • Virtual Laboratory for e-Science • Collaboration SURFnet on DAS-3 • SURFnet provides multiple 10 Gb/s light paths • Collaboration with French Grid’5000 • Towards a European scale CS grid?

  15. VL-e: Virtual Laboratory for e-Science

  16. Some VL-e results • VLAM-G workflow system • KOALA scheduler (see Mohamed’s talk) • Zorilla peer-to-peer system (see Drost’s talk) • Java-centric grid computing (see demo) • Ibis: Grid communication library • Satin: divide&conquer on grids • JavaGAT: Gridlab Application Toolkit • Applications of Ibis, Satin, JavaGAT: • Automatic grammar learning (UvA) • Protein identification (AMOLF) • Brain image analysis (VUmc)

  17. SURFnet6 • 4 DAS-3 sites happen to be on Subnetwork 1 (olive green) • Only VU needs extra connection Source: Erik-Jan Bos

  18. Adding 10G waves for DAS-3 WSS • Band 5 added at all participating nodes • WSS added for reconfigurability • “Spur” to connect VU • Full photonic mesh possible • Source: Erik-Jan Bos • Also see De Laat’s talk UvA MultiMedian “spur” – hardwired connections VU Leiden Delft

  19. International grid experiments • Distributed supercomputing on GridLab • Real speedups on a veryheterogeneous testbed • Crossgrid experiments • N-body simulations (astronomy) • Grid-based interactivevisualization of medical images

  20. Grid’5000 Grid’5000 500 500 1000 500 Rennes Lyon Sophia 500 500 Grenoble 500 Bordeaux 500 500 Toulouse Orsay

  21. Connecting DAS-3 and Grid’5000 • Optical 10 Gb/sec link between Paris and Amsterdam, funded by VL-e • See Cappello’s talk (and keynote @ CCGrid2007)

  22. Users of DAS-3 • ASCI, VL-e • MultimediaN • See MultimediaN talk + demo • NWO projects • StarPlane, JADE-MM, GUARD-G, VEARD, GRAPE Grid, SCARIe, AstroStream, CellMath,MesoScale, ….. • NCF projects (off-peak hours)

  23. New users • DAS is intended primarily for computer scientists from ASCI, VL-e, MultimediaN • How do we define “Computer Scientist”? • Need a light-weight admission test …

  24. Proposed test There are only 10 types of people in the world: Those who understand binary and those who don't. • = Pass as Computer Scientist Source:

  25. Acknowledgements • NWO • NCF • SURFnet • VL-e • MultimediaN • VU, UvA, TUD, Leiden • ASCI • ClusterVision • TUD/GIS • Stratix • Lex Wolters • Dick Epema • Cees de Laat • Frank Seinstra • Erik-Jan Bos • Henk van der Vorst • Andy Tanenbaum • Bob Hertzberger • Henk Sips • Aad van der Steen • Many others

  26. Special acknowledgement …. • Reasons why DAS is ``a simple grid that works’’ • 1) simple, homogeneous design • 2) Kees Verstoep

More Related