1 / 28

DataTAG project Status & Perspectives Olivier MARTIN - CERN

DataTAG project Status & Perspectives Olivier MARTIN - CERN. GNEW’2004 workshop 15 March 2004, CERN, Geneva. Presentation outline . Project overview Testbed characteristics and evolution Major networking achievements Where are we? Lambda Grids Networking testbed requirements

parson
Download Presentation

DataTAG project Status & Perspectives Olivier MARTIN - CERN

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DataTAG project Status & PerspectivesOlivier MARTIN - CERN GNEW’2004 workshop 15 March 2004, CERN, Geneva

  2. Presentation outline • Project overview • Testbed characteristics and evolution • Major networking achievements • Where are we? • Lambda Grids • Networking testbed requirements • Acknowledgements • Conclusions 2

  3. DataTAG Mission • EU  US Grid network research • High Performance Transport protocols • Inter-domain QoS • Advance bandwidth reservation • EU  US Grid Interoperability • Sister project to EU DataGRID TransAtlantic Grid 3

  4. Project partners http://www.datatag.org 4

  5. Funding agencies Cooperating Networks 5

  6. Brunel University CERN CLRC CNAF DANTE INFN INRIA NIKHEF PPARC UvA University of Manchester University of Padova University of Milano University of Torino UCL EU collaborators 6

  7. ANL Caltech Fermilab FSU Globus Indiana Wisconsin Northwestern University UIC University of Chicago University of Michigan SLAC Starlight US collaborators 7

  8. Workplan • WP1: • Establishment of a high performance intercontinental Grid testbed (CERN) • WP2: • High performance networking (PPARC) • WP3 • Bulk data transfer validations and application performance monitoring (UvA) • WP4 • Interoperability between Grid domains (INFN) • WP5 & WP6 • Dissemination and project management (CERN) 8

  9. GriPhyN PPDG iVDGL DataTAG/WP4 framework and relationships HEP applications, Other experiments Integration HICB/HIJTB Interoperability standardization GLUE 9

  10. Testbed evolution • The DataTAG testbed evolved from a simple 2.5 Gb/s Layer3 testbed (Sept. 2002) into an extremely rich multi-vendor 10 Gb/s Layer2/Layer3 testbed (Sept. 2003) • Alcatel, Chiaro, Cisco, Juniper, PRocket • Exclusive access to the testbed is granted through an advance testbed reservation application • Direct extensions to Amsterdam UvA/Surfnet (10G) & Lyon INRIA/VTHD (2.5G) • Layer 2 extension to INFN/CNAF over GEANT & GARR using Juniper’s CCC • Layer 2 extension to the OptiPuter project at UCSD (University of California San Diego) through Abilene and CENIC under way. • 1st L2/L3 Transatlantic testbed with native 10Gigabit Ethernet access. 10

  11. DataTAG testbed phase 1 (2.5Gbps) VTHD/INRIA Linux PCs ONS15454 stm16 (FranceTelecom) Linux PCs ONS15454 STM64 SURF NET SURFNET CESNET r06gva Alcatel7770 stm64 (GC) CNAF GEANT r06chi-Alcatel7770 Linux PCs stm16(Colt) backup+projects Alcatel 1670 Alcatel 1670 r05gva-JuniperM10 r05chi-JuniperM10 r04chi Cisco7609 stm16 (T-Systems) s01chi Extreme S5i Chicago Geneva 1G ethernet DataTAG testbed phase 2 (10Gbps) simplified 10G ethernet 2.5G STM16 10G STM64 Linux PCs Linux PCs 10Gbps Optical wave (T-Systems) ABILENE GEANT StarLight Force10 JuniperM10 Juniper T320 Juniper T320 Cisco7609 Cisco7606 VTHD/INRIA StarLight Cisco6509 Alcate l7770 edoardo.martelli@cern.ch last update: 20030909 11

  12. 12

  13. DataTAG testbed Alcatel Chiaro Cisco Juniper PRocket 13

  14. Main networking achievements (1) • Internet landspeed records have been beaten one after the other by the DataTAG project partners and/or teams closely associated with DataTAG: • Atlas Canada lightpath experiments during iGRID2002 (Gigabit Ethernet) and Telecom World 2003 (10Gigabit Ethernet, aka WAN-PHY) • New Internet2 landspeed record (I2 LSR) by Nikhef/Caltech team (SC2002) • FAST, GridDT, HS-TCP, Scalable TCP experiments (DataTAG partners & Caltech) • Intel 10GigE tests between CERN (Geneva) and SLAC (Sunnyvale) (CERN, Caltech, Los Alamos Nationa Laboratory, SLAC) • 2.38 Gbps sustained rate, single flow, 1TB in one hour • I2 LSR awarded during Internet2 Spring member meeting (April 2003) 14

  15. ATLAS Canada Lightpath trialsTRIUMF Vancouver & CERN Geneva through Amsterdam CANARIE 2xGbE circuits NetherLight StarLight SURFnet 2xGbE circuits “A full Terabyte of real data was transferred at rates equivalent to a full CD (680MB) in under 8 seconds and a DVD in under 1 minute” Wade Hong et al 09/2002 Bringing effective data transfer rates below one second per CD! Subsequent 10GigE WAN-PHY Experiments during Telecom World 2003 15

  16. 10GigE Data Transfer Trial On Feb. 27-28 2003, a terabyte of data was transferred in 3700 seconds by S. Ravot of Caltech between the Level3 PoP in Sunnyvale near SLAC and CERN through the TeraGrid router at StarLight from memory to memory with a single TCP/IPv4 stream. This achievement translates to an average rate of 2.38 Gbps(using large windows and 9kB “jumbo frames”). This beat the former record by a factor of ~2.5 and used the 2.5Gb/s link at 99% efficiency. European Commission Huge distributed effort, 10-15 highly skilled people monopolized for several weeks!

  17. 10G DataTAG testbed extension to Telecom World 2003 and Abilene/Cenic On September 15, 2003, the DataTAG project was the first transatlantic testbed offering direct 10GigE access using Juniper’sVPN layer2/10GigE emulation. Sponsors: Cisco, HP, Intel, OPI (Geneva’s Office for the Promotion of Industries & Technologies), Services Industriels de Geneve, Telehouse Europe, T-Systems 17

  18. Main networking achievements (2) • Latest IPv4 & IPv6 I2LSR were awarded, live from the Internet2 fall member meeting in Indianapolis, to Caltech & CERN during Telecom World 2003: • May 6, 2003: • 987 Mb/s single TCP/IP v6 stream • October 1, 2003 • 5.44 Gb/s single TCP/IP v4 stream between Geneva and Chicago: • 1.1TB in 26 minutes or one 680MB CD in 1 second • More records have been established by Caltech & CERN since then: • November 6, 2003: • 5.64 Gb/s single TCP/IP v4 stream between Geneva and Los Angeles (CENIC PoP) across DataTAG and Abilene. • November 11, 2003, • 4 Gb/s single TCP/IP v6 stream between Geneva and Phoenix (Arizona) through Los Angeles • February 24, 2004 • 6.25 Gb/s with 9 streams for 638 seconds, i.e. half a terabyte transferred between CERN in Geneva and the CENIC PoP in Los Angeles across DataTAG and Abilene. 18

  19. Internet2 landspeed record history (IPv4&IPv6) Impact of a single multi-Gb/s flow on the Abilene backbone 19

  20. Significance of I2LSRs to the Grid? • Essential to establish the feasibility of multi-Gigabit/second single stream IPv4 & IPv6 data transfers: • Over dedicated testbeds in a first phase • Then across academic & research backbones • Last but not least across campus networks • Disk to disk rather than memory to memory • Study impact of high performance TCP over disk servers • Next steps: • Above 6Gb/s expected soon between CERN and Los Angeles (Caltech/CENIC PoP) across DataTAG & Abilene • Goal is to reach 10Gb/s with new PCI Express buses • Study alternatives to standard TCP (Reno) • Non-TCP transport (Tsunami, SABUL/VDT) • HS-TCP, Scalable TCP, H-TCP, FAST, Grid-DT, Wesley+, etc… 20

  21. Geneva Juniper M10 AF AF Layer2: VLAN Layer2: VLAN BE 1 GE bottleneck IP-Qos configured BE Main networking achievements (3) • QoS • Advance bandwidth reservation • GARA extensions • AAA extensions 21

  22. Where are we? • The DataTAG project came up at exactly the right time: • Back in the late 2000, 2.5 Gb/s looked futuristic • 10GigE, especially host interfaces, did not really exist • However, it was already very clear that the standard TCP stack (Reno/Newreno) was problematic • Much hope was placed on autotuning (Web100/Net100) & ECN/RED like solutions • Actual bit error rates of transatlantic circuits were over-estimated • Much better shape than expected on over-provisioned R&D backbones such as Abilene, Canarie, GEANT • For how long? • One of the strongest proof made by DataTAG is the extreme vulnerability of production R&D backbones in the presence of high performance flows (i.e. 10GigE or even less) 22

  23. Where are we (cont)? • For many years the Wide Area Network has been the bottlemeck, this is no longer the case in many countries, thus making the deployment of data intensive Grid infrastructure, in principle, possible, e.g. • EGEE the DataGrid successor • Recent I2LSR records show, for the first time ever, that the network can be truly transparent and that throughput is only limited by the end hosts and/or campus network infrastructures. • Challenge shifted from getting adequate bandwidth to deploying adequate LANs and cybersecurity infrastructure as well as making effective use of it! • Non-trivial transport protocol issues still need to be resolved • The only encouraging sign is that this is now widely recognized • But we are still quite far from converging on a practical solution? 23

  24. Layer1/2/3 networking (1) • Conventional layer 3 technology is no longer fashionable because of: • High associated costs, e.g. 200/300 KUSD for a 10G router interfaces • Implied use of shared backbones • The use of layer 1 or layer 2 technology is very attractive because it helps to solve a number of problems, e.g. • 1500 bytes Ethernet frame size (layer1) • Protocol transparency (layer1 & layer2) • Minimum functionality hence, in theory, much lower costs (layer1&2) 24

  25. Layer1/2/3 networking (2) • « Lambda Grids » are becoming very popular: • Pros: • circuit oriented model like the telephone network, hence no need for complex transport protocols • Lower equipment costs (i.e. « in theory » a factor 2 or 3 per layer) • the concept of a dedicated end to end light path is very elegant • Cons: • « End to end » still very loosely defined, i.e. site to site, cluster to cluster or really host to host • Higher circuit costs, Scalability, Additional middleware to deal with circuit set up/tear down, etc • Extending dynamic VLAN functionality to the campus network is a potential nightmare! 25

  26. « Lambda Grids » What does it mean? • Clearly different things to different people, hence the « apparently easy » consensus! • Conservatively, on demand « site to site » connectivity • Where is the innovation? • What does it solve in terms of transport protocols? • Where are the savings? • Less interfaces needed (customer) but more standby/idle circuits needed (provider) • Economics from the service provider vs the customer perspective? • Traditionally, switched services have been very expensive, • Usage vs flat charge • Break even, switches vs leased, few hours/day • Why would this change? • In case there are no savings, why bother? • More advanced, cluster to cluster • Implies even more active circuits in parallel • Even more advanced, Host to Host • All optical • Is it realisitic? 26

  27. Networking testbed requirements • Multi-vendor • Unless a particular research group is specifically interested by the behaviour of TCP in the presence of out of order packets, running high performance TCP tests across a Juniper M160 backbone is pretty useless. • IPv6 achievable performance vary widely between different vendors • MPLS & QoS implementations also veary widely • Interoperability • Dynamic • Implies manpower & money • Partitionable • Reservation application • Reconfigurable • Avoid manual recabling, implies Electronic or Optical switch/patch panel • Extensible • Extensions to other networks • Implies collaboration • Not limited to network equipment, must also include high performance servers, high perf. Disks & NICs, • Coordination with other testbeds 27

  28. Acknowledments • The project would not have accumulated so many successes without the active participation of our North American colleagues, in particular: • Caltech/DoE • University of Illinois/NSF • iVDGL • Starlight • Internet2/Abilene • Canarie • and our European sponsors and colleagues as well, in particular: • European Union’s IST program • Dante/GEANT • GARR • Surfnet • VTHD • The GNEW2004 workshop is yet another example of successful collaboration between Europe and USA 28

More Related