1 / 16

EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester

EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester Thanks to a wireless connected Titanium in the garden@Houten. SURFnet. Manc. UvA. SuperJANET4. CERN. European Topology: NRNs & Geant. Gigabit Throughput on the Production WAN. Manc - RAL 570 Mbit/s

kimball
Download Presentation

EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester Thanks to a wireless connected Titanium in the garden@Houten DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  2. SURFnet Manc UvA SuperJANET4 CERN European Topology: NRNs & Geant DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  3. Gigabit Throughput on the Production WAN • Manc - RAL 570 Mbit/s • 91% of the 622 Mbit access link between SuperJANET4 and RAL • 1472 bytes propagation ~21s • Manc-UvA (SARA) 750 Mbit/s • SJANET4 + Geant + SURFnet • Manc – CERN 460 Mbit/s • CERN PC had a 32 bit PCI bus DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  4. Window Mbytes for UDP BW 750 Mbit/s 460 Mbit/s Window for BW 1 Gbit/s Gigabit TCP on the Production WAN (01) • Throughput vs TCP buffer size • TCP window sizes in Mbytes calculated from RTT*bandwidth DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  5. Gigabit TCP on the Production WAN Man-CERN • Throughput vs n-streams • Default buffer size slope = ~25 Mbit/s/stream up to 9 streams then 15 Mbit/s/stream • With larger buffers rate of increase per stream is larger • Plateaus at about 7 streams giving a total throughput of ~400 Mbit/s DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  6. UDP Throughput: SLAC - Man • SLAC – Manc 470 Mbit/s • 75% of the 622 Mbit access link • SuperJANET4 peers with ESnet at 622Mbit in NY DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  7. Gigabit TCP on throughput Man-SLAC • Throughput vs n-streams • Much less than for European links • Buffer required: rtt*BW (622Mbit) = ~14 Mbytes • With larger buffers > default, rate of increase per stream is ~ 5.4 Mbit/s/stream • No Plateau DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  8. UDPmon & Ripe MAN-SARA from 25 May 02 UDPmon Loss Throughput Mbit/s MAN – UvA ~750Mbit/s RiPE 1 way UvA  RAL ms RiPE 1 way RAL  UvA ms DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  9. iperf Pinger & UDPmon UK-Lyon From 25 May 02 – 04 Jun 02 UDPmon Loss Throughput ~140 Mbit/s Man  Lyon Ping Man  Lyon 40 ms rtt Iperf TCP Throughput ~ 47 Mbit/s Man  Lyon 1048576 byte buffer DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  10. Work on End Systems: PCI: SysKonnect SK-9843 • Motherboard: SuperMicro 370DLE Chipset: ServerWorks III LE Chipset • CPU: PIII 800 MHz PCI:64 bit 66 MHz • RedHat 7.1 Kernel 2.4.14 • SK301 • 1400 bytes sent • Wait 20 us • Sk303 • 1400 bytes sent • Wait 10 us • Frames are back-to-back • Can drive at line speed • Cannot go any faster ! Gig Eth frames back to back DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  11. PCI: Intel Pro/1000 • Motherboard: SuperMicro 370DLE Chipset:: ServerWorks III LE Chipset • CPU: PIII 800 MHz PCI:64 bit 66 MHz • RedHat 7.1 Kernel 2.4.14 • IT66M212 • 1400 bytes sent • Wait 11 us • ~4.7us on send PCI bus • PCI bus ~45% occupancy • ~ 3.25 us on PCI for data recv • IT66M212 • 1400 bytes sent • Wait 11 us • Packets lost • Action of pause packet? DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  12. UDP Throughput: Intel Pro/1000on B2B P4DP6 • Motherboard: SuperMicro P4DP6 Chipset: Intel E7500 (Plumas) • CPU: Dual Xeon Prestonia (2cpu/die) 2.2 GHz Slot 4: PCI, 64 bit, 66 MHz • RedHat 7.2 Kernel 2.4.14 • Max throughput 950Mbit/s • Some throughput drop for packets >1000 bytes • Loss NIC dependent • Loss not due to user  Kernel moves • Traced to discards in the receiving IP layer ??? DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  13. UDPmon UDPmon UDP UDP IP IP Eth drv Eth drv Gig Switch HW HW • No loss at switch • But Pause packet seen to sender Packet Loss: Where? • Intel Pro 1000 on 370DLE • 1472 byte packets • Expected loss in transmitter ! N Lost N Gen IpDiscards N Transmit N Received DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  14. Interrupt Coalescence: Latency • Intel Pro 1000 on 370DLE 800 MHz CPU DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  15. Interrupt Coalescence: Latency • Intel Pro 1000 on 370DLE 800 MHz CPU DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

  16. Interrupt Coalescence: Throughput • Intel Pro 1000 on 370DLE DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester

More Related