1 / 22

4 Gigabit Onsala - Jodrell Lightpath for e-VLBI

4 Gigabit Onsala - Jodrell Lightpath for e-VLBI. Richard Hughes-Jones. Progress [1] Oct Nov 07. EXPReS Requirements doc produced Path formally requested from JANET Meetings with SUNET & NORDUnet Agree 4 Gigabit path initial priority Organise wiki Discuss other RAst. needs:

aspringer
Download Presentation

4 Gigabit Onsala - Jodrell Lightpath for e-VLBI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 4 Gigabit Onsala - Jodrell Lightpath for e-VLBI Richard Hughes-Jones

  2. Progress [1] Oct Nov 07 • EXPReS Requirements doc produced • Path formally requested from JANET • Meetings with SUNET & NORDUnet • Agree 4 Gigabit path initial priority • Organise wiki • Discuss other RAst. needs: • Onsala-JIVE, Met-Onsala, Met-JBOLOFAR • SUNET 10 GE Onsala-Stockholm installed Nov 07 • NORDUnet install connectivity to Copenhagen • Request GÉANT2+ connection

  3. Progress [2] Dec 07 • JANET indicate insufficient GÉANT2+ access BW to UK • 10Gig packet switched + 2.5 Gig backup • (10-2.5) Gig SDH Lightpath • DANTE Provision 4 Gig path London PoP – Copenhagen • PC shipped to Stockholm • NORDUnet Install and connect PC • Tests from EXPReS1 in London over GÉANT2+ • With internal loopback packets get to Copenhagen and back • NORDUnet cant see light from DANTE • MCC Blade failed Alcatel called to Stockholm • JANET Propose direct 10 GE Lambda Manchester-London

  4. Current Test Path 10 GE Lambda over NORDUnet 4 Gig Lighpath over GÉANT2+

  5. 4 Gig Path JANET Optical

  6. Tests on GÉANT2 Testbed • UDP Fine at 4.2 Gigabit • TCP performance poor due to small Ethernet buffer size on Alcatel 1678 Metro Core Connect • TCP slow start sends bursts at 10 Gbit/s • MCC cannot buffer these with 10 Gbit/s in and 4 Gbit/s out • Evaluate new firmware v4.2 • Ethernet flow-control can now be enabled • London new firmware v4.2 • Copenhagen firmware v4.1

  7. Connect. Communicate. Collaborate UDP Throughput Lon-Stoc • MCC Flow control • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • rx-usecs=25 Coalescence ON • MTU 9000 bytes • Max throughput4.05 Gbit/s • No Packet loss • Sending host,3 CPUs idle • For <18 µs packets,1 CPU is >60% in kernel mode • Receiving host3 CPUs idle • For <20 µs packets, 1 CPU is ~30% in kernel mode

  8. Connect. Communicate. Collaborate UDP Throughput Stoc-Lon • MCC Flow control • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • rx-usecs=25 Coalescence ON • MTU 9000 bytes • Max throughput4.05 Gbit/s • Packet loss as expected • Sending host,3 CPUs idle • For <8 µs packets,1 CPU is >90% in kernel mode • Receiving host3 CPUs idle • For <20 µs packets, 1 CPU is ~30% in kernel mode

  9. Connect. Communicate. Collaborate UDP 1-way Delay Lon-Stoc • MCC Flow control ON • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • rx-usecs=0 Coalescence OFF • MTU 8000 bytes • Wait time 0 ie linespeed • Packets Q with 8.5µs spacing • Packets Leave Q 16 µs spacing

  10. Connect. Communicate. Collaborate TCP reno Throughput Lon-Stoc • MCC Flow control • 27 VC-4 • RTT 31 ms • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • rx-usecs=25 Coalescence ON • MTU 9000 bytes • TCP buffer 32 MByte 2*BDP • iperf Ave throughput3.85 Gbit/s • iperf Max throughput4.02 Gbit/s • 3 re-transmits

  11. Connect. Communicate. Collaborate TCP BIC Throughput Lon-Stoc • MCC Flow control • 27 VC-4 • RTT 31 ms • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • rx-usecs=25 Coalescence ON • MTU 9000 bytes • TCP buffer 32 MByte 2*BDP • iperf Ave throughput3.98 Gbit/s • iperf Max throughput4.02 Gbit/s • Several re-transmits

  12. Connect. Communicate. Collaborate TCP reno Throughput Stoc-Lon • MCC NO Flow control • 27 VC-4 • RTT 31 ms • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • rx-usecs=25 Coalescence ON • MTU 9000 bytes • TCP buffer 32 MByte 2*BDP • iperf Ave throughput192 Mbit/s • iperf Max throughput465 Mbit/s • TCP Cwnd to 0 • Many re-transmits

  13. Connect. Communicate. Collaborate TCP BIC Throughput Stoc-Lon • MCC NO Flow control • 27 VC-4 • RTT 31 ms • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • rx-usecs=25 Coalescence ON • MTU 9000 bytes • TCP buffer 32 MByte 2*BDP • iperf Ave throughput71.6 Mbit/s • iperf Max throughput192 Mbit/s • TCP Cwnd rapidly opens • Many more re-transmits

  14. Connect. Communicate. Collaborate TCP BIC Throughput Lon-StocBut Faster and Longer • MCC Flow control • 48 VC-4 • RTT 60 ms • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • rx-usecs=25 Coalescence ON • MTU 9000 bytes • TCP buffer 59 MByte 1 * BDP • txquelen 1300 • … • iperf Ave throughput6.5 Gbit/s • iperf Max throughput ~6.9 Gbit/s • Some glitches & re-transmits

  15. Summary • MCCs will be upgraded to v4.2 • General case – e.g. with trunking i/f with multiple Qs / VLANs users may have to perform traffic shaping. • Ethernet flow-control with short tails helps TCP • UDP performance provides that required for EXPReS • 28 VC-4 will be required. • Do we wish to move PC to Onsala when convenient ? • What are our timescales?

  16. iBoB hot news

  17. iNetTest Progress • iBoB FPGA running with 200 MHz clock • Statistics, histograms and timing events implemented • Ethernet packet jitter measured 10GE_port0 10GE_port1 • Two iNetTest units tested Back-2-Back • UDP performance at 10 Gigabit linespeed

  18. Connect. Communicate. Collaborate iNetTest B2B UDP Packet Jitter • 10Gig Ethernet CX4 links • 8000 byte packets • 5 ns bins • FWHM 10 ns • No shape variation with packet size 3000 Bytes 8192 Bytes

  19. Connect. Communicate. Collaborate iNetTest B2B UDP Packet size • 10Gig Ethernet CX4 links • 8000 byte packets • Slope is 1 for 200MHz • So we are really running at that speed!!

  20. Any Questions ?

  21. For More Information • www.geant2.net • www.dante.net • For latest news and factsheets http://www.geant2.net/media • For research activities http://www.geant2.net/research • ALICE http://alice.dante.net/ • RedCLARA http://www.redclara.net • REUNA http://www.reuna.cl/joomla/index.php

  22. GÉANT2 Topology November 2006

More Related