1 / 33

Size Matters: Network Performance on Jumbo Packets Summer 2004 Joint Techs Loki Jorgenson, Chief Scientist, Apparent N

Size Matters: Network Performance on Jumbo Packets Summer 2004 Joint Techs Loki Jorgenson, Chief Scientist, Apparent Networks. 9k MTU Project(s). test global path MTU on Abilene, CA*net4, CUDI and other R & E networks, plus create a useful researcher mapping tool

parry
Download Presentation

Size Matters: Network Performance on Jumbo Packets Summer 2004 Joint Techs Loki Jorgenson, Chief Scientist, Apparent N

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Size Matters:Network Performance on Jumbo Packets Summer 2004 Joint TechsLoki Jorgenson, Chief Scientist, Apparent Networks

  2. 9k MTU Project(s) • test global path MTU on Abilene, CA*net4, CUDI and other R & E networks, plus create a useful researcher mapping tool • Internet2 ATEAM - Advanced Test Engineering and Measurement • Bill Rutherford (Rutherford Research/GAIT) • Kevin Walsh, Nathaniel Mendoza (San Diego Supercomputing Center/SDSC) • John Moore (Centaur Internet2 Technology Evaluation Center ITEC/NCSU North Carolina State University) • Loki Jorgenson (Apparent Networks/SFU) • CA*net testing • Bryan Caron (Network Manager Subatomic Physics, University of Alberta) • Damir Pobric (Network engineer - CANARIE)

  3. Why Jumbo? • Performance Benefits • High performance data transfers • Grid networks • Meteorology / physics / biotech • Collaborative/interactive multi-media • Performance Requirements • End-to-end path • From NIC to NIC MTU requirement • End station is typically the bottleneck • Gig-E to the desktop

  4. Steady State TCP • If TCP window size and network capacity are not rate limiting factors then (roughly): • 0.7 * Max Segment Size (MTU) • e2e throughput < • Round Trip Time (latency) sqrt[loss] • M. Mathis, et.al. • Double the MTU, double the throughput • Half the latency, the throughput (shortest path matters) • Half the loss rate, 40% higher throughput

  5. Frame Size vs. MTU vs. MSS – An Ethernet Example

  6. MTU Performance Testing: ATEAM • Objectives: • Measurements using multiple methodologies • SmartBits • iPerf • AppareNet/aNA • Determine current dependency on transmission packet size • Determine extent of jumbo MTU access

  7. About aNA • AppareNet Network for Academics • Currently 16 sequencers across CA*net and Abilene • NIS in Vancouver, Canada • 10 Gig-E/Jumbo hosts • 4 hosts in Canada • BCnet • Netara Alliance • CA*net NOC • ACORN-NS • Upgrade ANA to v2.5 (mid-August 2004) from v1.6.2 • Web access • On-demand deployment

  8. About AppareNet • Uses light, non-intrusive active probing • ICMP or UDP packets in various configurations • Point-and-shoot to most IP addresses • Performs network path characterization • Performs expert system diagnostics • Single-ended  two-way measures (e.g. half-duplex different from full-duplex) • Samples network to generate same view as best effort application (pre-TCP)

  9. Example Measurement Paths: SDSC to Ottawa

  10. Example Measurement Paths: SDSC to Halifax

  11. Example Measurement Paths: SDSC to Mexico

  12. Why use Jumbo Frames? Increasing MTU gives better performance. Advanced Test Engineering and Measurement (ATEAM) performance measurements taken across the Abilene and CA*net4 backbone http://www.ncne.nlanr.net/training/techs/2003/0803/presentations/0803-moore1_files/v3_document.htm 9000 MTU 8192 MTU 7168 MTU 6144 MTU 5120 MTU 4096 MTU 3072 MTU 2048 MTU 512 MTU

  13. More GigE Test Results

  14. Preventing MTU conflicts – Network Negotiation

  15. GigE Black Hole Hop • What is happening?: • RFC 1191 and “TCPSlow Start” are interacting • Packets are lost • Retransmission happen, causing performance degradation • Client responds to some packets, keeping connectionopen • Overall performance appears slow to client

  16. MTU handling via RFC 1191 PMTU discovery • Advantages: • Router is not loaded • Maximum performance achieved • Disadvantages: • reliance on ICMP • easy to misconfigure • Applications: • almost all modern applications

  17. Intel Pro XT 1000 version 6.2.22.1

  18. Intel Pro XT 1000 version 7.0.36.0

  19. Intel Pro XT 1000 version 7.4.19.0

  20. Avoiding GigE MTU problems • Maintain logical Layer 3 diagrams • Assign MTUs based on a per-subnet basis • Be consistent with MTU values used • Use 1500 bytes for legacy Ethernet (no registry hacks) • Use recommended 9000 bytes MTU for GigE when jumbo frames are used (standard for CAnet and Abilene networks) • Don’t forget to add 18 bytes when adjusting frame size (e.g. set NIC to 9018 bytes frame size to maintain a 9000 byte MTU) • Don’t arbitrarily filter out ICMP messages

  21. Resources • Path MTU tools: • ANA pMTU service – uses ANA sequencers across I2 and CA*nethttp://pathmtu.apparenet.com:8282/ana@apparenet.com:guest42 • NCNE MTU Discovery Service – uses service located at NCNE http://www.ncne.org/jumbogram/mtu_discovery.php • pMTU Applet - Java-based client for end-user station http://sourceforge.net/projects/mtu-calculator/http://ana.apparenet.com:8282/pMTU/Download • Jumbo MTU Performance whitepaper • http://www.apparentNetworks.com/wp/

  22. Path MTU Mapping Service

  23. pMTU Java Client pMTU applet available from SourceForge

  24. pMTU Database and Map Visualization pMTU Database and MTU map

  25. pMTU Project Status • Initial prototypes available • Open source (SourceForge) projects • pMTU Applet • Java/Swing client • Webstart • Native code driver • Windows only • Linux-ready • pMTU DB/Map • mySQL/Servlet • accepts connections • limited map capability • saves Applet results

  26. Current Work – Application Performance • End-user Performance Benefits • End-to-end requirement • Very limited jumbo-access from campus • Influence of real-world application dynamics • Identify classes of need that require jumbo • Identify classes of implementation susceptible to jumbo

  27. Target Implementations • Distributed File Systems • Simple, commonly useful utility • Challenged by latency, loss • Often data-intensive • Persistent use (not one-time) • Non-denominational • Visualization Server • Visual/subjective benefit • Very demanding • Resolution dependent load • Specific to particular fields/users

  28. Resources • On-Line Path MTU tools: • ANA pMTU service – uses ANA sequencers across I2 and CA*nethttp://pathmtu.apparenet.com:8282/ana@apparenet.com:guest42 • NCNE MTU Discovery Service – uses service located at NCNE http://www.ncne.org/jumbogram/mtu_discovery.php • pMTU Applet - Java-based client for end-user station http://sourceforge.net/projects/mtu-calculator/http://ana.apparenet.com:8282/Download • Jumbo MTU Performance whitepaper • http://www.apparentNetworks.com/wp/

  29. Fin • End of Presentation • aNA.apparenet.com

  30. Fin • Backup Slides

  31. GigE Black Hole Hop

  32. MTU handling via fragmentation

  33. MTU handling via RFC 1191 PMTU discovery

More Related