1 / 36

Network Power Users The Case for Jumbo Packets with WestGrid Examples

Network Power Users The Case for Jumbo Packets with WestGrid Examples. What are Jumbo Frames & What is 9k MTU?. 2 yrs. 0.5 yrs. 4 yrs. 9k DDS Project. 9k MTU Project. 9k APP Project. Core R&E Router Troubleshooting Large Xfer Measurements International Links

brac
Download Presentation

Network Power Users The Case for Jumbo Packets with WestGrid Examples

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Network Power UsersThe Case forJumbo Packetswith WestGrid Examples

  2. What are Jumbo Frames & What is 9k MTU?

  3. 2 yrs 0.5 yrs 4 yrs 9k DDS Project 9k MTU Project 9k APP Project Core R&E Router Troubleshooting Large Xfer Measurements International Links Internet 2 Sponsored Physics Participants Manual Lightpath TRIUMF to CERN Test Few 9k Taps onto Core Main Tap SDSC Lightpath R&E Non-Routed… UCLP Build Dedicated 9k Links ZX GBICs/quad gigE/campus MM CANARIE NETERA BCNET 3 Universities Only 9k Link on Campus Physics & Biochem Participants HEPnet - WestGrid - TRIUMF 9k Node - HPC - 9k Node Viz HPC 9k Tests NFS HPC 9k Tests Lightpath R&E Clone-Tune 9k APP… Handoff Provide 9k Switch Biochem UVic - SFU - UofA Prelim 9k SOA Framework Drug Discovery System Flow Tune 9k SOA Framework Integrate 9k SOA Framework Demo 9k SOA-DDS Based Collab 9k SOA-DDS Software Dev Examples Complete Handoff … ;-) 9kMTU - APP- DDSProject Evolution…

  4. 9000 MTU 8192 MTU 7168 MTU 6144 MTU 5120 MTU 4096 MTU 3072 MTU 2048 MTU 512 MTU 9K MTU Project - Results

  5. 0.7 * Max Segment Size (MTU) • e2e throughput < • Round Trip Time (latency) sqrt[loss] • M. Mathis, et.al. How do Jumbo Packets Affect Bandwidth? • If TCP window size and network capacity are not rate limiting factors then (roughly): • Double the MSS, double the throughput • Effect of slow start? • Effect of irregular flow?

  6. 9kAPP– Approaching Application Performance • Need metrics for defining network impact on dependent applications • Best current example – MOS as indication of VoIP performance • Models of network dependence required • Applicable to QoS/SLA

  7. 9kAPP– User Experience • User expectations of applications • Examples: • Interaction with 3D models • Collaboration with multiple models/data/voice/video • Massive data set manipulation • Collaborative HPC simulations

  8. 9kAPP- General Models of Network Dependency • Near Real Time (nrt) • Congestion, Drop, MTU & Transit time sensitive • Transactional (tr) • Congestion, MTU & Drop sensitive • Data Transfer (dt) • Congestion, Drop, MTU & Transit time sensitive • Best Effort (be) • Not sensitive

  9. 9kAPP – Real-Time Applications • Examples – IPTV and Voice-over-IP • Requirement – support human interaction through highly subjective perceptive processes • Nature - asynchronous, constant, low-rate, non-TCP, streams • Dependencies • Highly sensitive to bursty loss • Sensitive to latency, particular in conversation context • Robust to jitter, up to some limit

  10. 9kAPP – Synchronous/Transactional • Examples - interactive collaborative systems; distributed file systems • Requirement - maintain some form of state at two or more remote locations; • Nature - intensive, bursty, synchronous traffic; varies from very small amounts of data to huge exchanges • Dependencies – requires high transfer rates, irregular; highly sensitive to latencies; intolerant of slow-start

  11. 9kAPP - Data Transfer • Example – FTP; data backup; emergency recovery • Requirement – transfer massive amounts of data, as quickly as possible • Nature - sustained one-way flows at maximum rates of transfer • Dependencies – sensitive to the characteristics of the end-host transmission protocols (i.e. TCP); high capacity; high impact on other flows

  12. 9kAPP - Best Effort • Examples - e-mail, Web browsing, and remote login • Requirement – sufficient resource to maintain minimal state or connection • Nature - largely stateless; low rates of data transfer required; not gated by human response • Dependencies – no critical requirements for network responsiveness

  13. 9kAPP - Choice of Application Performance • Key performance factor – jumbo packets • Previous 9k MTU testing applicable to simple data transfer use case • Make the case for a more demanding 9k application performance category • High performance: • Interactive/collaborative visualization • WestGrid visualization as exemplar • Distributed file systems • WestGrid Gridstore as exemplar

  14. Application Performance in Context

  15. 9kAPPProject Components • Three networks: Netera, CA*net4, BCNET

  16. 9kAPPProject Sites & Nodes • Three sites: UofA, SFU, UVic • Four nodes • TRIUMF • HEPnet • Vizserver • Gridstore • Why these? • Physics expertise • 9k lightpath ready • WestGrid applications • vizserver • gridstore

  17. 9kAPPProject - Define 9K paths • Two phases: Viz & Grid • Phase 1 Viz • TRIUMF • HEPnet • Vizserver • Phase 2 Grid • TRIUMF • HEPnet • Gridstore

  18. 9kAPPProject - Phase 1 - Viz • Vizserver session – optional compression • IRIX OS – render pipe • TCP: vary MTU 68 – 9000 at server • Measure performance vs. MTU • Vizclient session – wraps local openGL • Fully interactive local X session • refresh requests in render pipe • VMD: Visual Molecular Dynamics • Xserver & Graphics calls to render pipe • Collaborative VMD sessions

  19. 9kAPPProject - Network Build • 3 Lightpaths - UCLP - SONET • Each path STM-24c dropped on gigE • New quad gigE blade BCNET ONS • 2 new ZX GBICs • SM fibre run to SFU lit • SFU Campus Network • ZX GBIC installed in Enterasys ER16 • Phase 1 - Viz • MM fibre run direct to Cisco 6509 • Phase 2 - Grid • MM fibre run via Enterasys N7

  20. 9kAPPProject - Short Circuit Routing 1.5k 9k

  21. 9kAPPProject – Testing Procedure - pMTU Accumulated data from 9k MTU and 9k APP Projects

  22. 9kAPPProject – pMTU Issues • As MTU increases and increasing varies between hops • determine optimal pMTU before and changes during application use • locate problem hops with unusual behavior • Is larger effective lower layer pMTU actually better from the application perspective? • Are packets actually sized appropriately, by the packetization layer, given a larger pMTU? • What are some effects of larger pMTU under congestion conditions?

  23. 9kAPPProject - Phase 2 - Grid • Distributed file system • Possible candidates • CXFS, GPFS, NFSv4 • Possible clients/servers • SGI IRIX, IBM AIX, Linux • Preliminary model • NFSv4 on Linux & IBM AIX(gridstore) • Primary use cases • File sharing – massive data sets • Physics • Bioinformatics

  24. 9kAPPProject - Grid NFS Application • NFS session • AIX NFSv4 server • Linux NFSv4 clients • over TCP not UDP • NFS server – wraps local HPC file system • Fully interactive local file system session • Fast metadata updates for directory browsing

  25. 9kAPPProject – Grid Network & Testing • Network reconfigured with intermediate link • Viz performance rechecked • Gridstore VLAN setup • Fractional quad gigE • Test via NFSTest suite (opensource) • TCP: vary MTU 68 – 9000 at server • Probe network after MTU alteration • Measure performance vs. MTU • Linux NFSv4 to AIX NFSv4 • Probe while test in progress

  26. 9kAPPProject - Grid - NFS - Tuning • Objectives derived from WestGrid gridstore • Baseline performance – simple NFS client • Session • TCP - NFSv4 • NFStest suite • Time to complete vs. MTU • Individual test performance vs. MTU • Blocksize and other tuning considerations • NFS filesystem mount options • Block size = 8192 bytes • Fragmentation factors • Native filesystem block size

  27. 9kAPPProject • Bill Rutherford (Rutherford Research/RRX – Project Coordinator) • Loki Jorgenson (Apparent Networks/SFU – Project Coordinator) • Thomas Tam (CANARIE/CA*net4 – CANARIE/UCLP Coordinator) • Bryan Caron (TRIUMF/UofAlberta – TRIUMF/UCLP Coordinator) • Randy Sobie (HEPnet/UVic – HEPnet President/Grid Integration) • Brian Corrie (WestGrid/IRMACS/SFU - IRMACS Coordinator) • Rob Ballantyne (IRMACS/SFU - IRMACS Network Coordinator) • Martin Siegert (WestGrid/SFU – WestGrid/GridStore Coordinator) • Dave Bickle(HEPnet/UVic – HEPnet Coordinator/Grid Integration) • Ken Howard(Network Services/UVic – Network Coordinator) • Peter van Epp(Network Services/SFU – Network Coordinator)

  28. 9kDDSProject – Drug Discovery System • Based on 9k APP Project • Combined Physics, Grid, Bioinformatics • Joint development of network & software • Share network expertise • Help develop preliminary software • SOA approach • Collaborative viz • Distributed file systems • Instrument interfaces • Grid integration • Lightpath integration

  29. 9kDDSProject – Network Overview

  30. Future – Performance Profiles by Application • APP Network Performance Profile • Build up statistical APP profiles • Use APP profiles to optimize context • Next Generation Router Design • Use APP Profiles • Allocate resources • Design microflow queues • Identify MTU issues • Dynamically configure path mechanics

  31. End of Presentation Note: 9k APP Project Meeting in Room 1535 at 3:00

  32. BCNET ANC 2005 Outline…

  33. Outline • Short overview of previous 9K work • What is 9k? • 9k XXX Project snapshots? • Bandwidth value … example data? • How effect bandwidth? – equations? • Application performance – definition • User experience … limits? • Near Real-time (nrt) • Transactional (tr) • Bulk transfer (bt) • Best-effort (be) • Value of MOS… expand? to the VoIP industry • Project objectives • Isolate a single simple performance factor – packet size? • Identify prospective applications • Interactive collaborative visualization (nrt) • Distributed file system (tr + bt + nrt[metadata]) • Characterize application performance in context … • stack mechanics … grid integration?

  34. Outline – cont. • Project components • Three sites: UofA, SFU, Uvic • Why these sites … only 9k available + RTT factor? • Three networks: BCNET, Netera, CA*net4 • WestGrid applications • Brian’s Westgrid vizserver… IRMACS • Martin’s Westgrid grid storage • Define 9K paths • Application profiling • Visualization server – phase 1 • Identify system • Define primary use case … why vmd … why collab? • Define network profile … describe nw build? • Identify testing procedure … pMTU tests … issues

  35. Outline – cont. • Distributed file system – phase 2 • Identify candidates – CXFS, GPFS, NFSv3-4 • Identify possible clients/servers • Define primary use cases … physics …DDS file sharing • Define network profile • Identify testing procedure (NFStest) • Basic types of test … ? • Futures • 9k DDS … trend to separate purpose nw… key role of UCLP • Performance profiles by application … ??? • Ng rtr … mtu issues … app microflow queues? • Credits • Meeting Reminder

  36. BCNET ANC 9k APP Project Meeting Agenda – April 26 3-4pm • Review of current status • Lightpaths • IRMACS • Gridstore • HEPnet • TRIUMF • Viz test • Network • Probes & schedule • Tests • Demos (incl special medicine collab demo UVic – UofA) • Gridstore test • Network • Probes & schedule • Tests • Demos • Follow on • 9k DDS preliminary • UCLP integration … ? • SOA ideas… ?

More Related