1 / 35

Experiences with the QWEST/SCinet Bandwidth Challenge 2000-2002

This text discusses the experiences and challenges faced during the QWEST/SCinet Bandwidth Challenge from 2000 to 2002, including topics such as network throughput, congestion avoidance, and the need for full control of network throttling. The text also explores the importance of low latency and interactivity in visualization applications, as well as the future of bandwidth management and the use of UDP-based protocols. The text concludes with acknowledgements to various organizations involved in the project.

janetortega
Download Presentation

Experiences with the QWEST/SCinet Bandwidth Challenge 2000-2002

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experiences with theQWEST/SCinet Bandwidth Challenge2000-2002 John Shalf, Wes Bethel LBNL

  2. Shaky City Data Caches HPC Resources Simulations Sensor Nets STM Collaborators Handheld Devices Dr Jane

  3. Backend Source Volume Viewer Component Visapult Architecture

  4. Source Volume Visapult Architecture 3D 2D

  5. Source Volume Visapult Architecture 3D Gigabits 2D Megabits

  6. Source Volume Visapult Architecture 3D Gigabits 2D Megabits

  7. SC2000 Demo Configuration

  8. SC2000 Network Throughput

  9. Refactoring the Design • Congestion avoidance • Good for internet • Bad bad baaaad for PVCs and other dedicated networks. (switched lambdas?) • Multistream TCP • Erratic performance • Requires a lot of tuning • Unfriendly to other users • Unfriendly to visualization applications • We want full control of the “throttle” • Very much like network video

  10. Refactoring the Design • TCP is the wrong thing for interactive vis! • Layer 3 latency/jitter (all buffering effects) • Poor response to bursty traffic • Vis needs interactivity and minimal latency! • Network Video / UDP streams • Present packets to app. immediately (low latency) • Full control of data rate • Lossy, but effects of loss can be managed • SOCK_RDM

  11. Effect of Loss on Visapult Evolving Binary Black Hole Merger Simulation (100 timesteps)

  12. SC2001 BWC Testbed

  13. SC2001 BWC Results • 3.3 Gigabits Peak (MRTG of ESNet/QWEST OC-48) • Lots of time spent tuning (packet pacing is nontrivial)

  14. SC2002 Planning • Better UDP packet pacing with poor clock granularity • Error diffusion algorithm from dithering • Omnidirectional viewing for Visapult • Nascent Grid Infrastructure • Global Grid Testbed Collaboration (GGTC) • GGTC won 3 out of 4 HPC challenge awards

  15. Global Testbed

  16. Bandwidth Scavenging Service Czech Republic 90% utilization on path from Brno To GEANT (to SC) via Praha

  17. Steady @ 16+ Gigabits!

  18. Why do we do this?? • Opportunity to test the next generation network capacity (2 years before production) • Opportunity to test software ideas that will be required to make use of the next generation network (2 year lead) • Opportunity to test future network engineers (give them a 2 year lead on the problems with future networks) • Prescient • I-Way 1995 (presaged vBNS experiences and “The Grid”) • 1997 multi-stream tests (presaged multi-stream TCP protocols) • 2001 10GigE and UDP-based protocols • 10GigE backbone for SC2002 • RUDP, SABUL, TSUNAMI user-space UDP-based protocols

  19. What should you know for 2003 • Manual throttle (UDP-based protocols) are here to stay. • Hopefully SOCK_RDM will cover most needs • Whaaa? Those idiots are going to burn down the network! Next big thing: resource management • RSVP & DiffServ were developed to manage this very situation with regard to network video • RSVP & DiffServ are never going to happen • Gregory Bell, “Failure to Thrive: QoS and the Culture of Operational Networking,” Proceedings of the ACM SIGCOMM 2003 Workshops, RIPQoS Workshop. • Next Big Thing? : Pluggable Congestion Management • AIMD for internet (can even mimic multistream TCP behavior) • Fixed rate for PVCs and switched lambdas • Get hints from monitoring services (NWS) to manage flow end-to-end

  20. Concern about the future of BWC • The future and eternal reigning champion of the BWC?

  21. Concern about the future of BWC • The future and eternal reigning champion of the BWC? iperf --udp -b <infinity>

  22. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this?

  23. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this? • There is not technical solution to this problem • It is a social problem, not a technical one

  24. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this? • There is not technical solution to this problem • Can’t ban UDP • Can’t eliminate high bandwidth category

  25. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this? • There is not technical solution to this problem • Can’t ban UDP • Can’t eliminate high bandwidth category • Must answer • Does this advance science or art? • Is this a glimpse of the future?

  26. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this? • There is not technical solution to this problem • Can’t ban UDP • Can’t eliminate high bandwidth category • Must answer • Does this advance science or art? • Is this a glimpse of the future? • What the heck are judges for anyways?

  27. Wes Bethel Brian Tierney Dan Gunter Jason Lee Eli Dart Mike Bennett Chip Smith John Christman Raju Shah Ed Seidel Gabrielle Allen Tom Goodale Thomas Radke Mirek Ruda Ludek Matskya Acknowledgements • NERSC • LBNL • NCSA • PSC • ANL • AEI/Potsdam • University of Amsterdam • ESNet • QWEST • Abilene • StarLight • Teragrid • Force10 Networks • Scinet

  28. More Information • Visapult: • V1.0: http://vis.lbl.gov/projects/visapult • V2.0: http://vis.lbl.gov/projects/visapult2 • Cactus • http://www.cactuscode.org • VisPortal (Grid Portal for managing Visapult) • http://frost.lbl.gov/visportal/ • SC2002 Global Grid Testbed • http://scb.ics.muni.cz/static/SC2002/Testbed.html • Related Publications • Special Issue on Visualization on the Grid, IEEE Computer Graphics and Applications, Volume 23, Number 2, March/April 2003.

More Related