Experiences with the QWEST/SCinet Bandwidth Challenge 2000-2002

1. Experiences with theQWEST/SCinet Bandwidth Challenge2000-2002 John Shalf, Wes Bethel LBNL

2. Shaky City Data Caches HPC Resources Simulations Sensor Nets STM Collaborators Handheld Devices Dr Jane

7. Backend Source Volume Viewer Component Visapult Architecture

8. Source Volume Visapult Architecture 3D 2D

9. Source Volume Visapult Architecture 3D Gigabits 2D Megabits

10. Source Volume Visapult Architecture 3D Gigabits 2D Megabits

11. SC2000 Demo Configuration

12. SC2000 Network Throughput

13. Refactoring the Design • Congestion avoidance • Good for internet • Bad bad baaaad for PVCs and other dedicated networks. (switched lambdas?) • Multistream TCP • Erratic performance • Requires a lot of tuning • Unfriendly to other users • Unfriendly to visualization applications • We want full control of the “throttle” • Very much like network video

14. Refactoring the Design • TCP is the wrong thing for interactive vis! • Layer 3 latency/jitter (all buffering effects) • Poor response to bursty traffic • Vis needs interactivity and minimal latency! • Network Video / UDP streams • Present packets to app. immediately (low latency) • Full control of data rate • Lossy, but effects of loss can be managed • SOCK_RDM

15. Effect of Loss on Visapult Evolving Binary Black Hole Merger Simulation (100 timesteps)

16. SC2001 BWC Testbed

17. SC2001 BWC Results • 3.3 Gigabits Peak (MRTG of ESNet/QWEST OC-48) • Lots of time spent tuning (packet pacing is nontrivial)

18. SC2002 Planning • Better UDP packet pacing with poor clock granularity • Error diffusion algorithm from dithering • Omnidirectional viewing for Visapult • Nascent Grid Infrastructure • Global Grid Testbed Collaboration (GGTC) • GGTC won 3 out of 4 HPC challenge awards

19. Global Testbed

22. Bandwidth Scavenging Service Czech Republic 90% utilization on path from Brno To GEANT (to SC) via Praha

23. Steady @ 16+ Gigabits!

25. Why do we do this?? • Opportunity to test the next generation network capacity (2 years before production) • Opportunity to test software ideas that will be required to make use of the next generation network (2 year lead) • Opportunity to test future network engineers (give them a 2 year lead on the problems with future networks) • Prescient • I-Way 1995 (presaged vBNS experiences and “The Grid”) • 1997 multi-stream tests (presaged multi-stream TCP protocols) • 2001 10GigE and UDP-based protocols • 10GigE backbone for SC2002 • RUDP, SABUL, TSUNAMI user-space UDP-based protocols

26. What should you know for 2003 • Manual throttle (UDP-based protocols) are here to stay. • Hopefully SOCK_RDM will cover most needs • Whaaa? Those idiots are going to burn down the network! Next big thing: resource management • RSVP & DiffServ were developed to manage this very situation with regard to network video • RSVP & DiffServ are never going to happen • Gregory Bell, “Failure to Thrive: QoS and the Culture of Operational Networking,” Proceedings of the ACM SIGCOMM 2003 Workshops, RIPQoS Workshop. • Next Big Thing? : Pluggable Congestion Management • AIMD for internet (can even mimic multistream TCP behavior) • Fixed rate for PVCs and switched lambdas • Get hints from monitoring services (NWS) to manage flow end-to-end

27. Concern about the future of BWC • The future and eternal reigning champion of the BWC?

28. Concern about the future of BWC • The future and eternal reigning champion of the BWC? iperf --udp -b <infinity>

29. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this?

30. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this? • There is not technical solution to this problem • It is a social problem, not a technical one

31. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this? • There is not technical solution to this problem • Can’t ban UDP • Can’t eliminate high bandwidth category

32. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this? • There is not technical solution to this problem • Can’t ban UDP • Can’t eliminate high bandwidth category • Must answer • Does this advance science or art? • Is this a glimpse of the future?

33. Concern about the future of BWC • The future and eternal reigning champion of the BWC? • How do we prevent this? • There is not technical solution to this problem • Can’t ban UDP • Can’t eliminate high bandwidth category • Must answer • Does this advance science or art? • Is this a glimpse of the future? • What the heck are judges for anyways?

34. Wes Bethel Brian Tierney Dan Gunter Jason Lee Eli Dart Mike Bennett Chip Smith John Christman Raju Shah Ed Seidel Gabrielle Allen Tom Goodale Thomas Radke Mirek Ruda Ludek Matskya Acknowledgements • NERSC • LBNL • NCSA • PSC • ANL • AEI/Potsdam • University of Amsterdam • ESNet • QWEST • Abilene • StarLight • Teragrid • Force10 Networks • Scinet

35. More Information • Visapult: • V1.0: http://vis.lbl.gov/projects/visapult • V2.0: http://vis.lbl.gov/projects/visapult2 • Cactus • http://www.cactuscode.org • VisPortal (Grid Portal for managing Visapult) • http://frost.lbl.gov/visportal/ • SC2002 Global Grid Testbed • http://scb.ics.muni.cz/static/SC2002/Testbed.html • Related Publications • Special Issue on Visualization on the Grid, IEEE Computer Graphics and Applications, Volume 23, Number 2, March/April 2003.