Protocol Boosters A Kernel-Level Implementation A Hardware-Level Implementation And then some …..

1. Protocol Boosters A Kernel-Level ImplementationA Hardware-Level ImplementationAnd then some ….. W. S. Marcus Senior Research Scientist Telcordia Technologies wsm@research.telcordia.com

2. My Objectives • Introduce Protocol Booster design methodology • Describe a kernel-level implementation • Describe a hardware-level implementation • Highlight applications and results

3. Protocol Boosters:What are they? • Conceived as software/hardware modules that: • Transparently,selectively, and robustly enhance existing communications protocols. • Are used to incrementally construct new communications protocols. • Allow protocol design to track improvements in networking technologies. • Reduces inefficiencies associated with designing for the worst-case. • Permit on-the-fly adaptation/customization of a protocol.

4. BASE PROTOCOL New Protocol Booster Message Exchange Multi- Element Protocol Booster Element A Multi- Element Protocol Booster Element A’ Single Element Protocol Booster Host X Host Y Protocol Boosters:What are they? (continued)

5. Protocol Boosters:Example: FZC UDP/IP UDP/IP’ Protocol FZC Parity Packet Stream FZC Parity Packet Generator MSGn+2 MSGn+1 MSGn MSGn-1 MSGn-2 FZC Packet Re- Generation Host X Host Y FZCy+1 FZCy lossy wireless network

6. Protocol Boosters:Example: FZC UDP/IP UDP/IP’ Protocol FZC Parity Packet Stream FZC Parity Packet Generator MSGn+2 MSGn+1 MSGn MSGn-1 MSGn-2 Host X Host Y FZCy+1 FZCy lossy wireless network

7. Protocol Boosters:Example: FZC (continued) • Desirable for • Applications with latency constraints • Applications for multicast distribution • Applications that respond to packet loss with retransmissions • Networks with no, or slow, return channels • Not desirable for congestion loss (use other boosters) • Employs a novel FEC scheme that • Based on Reed-Solomon Code • Allows fast software implementation (Mb/s) • Receiver need not know the number of parities sent • Receiver need not wait for, nor calculate, any parity packets beyond the number of missing data packets.

8. Protocol Boosters: Protocol Boosters and other adaptation technologies • Link Layer Services • Protocol Conversion/Termination • Protocol Boosters • Active Networking • Basic differences • transparency • robustness • selectivity • dynamism • ease of deployment • flexibility • generality

9. Protocol Boosters: Some Guiding Principles: • Protocol Boosters • Can reside anywhere in the network. • Can operate within the confines of a single network element. • Can be distributed over several network elements. • Can add, delay, or delete end-to-end messages. • Never modifies syntax or semantics of end-to-end protocol exchanges. • Transparent to protocol being boosted and elimination of any part of the booster does not, in itself, prevent end-to-end communications. • Are instantiated or revoked on-the-fly based on policies (e.g., observed network behavior, time of day, etc.) • Can be nested and concatenated. • Are an Active Networking programming model.

10. Active Networks: General Principles: • Architecture for supporting rapid, reconfigurable, and dynamic services on a per packet basis!!! • Each packet can deliver new functionality into the interior of the network. • Secure packet processing • Safe packet processing • Techniques for code mobility and service deployment

11. Protocol Boosters: Booster Modules • Monitoring • History, Trace • Debugging • Add, Delay, Delete, Duplicate • Error Control Family • ARQ-R, ARQ-A, ACK-COMP • FZC, ED, RO, DUP-DTEC, MSS • FEC, DAT-COMP, SEC <-- Violate guiding principles • PMOD

12. Protocol Boosters: Implementations • Adhere to guiding principles. • Focus on performance, e.g., below the EE/NodeOs line. • Kernel-level implementation for boosting IP • Procedures followed by a protocol • ARQ, FZC, flow control, signaling etc. • Packet oriented processing • Hardware-level implementation for boosting ATM at the OC-3c line rate. • Core functions used by a protocol • FEC, CRC checking and calculation, encryption, authentication, packet filtering, compression etc. • Bit oriented processing

13. Protocol BoostersKernel-levelimplementation II: Linux 2.0.32 • Individual protocol booster modules are implemented as loadable kernel modules, lkms. • Six interfaces per module • instance manager (includes /proc file system hooks) • input, output, and forward interfaces • booster channel interface (out-of-band channel) • ioctl interface • A policy manager (lkm) uses a variant of the Linux IP firewall mechanism to construct flow traps directed at sequences of booster modules. • A booster manager manages the loading/unloading of the individual boosters via the kerneld facilities, presents statistics via the /proc filesystem, manages the flow of packets through the booster modules. • Booster deployment via a client/server paradigm (presently under human intervention).

14. Protocol BoostersKernel-levelimplementation II: Linux 2.0.32 (continued) USER B USER C USER D USER E SERVER user kernel TCP UDP Policy Manager Booster A Booster Manager BOOST Booster B Booster C ICMP IGMP IP Device

15. Protocol BoostersKernel-levelimplementation II: Linux 2.0.32 (continued) • Booster Construction timer queue Trap booster instance booster instance booster instance Mux booster sequence event queue 0

16. Protocol Boosters :Programmable Protocol Processing Pipeline (P4) Bypassbuffer Switchingarray IIF OIF PE …… PE PE

17. Protocol Boosters :Programmable Protocol Processing Pipeline (P4) PE PE PE PE PE PE • SRAM (Altera 10K-series) based programmable devices • Switching array (reconfiguration time = 1us) • Processing implemented in hardware • Dynamic reconfiguration during run time (device download time = 100ms)

18. ttcp XXP F*C Loss F*C XXP ttcp Protocol Boosters:Experimental Setup • Uses ttcp [with UDP and TCP options] • F*C adds redundant packets • Loss module removes packets • FZC operates at access link line rates • Unicast and multicast tested • FEC operates at OC3c rates Ethernet Transmitter Receiver

19. Protocol Boosters:Kernel-level Implementation:FZC with Random Errors

20. Protocol Boosters:Kernel-level Implementation:FZC with Bursty Errors

21. Protocol Boosters:P4 Implementation:FEC results

22. Protocol Boosters:Kernel-Level:Demonstration QoS Testbed DynamicNetwork Noisy Wireless Link Host B Host A Host C High Delay Network Congested Network Host D Local retransmissions across noisy wireless access network Add (or retransmit) parity packets on multicast trees Perform ACK manipulation over high delay networks Local forwarding of packets to host in dynamic network Applied selectively (e.g., mobile node signalling)

23. Wired-2 Net Wireless-2 Net Wired-1 Net Wireless-1 Net Protocol Boosters:Kernel-Level:Demonstration QoS Testbed 10 Mb/s wired Ethernet Backbone Net • 5 PCs running Linux 2.0.32 enhanced with: • DVMRP (xerox mrouted v3.9.beta3), Mobile IP (Stanford MosquitoNet v1.0.4), DHCP (ISC dhcpd v1.0) • Protocol Boosters (v0.3), Multicast Proxy (Bellcore v0.1) 192.4.12.151 192.4.12.206 192.4.12.150 lotus twolf tdevil 192.168.52.129 192.168.51.129 192.168.50.129 192.168.51.130 rose 2 Mb/s (915 MHz ISM band) WaveLAN (2 virtual nets) 192.168.51.193 192.168.52.194 192.168.50.194 Roaming 192.168.51.194 daisy

24. Protocol Boosters:Kernel-Level:Demonstration QoS Testbed 10 Mb/s wired Ethernet TCP TCP • 6 boosters used! • FZe adds parity packets; FZd regenerates data packets • DLY delays packets (sets RTT); LOS drops packets (sets p) • MSS reduces TCP’s Max Segment by 16 bytes (for parity) • POR limited reordering (to prevent triple-DUP ACK) • TCP throughput is O(RTT/px) where x=1/2 for small p • FEC reduces p to increase effective throughput MSS FZe DLY LOS FZd POR Source Router Router

25. Protocol Boosters:Kernel-Level:Demonstration QoS Testbed

26. Protocol Boosters:Kernel-Level:Demonstration QoS Testbed Parity Data 3 Data 2 Data 1 • FEC booster in multicast distribution tree • Transparently adds h parity packets (not change data packets) • Downstream receivers recover any h missing data packets • Same parity recovers different packets at each receiver • Preemptively add parity packets (real-time applications) • Reactively retransmit parity (max number of missing packets) FEC Booster Parity Data 3 Data 2 Data 1 Parity Data 3 Data 2 Data 1

27. Protocol Boosters:Hardware-level:PMODs General: • Decide which booster is necessary • Activate booster at right place and right time • Coordinate dependencies among boosters • Signaling P4 Specific: • Manage limited hardware resources • Predict the need for booster and configure in advance

28. Protocol Boosters:Hardware-level:PMODs P4: • BER-monitor based on AAL-5 CRC-32 • Checks AAL-5 packet • bad: X=1, good: X=0 • Calculates moving average: • Y[n]=Y[n-1]-X[n-256]+X[n] Controller: • Collects data from BER monitor (reads Y) • Relates BER-monitor data with actual BER

29. Protocol Boosters:Conclusions • Devised an useful model for incremental protocol construction • Demonstrated high-performance implementations of the model • Demonstrated the value of Protocol Boosters for UDP and TCP applications • Anticipate porting ‘best of the breed’ to SwitchWare/PLAN • Technology Transfer to Army Research Lab Work (Airborne Communications Node - ACN) • TCP performance improvements promising, but open issues • How adapt to different versions of TCP (e.g., when ACK) • Better understand FZC booster and TCP interaction • Policies