VTL: A Transparent Network Service Framework

1. VTL: A Transparent Network Service Framework John R. Lange and Peter A. Dinda Prescience Lab Department of Electrical Engineering and Computer Science Northwestern University http://plab.cs.northwestern.edu

2. Transparent Network Services • Manipulate data and signaling of flows/connections to add services to existing unmodified applications and OSes • High Level transformations of Low Level traffic • Transparency: Manipulations invisible to guest environment • VTL (Virtual Traffic Layer) • A framework for creating Transparent Network Services • Wide range of possible services • Many useful for HPDC

3. Outline • Defining Transparent Network Services • Motivation • VTL Framework • Architecture • Performance • Example Transparent Network Services • Protocol Transformations • Anonymous Networking • … • Conclusion and Future Work

4. Transparency • Improving Existing Unmodified Applications • Invisible to connection end points • No changes to guest environment • Seamless integration of networking techniques • Transparency readily available with VMS • Provide transparent bridge • Service integration below virtual hardware

5. Network Services • Implement high level functions • Operate on low level network traffic • Monitor • Control • Manipulate • Traffic Data • Signaling • Unique challenges in Virtual Environments • E.g. Migration

6. Motivation • HPDC 2005 -- VRESERVE • Automatic Optical Network Reservations for unmodified applications • Demonstrated performance gains over standard internet routes • Performance Issues • TCP applications ill suited for optical networks J. Lange, A. Sundararaj, and P. Dinda, Automatic Dynamic Run-time Optical Network Reservations, Proceedings of the 14th IEEE International Symposium on High Performance Distributed Computing, (HPDC 2005)

7. TCP over Optical Networks • Optical Networks have high BDPs • Bandwidth Delay Products • Very High bandwidth • Long distance • High relative latency • TCP breaks down D. Petravick, Fermilab

8. Typical BDP values • Assume endpoints are on opposite ends of the earth • Real world example: CERN and StarLight • Latency lower bound is ~60ms • Half circumference of earth / Speed of light • CERN <–> FNAL has a measured ~60 ms delay • D. Petravick, Fermilab • Optical Networks currently operate at 10 Gbps • But 1 GigE NICs are most common • TCP Window Size (BDP): • 10 Gbps ~= 70 MB • 1 Gbps ~= 7MB • SACK lookups cause TCP timeouts • Window size  1

9. Transparently Optimize high BDP flows • High performance protocols exist • UDT/SABUL, RBUDP, etc… • But applications must be configured for them • Need method of transforming TCP to UDT • Opens UDT connections based on SYNs • Transmits data segments over UDT

10. VTL • Transparent Network Service Framework • Network device interface • Packet modification and creation • Rapid prototyping and evaluation • Capabilities • Virtual TCP endpoint • Transparent packet generator • Acks, keep-alive • Packet header and content modifications • Not confined to virtual machines

11. VTL Components • Network Interface API • Reads/Writes packets to/from network interfaces • Packet Access API • Reading and writing packet data • State Models • Maintain state of connection endpoints

12. Network Interface API • Common interface for packet capture and injection • Virtual or Real devices • Unix or Windows • Built on PCAP and libnet • Operations • Connect/Disconnect • Read/Write • Packet notifications

13. Packet Access API • Packet inspection and modification • Primitives to access standard fields • Higher level functions built on primitives • Packet class queries • Field swapping • Header calculations • Derivative packet creation

14. Connection State Models • Maintain and manipulate protocol state • Layered architecture • Create packets belonging to a connection • State kept for both connection endpoints • Generate packets from either endpoint • API operation • Manual or packet based • Model Initialization • State Updates • Packet Creation

15. VTL Configuration Hosting Server (Windows or Unix) VM VMM (VMWare, Xen, etc) Host-only interface VNET Overlay Module UDT Flow Over Optical Network VTL VNET Physical interface Sundararaj, A., Gupta, A., , and Dinda, P. Increasing application performance in virtual environments through run-time inference and adaptation. In Proc. of the 14th IEEE International Symposium on High Performance Distributed Computing (HPDC) (July 2005)

16. Baseline Performance • Limited by Network Interface API • Implemented in user space • PCAP + libnet • Experimental setup • Simple interface bridge (virtual->real) • Xen bridge • Single process (half duplex) • Two processes (full duplex)

17. Baseline Performance Overhead Measurements Bandwidth (MB/s) Xen Bridge One VTL Process Two VTL Processes

18. Protocol Transformation for High BDP networks • Addresses performance of TCP over optical • VTL allows transformation of TCP flows to other transport protocols • VTL module acts as virtual TCP endpoint • Implements TCP states • SYN sequence (open) • FIN sequence (close) • Data Transfer over new protocol (established)

19. Code Example – Creating Packets int create_data_pkt(vtl_model_t * model, char * data, int data_len) { RawEthernetPacket data_pkt; create_empty_pkt(model, &data_pkt, INBOUND_PKT); memcpy(TCP_DATA(data_pkt), data, data_len); compute_ip_len(&data_pkt, data_len); compute_ip_checksum(&data_pkt); compute_tcp_checksum(&data_pkt); sync_model(model, &data_pkt); queue_pkt(&data_pkt); }

20. Performance Evaluation Setup • Comparing TCP vs. VTL + UDT • Added artificial latency to gigabit switch • Linux iproute2 + tc netem • TTCP benchmark • Standard TCP (Host to host) • TCP with intelligent socket buffers (Host to host) • VTL + UDT (Xen VM to Xen VM) • Note: No virtualization present for TCP tests • Same hardware

21. Performance Bandwidth (MB/s) Latency (ms)

22. More Transparent Network Services • Socks (TOR) • Subnet Tunneling • VM Migration Support (TCP keep alive) • Stateful Firewall • Performance Enhancing Proxies • RFC 3135 • Local acknowledgements

23. Anonymous Networking for Any Application • Tor Anonymous Network (http://tor.eff.org) • Anonymizes source of any TCP connection • Functions as a SOCKS proxy • Requires SOCKS application support VM Tor Server VMM (VMWare, Xen, etc) TOR NETWORK SOCKS Connection VTL Host-only interface TCP Connections + DNS lookups VTL Interface Hosting Service

24. Tor + VTL • VTL implements transparent SOCKS interface • VTL simulates a TCP endpoint • Extracts data segment from TCP packet and transmits it over SOCKS tunnel • Data from SOCKS is encapsulated into TCP packets and delivered to VM • Gotchas • DNS is UDP based • VTL handles DNS case for UDP • ARPs • VTL answers ARPs with a fake MAC address • All tcp connections from a VM are anonymized • No modification to OS or applications • User not restricted to applications implementing socks

25. Transparent Security • Iptables and Windows Firewall are now ubiquitous • Not perfect • Successful attacker can alter rules • Only as strong as the weakest link • VTL rules are not accessible by VM • Even if VM is compromised firewall rules are safe

26. Subnet Tunneling 123.123.1.0/24 10.10.0.0/16 123.123.1.1 VNET Proxy (PROXY1) Gateway (GW1) 123.123.1.50 VM1 LAN connection Internet VNET Overlay (Internet) Gateway Router 234.234.1.1 MAC Address Mismatch! VM2 Gateway (GW2) VNET Proxy (PROXY2) 234.234.1.50 234.234.1.0/24

27. Subnet Tunneling • Two VMs on different subnets communicating • Fast Path link is available between them • Bypasses routers • VMs use subnet gateway • Set gateway MAC as destination • VTL rewrites destination MAC addresses • Route packets on fast path link

28. Network Suspension during VM Migrations • A VM is suspended for a long duration • i.e. VM is migrating over WAN • Open TCP connections begin to timeout • In order to maintain connections VTL generates keep-alive packets • Secondary service must handle routing • i.e. VNET

29. Cooperative Selective Wormholing • Distributed traffic aggregation for Network Intrusion Detection Systems • Wormhole • Tunnel traffic from a remote sensor to backend NIDS • VTL mechanisms for packet capture and injection • Cooperative • Volunteer machines aggregate traffic • VTL implementation cross platform • Selective • Aggregates traffic that Volunteer client is not interested in • VTL mechanisms for packet inspection • J. Lange, P. Dinda, and F. Bustamante, Vortex: Enabling Cooperative Selective Wormholing for Network Security Systems, Proceedings of the 10th International Symposium on Recent Advances in Intrusion Detection (To Appear)

30. Future Work • Generalizable to complete IO framework • Performance • VMM based implementation • Automatic Service Adaptation

31. Conclusion • Transparent Network Services allow high level transformations of low level network traffic • VTL • A framework for creating Transparent Network Services • Wide range of potential services • Many useful for HPDC

32. Prescience Lab • http://plab.cs.northwestern.edu • Virtuoso • http://virtuoso.cs.northwestern.edu • John Lange • http://www.artifex.org/~jarusl

33. Vortex • Cooperative Selective Wormhole implementation • VTL • Traffic capture and injection • Packet modifications • Rewrite addresses • Anonymize packets • Cross platform functionality

34. Vortex Architecture VM Based Honeypot Commodity PC IDS Analysis Backend Windows/UNIX VM VNET Proxy Apps Vortex Physical Honeypot Operating System VTL Firewall PCAP libnet NIC VNET Overlay Backend Network

35. Subnet Tunneling 123.123.1.0/24 10.10.0.0/16 123.123.1.1 VNET Proxy (PROXY1) Gateway (GW1) 123.123.1.50 VM1 LAN connection Internet VNET Overlay Gateway Router 234.234.1.1 VM2 Gateway (GW2) VNET Proxy (PROXY2) 234.234.1.50 234.234.1.0/24