IP over P2P: Enabling Self-configuring Virtual IP Networks for Grid Computing

1. IP over P2P: Enabling Self-configuring Virtual IP NetworksforGrid Computing Arijit Ganguly, Abhishek Agrawal, P. Oscar Boykin, Renato Figueiredo University of Florida IPDPS 2006

2. What is the talk about? • Convergence of Grid and P2P technologies1 • Context of network virtualization 1 On death, taxes, and the convergence of peer-to-peer and Grid Computing. Foster et al. IPTPS 2003

3. Outline • Virtual networking and Grid Computing • Related work • Our approach – IP over P2P • Experimental evaluation • Conclusion and Future work

4. Tunnel Background - Virtual Private Networks Rhodes, Greece • Install Cisco VPN client • Connect to VPN gateway Internet access User inside ACIS private network Internet router VPN gateway NAT/Firewall Files, emails, compute cycles printers

5. Purdue Purdue NAT NAT router LSU router router LSU router Internet Internet router router router router SSH only Firewall Firewall SSH only NAT/Firewall NAT/Firewall Northwestern Florida Northwestern Florida Grid user Grid user Grid scenario Issues: • Idiosyncrasies of heterogeneous access • Network Address Translation • Traffic generated by untrusted code from Grid users – DoS attacks, viruses

6. Virtual Network Purdue NAT router LSU router Internet router router Firewall SSH only NAT/Firewall Northwestern Florida Grid user Virtual network of Grid resources

7. Virtual networking for Grids • VNET (Northwestern University) • Bridge a remote Virtual Machine (VM) to a client network • VIOLIN (Purdue University) • Virtualized network components • Isolated from real physical network • ViNe (University of Florida) • Virtual IP network of Grid resources • To be presented on Friday (Session 32) Common technology: Overlay tunneling What differentiates us: P2P routing

8. Motivations for P2P • Scalability and Self-configurability • Manual effort required to add a new node constant • Independent of size of the network • Resiliency • Robust P2P routing • Accessibility • Ability to traverse NAT • Hole punching1 1 RFC 3489 - STUN - Simple traversal of User Datagram Protocol through Network Address Translators

9. Our approach – IP- over-P2P (IPOP) • Isolation • Virtual address space decoupled from Internet address space • Self-configurability • Automatic setup of routes and topologies • Decentralized • No global state • No central points of failure • VM mobility • Decentralized NAT traversal • No changes to NAT configuration • No globally deployed STUN servers #affiliation condor_wow #transport udp #port 15000 #number of remote TAs 2 #list of TAs brunet.udp://planetlab-01.bu.edu:15000 brunet.udp://planetlab1.cs.purdue.edu:15000 #virtual interface tap0 #virtual IP address of tap0 172.16.1.5 #MAC address of tap0 CB:DF:E7:20:60:35

10. IPOP - Architecture Overview • IP tunneling over P2P overlay networks • UDP, TCP • Virtual IP packet capture and injection through tap interface • Builds upon Brunet P2P library

11. Node X Node Y application IPOP IPOP tap0 tap0 (172.16.0.9) (172.16.0.10) eth0 eth0 Socket s = new Socket(“172.16.0.10:3000”); s.connect(); ServerSocket serv = new ServerSocket(“172.16.0.10,3000); serv.accept() IPOP – Packet capture and routing • Extract IP from Ethernet • Encapsulate IP inside P2P • Extract IP from P2P • Encapsulate in Ethernet application Y X

12. Brunet P2P architecture • Ring-structured overlay network topology • Nodes ordered on 160-bit addresses • Overlay link: • Near: neighbor connections • Far: connections across ring U V X Y Multi-hop path between X and Y Far connection Near connection

13. Brunet P2P architecture (2) • Routing • Constant number of connections • O(log2(n)) overlay hops • O(log(n)) connections • O(log(n)) overlay hops • n connections • 1-hop • C# library, supports: • Connection setup and maintenance • NAT traversal

14. Sends packet: Src = 10.5.144.69:5000 Dst = 216.239.37.99:80 Translated: Src = 128.227.56.83:5126 Dst = 216.239.37.99:80 Sends packet: Src = 216.239.37.99:80 Dst = 10.5.144.69:5000 Sends packet: Src = 216.239.37.99:80 Dst = 128.227.56.83:5126 NAT Tables Network Address Translation (NAT) 128.227.56.83 216.239.37.99 10.5.144.69 Public host NAT Host A 10.5.144.69:5000  128.227.56.83:5126 Outgoing packet to 128.227.56.83:5126 Applications on NATed hosts can learn their NAT assigned IP:port

15. N:Y S:B R:A M:X Exchange each other’s NAT assigned IP:port M:X N:Y S:B R:A Src = S:B Dst = M:X Src = N:Y Dst = M:X Dropped Src = R:A Dst = N:Y Src = M:X Dst = N:Y Src = M:X Dst = S:B NAT M NAT N 128.139.156.90 128.227.56.83 NAT traversal – Behind NATs Outgoing packet to N:Y (hole punched) Outgoing packet to M:X (hole punched) Allow

16. Experiments • Latency overhead and throughput of single overlay link • LAN and WAN • MPI application over IPOP • Light Scattering Spectroscopy (LSS) • Multi-hop routing experiments • More than 100 node network on PlanetLab

17. Latency (single IPOP link) • Two IPOP nodes separated by single overlay hop • ACIS – ACIS for LAN • ACIS – VIMS for WAN • Ping times between two nodes • 6ms-11ms overhead per packet for ICMP ping • Relative overhead is smaller in Wide-Area ACIS: Florida VIMS: Virginia

18. Latency overhead - analysis • Reasons for high LAN overhead: • Double traversal of kernel stack • C# runtime • User-level overlay – context switches • Other user-level overlays (VNET, Violin) report few-ms latency overheads

19. Throughput (single IPOP link) • Two IPOP nodes separated by single overlay hop • ACIS – ACIS for LAN • ACIS – VIMS for WAN • “ttcp” • file transfer sizes (13.09 MB, 92.97 MB) • 1.9MB/s LAN bandwidth (20% of physical 9.4 MB/s) • 1.2MB/s WAN bandwidth (80% of physical 1.5 MB/s) ACIS: Florida VIMS: Virginia

20. LSU Florida router NAT/Firewall Firewall router Internet router NAT VIMS NAT/Firewall Real Application – Parallel LSS • MPI + NFS + SSH11 1 Support for Data-Intensive, Variable-Granularity Grid Applications via Distributed File System Virtualization - A Case Study of Light Scattering Spectroscopy. Figueiredo et al. CLADE 2004

21. Real Application – Parallel LSS • With IPOP, could run “parallel LSS” unmodified • No changes to NAT/Firewall rules • Achieve parallel speedup

22. PlanetLab experiments • Demonstrate ease of adding a new node and achieving IP routability in WAN environment • 118 node TCP-based overlay on PlanetLab • Connect two IPOP nodes in ACIS lab to PlanetLab network • Measure ping times between nodes • Average: 1617 ms; Std Dev: 2098 ms

23. Planetlab experiments (analysis) • Issues: • High-load (>10) on nodes in routing path • Geographically unaware p2p routing • Packets between machines in Florida routed through machines in California • Improvements: • Direct overlay link setup between communicating nodes • No concerns of load and inefficient p2p routing

24. Conclusion • Our contribution: • Novel virtual IP network based on P2P overlay • Scalable andSelf-configurable • Resilient • NAT traversal • Experiments showed feasibility of using P2P approach for virtual networking

25. Future work • Overhead of TCP or UDP • Raw sockets or Ethernet-based overlay edges • Kernel level extensions • Tap module with encapsulation and bridging • Reduce context switches

26. Related Work • Virtual Networking • VIOLIN • VNET • ViNe (Session 32) • Internet Indirection Infrastructure (i3) • Support for mobility, multicast, anycast • Decouples packet sending from receiving • Based on Chord p2p protocol • IPv6 tunneling • IPv6 over UDP (Teredo protocol) • IPv6 over P2P (P6P)

27. Acknowledgments • In-VIGO team at UFL • National Science Foundation • Middleware Initiative (http://www.nsf-middleware.org) • Research Resources Program • nCn center • Resources • Peter Dinda (Northwestern University) • SURA/SCOOP • IBM Shared University Research Questions?

28. Thank You