270 likes | 413 Views
Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments. Renato Figueiredo Associate Professor Center for Autonomic Computing ACIS Lab University of Florida. Outlook. Architecting autonomic virtual networks
E N D
Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments RenatoFigueiredo Associate Professor Center for Autonomic Computing ACIS Lab University of Florida
Outlook • Architecting autonomic virtual networks • Isolation, security, encapsulation, dynamic configuration, migration • Self-configuration, self-healing, self-optimization • Applications in cloud and collaborative environments • Virtual Private Clusters • Social VPNs • Archer: a collaborative environment for computer architecture simulation • Ongoing/future work
Background Resource aggregation: Cross-institution sharing, opportunistic computing, on-demand provisioning N A T Self-configuring End-to-end Virtual Private Network N A T Public Internet Collaboration, entertainment: streaming, data sharing, games 3
Self-organizing virtual networks • Focus: • Software overlays that provide virtual network infrastructure over existing Internet infrastructure • Why virtual? • Support unmodified TCP/IP applications and existing Internet physical infrastructure • Hide heterogeneity of physical network (firewalls, NATs), avoid IPv4 address space constraints • Why self-organizing? • Autonomous behavior: low management cost compared to typical VPNs • Decentralized architecture for scalability and fault tolerance
Virtual networking • Isolation: dealt with similarly to VMs • Multiple, isolated virtual networks time-share physical network • Key technique: tunneling (VPNs) • Related work • Grid computing • VNET (P. Dinda at Northwestern U.) • Violin (D. Xu at Purdue U.) • ViNe (J. Fortes at U. Florida) • PVC (F. Cappello at INRIA) • “P2P” VPNs • Hamachi, tinc, Gbridge
The IP-over-P2P (IPOP) Approach • Isolation • Virtual address space decoupled from Internet address space • Self-managing • Self-organizing, self-healing topology • Decentralized – structured peer-to-peer (P2P) • No global state, no central points of failure • Self-optimizing IP overlay routing • On-demand direct/relay connections • Self-configuring decentralized NAT traversal
Use case scenarios • Sharing resources/services in a virtual end host • VM provides isolation • Virtual appliances provide software encapsulation • Distributed virtual appliance clusters • Homogeneous software environment on top of heterogeneous infrastructure • Homogeneous virtual network on top of wide-area, NATed environments • Cross-institution collaboration; cloud-bursting
Virtual machines VM image Example: virtual clusters “WOWs” • Wide-area • Virtual machines (VMs) • Self-organizing overlay IP tunnels, P2P routing NOWs, COWs • Local-area • Physical machines • Self-organizing switching (e.g. Ethernet spanning tree) Installation image Switched network Physical machines
Use case scenarios • There are various successful overlays enabling peer-to-peer communication among users • VoIP sessions over skype • File transfers over bittorrent • iChat (video, chat, desktop sharing) • Application (and/or platform) specific • Users: richer set of applications over a generic IP network for communication and collaboration • But they don’t have public IPs, and don’t want to directly connect to all users – hence NATs • And they don’t want to or know how to configure and discover network services manually
carol.facebook.ipop 10.10.0.2 node0.alice.facebook.ipop 10.10.0.3 • Alice’s services: • Samba share • RDP server • VoIP, Chat • Advertise to Bob, Carol Social Network API Alice’s public keys Bob’s public keys Carol’s public key Social network Information system Social network (e.g. Facebook) Social Network Web interface Example: Social VPNs Overlay network (IPOP) Bob: browses Alice’s SMB share Alice Carol Bob
IP-over-P2P Tunneling • As in many other VPNs, use virtual network device to capture/inject IP (e.g. tap/tun) • Tunnel IP over UDP or TCP • Unlike traditional VPNs, tunnels are not established by an administrator • Rather, IPOP implements self-organizing techniques to discover, establish and maintain overlay links • Each IPOP peer is capable of picking packets, injecting packets, and routing
Unmodified applications Connect(10.10.1.2,80) Capture/tunnel, scalable, resilient, self-configuring routing and object store 10.10.1.1 Isolated, private virtual address space 10.10.1.2 Virtual network architecture Application Virtual Router Wide-area Overlay network VNIC Virtual Router Application VNIC
Overlay architecture • Bi-directional structured overlay (Brunet library) • Constant number of edges (K) per node • O((1/k)log2(n)) overlay hops • Self-organizing topology Ordered ID space Near edge Overlay router Overlay router Shortcut (far) edge
Overlay Edges • Abstract bi-directional communication channels • Edges can use various transports: • UDP; TCP; DTLS; Tunnel • UDP/DTLS: • NAT traversal UDP edge TCP edge “Tunnel” edge Overlay router Overlay router
NAT traversal • Reflection: learn NAT-mapped endpoints • From public overlay peers • Peers exchange “connect to me” through overlay • Set up hole punching • Self-configuring 2. Exchange learned Endpoint with peer 1. Reflection: udp://IP:port 3. Simultaneous open: NAT traversal
Self-healing structure • Greedy routing relies on consistent bi-directional ring topology • Faults in structure due to routing outages, symmetric NATs • Tunnel near edges Tunnel edge Unavailable physical path Peers exchange neighbor set
Self-optimization • Create direct edges based on traffic inspection • O(log2(N)) -> O(1) • Direct connection when NAT traversal possible • Relay through a peer – “far” tunnel edge 2. Exchange learned Endpoint with peer 1. Reflection: udp://IP:port 3. Simultaneous open: NAT traversal
Received by left and right neighbors Bootstrapping Forwarder New P2P node CTM request • Forms a “leaf” connection with a well-known node • Selected at random from list of “bootstrap” nodes • Sends “Connect to me” CTM request addressed to itself • Received by nearest neighbors
Autonomous IP allocation • One P2P overlay supports multiple IPOP namespaces • IP routing within a namespace • Each IPOP namespace: a unique string • Distributed Hash Table (DHT) stores mapping • Key=namespace • Value=DHCP configuration (IP range, lease, ...) • IPOP node configured with a namespace • Query namespace for DHCP configuration • Guess an IP address at random within range • Attempt to store in DHT • Key=namespace+IP • Value=IPOPid (160-bit) • IP->P2P Address resolution: • Given namespace+IP, lookup IPOPid
Avoiding overlay overheads LAN Router Application Virtual Router NIC Wide-area Overlay network Local Interface VNIC Application Virtual Router NIC Application VNIC
VN Interfaces Each machine has local VN Interface ARP, DHCP captured locally Router responds as gateway DHCP: DHT put/get
Supporting VN Routers Single VN (Router) for entire cluster Avoid need for VN software stack on end host Avoid VN overhead on LAN communication
VN Hybrid VN instance for each member in a cluster VN hosts in the same LAN bypass VN software stack
Autonomic features • Self-configuration [IPDPS’06, HPDC’06, PCgrid’07] • Routing tables using structured P2P links • NAT traversal, DHCP over DHT • Self-optimization [HPDC’06] • Direct shortcut connections created/trimmed based upon IP traffic inspection for fast end-to-end IP tunnels • Proximity neighbor selection based on network coordinate estimates for improved structured routing • Self-healing [HPDC’08] • “Tunnel” edges created to maintain overlay structure to deal with routing outages and NATs/firewalls that are not traversable • VLAN routers, overlay bypass within VLAN [VTDC09, SC09]
Overlay security architecture • Abstract senders encapsulate security logic • Supports both edge (point-to-point) and IPOP (end-to-end) authentication and encryption • Public key infrastructure • Keys/certificates • Symmetric key exchange • DTLS (Datagram TLS) library or native IPOP stack • UDP-based; amenable to NAT traversal • IPsec tunneling also supported
Performance • IPOP implementation • C# user-level router • Tap virtual network device
Security management • Overlay point-to-point and/or end-to-end security need to be configured • PKI management can be complex and error-prone • Certificate signing/distribution, revocation • Approach: leverage Web 2.0, social networking infrastructures for security management • SocialVPN: enable point-to-point VPN connectivity among socially-networked peers • GroupVPN: enable sharing of resources with all-to-all VPN connectivity within a group of users