Building a massively scalable serverless VPN using Any Source Multicast

1. Building a massively scalable serverless VPN using Any Source Multicast • Athanasios Douitsis • Dimitrios Kalogeras National Technical University of Athens

2. IPSec IP TCP+SSL IP IP IPSec UDP L2TP PPP IP Popular VPN solutions • PPP over L2TP over IPSec transport mode • IPSec tunnel mode • OpenVPN Trend: Usage of a central VPN concentrator A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

3. VPN central concentrator considerations • Single point of failure • Reliability impact • Security impact • Passage of all client traffic through the concentrator • Impact on VPN concentrator resources (CPU, network) • Impact on network near the VPN concentrator A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

4. Alternative: P2P instead of hub topology P2P Communication through the multicast cloud No need for a central VPN server A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

5. Any Source Multicast as a Shared Medium • Election of a predefined common multicast group (G) as the shared medium • Easy subscription of any node to the shared medium (IGMP join G) • Transmissions inside the shared mediumare received by all listeners • Any node can also transmit messages to the shared medium G • No contention issues inside G • All VPN members directly connected to the L2 VPN A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

6. UDP Ethernet Encapsulation of L2 packets inside UDP multicast • One Ethernet frame inside each UDP packet • UDP Destination = multicast group G • UDP source = actual node IP address (unchanged) • Ethernet Source = Host generated MAC address (some constrains apply) • Ethernet destination = Destination MAC address (more on that later) A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

7. Duality between Ethernet and IP multicast personality • For each single node: • Real global source IP address • Virtual VPN source MAC address • 1-1 relationship between global IP address and VPN Source MAC address • Generation of VPN Source MAC address from global IP address: Just add two bytes at the front • MAC uniqueness is guaranteed • Example: If Source IP == 1.2.3.4, then VPN Source MAC := 0a:0a:01:02:03:04 • Make sure 0a:0a doesn’t clash with real vendor A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

8. UDP: Source: 1.2.3.4 Destination: 224.1.2.3 Ethernet: Source: 0a:0a:01:02:03:04 Destination: 0a:0a:05:06:07:08 Encapsulation of Ethernet inside UDP: explanation Host A sends a packet to the VPN LAN Host A: Real IP: 1.2.3.4 Virtual MAC: 0a:0a:01:02:03:04 The multicast enabled IP network takes the packet and sends it to all 224.1.2.3 subscribers multicast cloud Host B: Real IP: 5.6.7.8 Virtual MAC: 0a:0a:05:06:07:08 Host C: Real IP: 9.10.11.12 Virtual MAC: 0a:0a:09:10:11:12 Subscriber B receives the packet and forwards it through its networking stack Subscriber C receives the packet but is really not interested as its MAC != packet destination MAC A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

9. Extension: Many MACs behind an IP • Hiding of many VPN MAC addresses behind one single IP address • 1-to-many relationship between global IP address and VPN MAC address • Generation of VPN MAC address from global IP address: Again, Just add two bytes at the front • Example: If Global IP == 1.2.3.4, then MAC := 0a:xx:01:02:03:04 • 01< xx< ff • 256 MACs max behind one real IP • Nice for virtualization setups A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

10. Optimization: Usage of multicast only when needed • Modern Ethernet Switches: • MAC to port lookup table • Delivery of Broadcasts (mostly ARP, ICMPv6 etc) to all ports • Delivery of packets with unknown dest. MAC to all ports • Delivery of packets with known dest. MAC only to corresponding port • Modification of our virtual L2 VPN towards the same goal. • MAC to global IP table • Broadcasts (mostly ARP, ICMPv6 etc) to all G subscribers • Packets with unknown dest. MAC to all G subscribers • Packets with known dest. MAC only to corresponding IP using Unicast! A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

11. UDP: Source: 1.2.3.4 Destination: 224.1.2.3 UDP: Source: 5.6.7.8 Destination: 1.2.3.4 Ethernet: Source: 0a:0a:01:02:03:04 Destination: 0a:0a:05:06:07:08 Ethernet: Source: 0a:0a:05:06:07:08 Destination: 0a:0a:01:02:03:04 Unicast Optimization explained Host A sends a packet to the VPN LAN Host A: Real IP: 1.2.3.4 Virtual MAC: 0a:0a:01:02:03:04 The multicast enabled IP network takes the packet and sends it to all 224.1.2.3 subscribers multicast cloud Host B: Real IP: 5.6.7.8 Virtual MAC: 0a:0a:05:06:07:08 Subscriber B receives the packet and adds the appropriate entry in it MAC-to-IP table Subscriber B responds with a direct unicast packet to A because it knows its global IP A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

12. Implementation • Kernel interface • Usage of the versatile tun/tap driver • virtual tap0 ethernet device • /dev/net/tap character device • User space application • Reads from /dev/net/tap and writes to UDP socket • Reads from UDP socket and writes to /dev/net/tap A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

13. Ingress data flow • Arrival of packet from network • Reading of the packet from the socket • (optional) Parsing of the packet and caching of MAC-to-IP pair • De-capsulation of the Ethernet Frame from the UDP packet • (optional) Other kinds of meddling with the de-capsulated Ethernet frame • Writing of Ethernet Frame to /dev/net/tap • Kernel sees an Ethernet Frame coming from i/f tap0 A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

14. Egress data flow • Generation of an Ethernet Frame on the tap0 interface by the kernel • Reading of the Ethernet Frame from the /dev/net/tap device by the user space VPN application • (optional) Consultation of the MAC-to-IP cache table • Encapsulation of the Ethernet Frame inside a UDP packet • (optional) Other kinds of meddling with the soon-to-be-transmitted Ethernet packet • Transmission of the packet either as multicast or Unicast A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

15. Ingress/Egress explained Egress Data flow VPN Member Host Kernel Space User Space send() to socket multicast socket VPN Client tun/tap driver /dev/net/tap char device read() from device A frame is generated i/f tap0 multicast cloud VPN Member Host recv() from socket Kernel Space User Space Ingress Data flow multicast socket VPN Client tun/tap driver /dev/net/tap char device write() to device i/f tap0 The frame is delivered inside tap0 A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

16. Security considerations • Problem: Multicast Group joinable and listenable by anyone/anywhere • Possible solution #1. Communications are secured at the encapsulation layer, e.g. Secure Multicast. • Multicast Group Domain of Interpretation (RFC3547). Downside: group controller/key server required. • But: “Normal” IPSec perfectly usable for unicast communications • Possible solution #2. Communications are secured inside the VPN LAN, e.g. secure LAN. • Usage of IPSec inside the VPN LAN • Possible solution #3. Use secure protocols (>L3) inside the VPN LAN • HTTPS, SSH, SFTP A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

17. Conclusion: Benefits • Resiliency: No central server needed • Scalability: Solution can scale to very large number of nodes with the Unicast optimization enabled • Transparency: tap0 for all intents and purposes an ordinary Ethernet interface • Portability: Simple implementation easily portable to any platform. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

18. But: Drawbacks • Security provisioning somewhat tricky • server required for GDOI • IP Multicast required on all nodes (some networks still don’t support multicast) A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

19. Extensions / Future Ideas • Virtual Routers between different VPNs • Physical Gateways to a VPN • bridging of a real ethernet device with a tap • Packet filters on tap devices • Many virtual VPN members inside one physical entity • Can work well with hardware virtualization A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast

20. Thank You! • Questions? • {adouitsis|dkalo}@noc.ntua.gr A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast