CMPE 252A : Computer Networks

CMPE 252A : Computer Networks Chen Qian Computer Science and Engineering UCSC Baskin Engineering Data link layer

MAC addresses and ARP • 32-bit IP address: • network-layer address for interface • used for layer 3 (network layer) forwarding • MAC (or LAN or physical or Ethernet) address: • function:used ‘locally” to get frame from one interface to another physically-connected interface (same network, in IP-addressing sense) • 48 bit MAC address (for most LANs) burned in NIC ROM, also sometimes software settable • e.g.: 1A-2F-BB-76-09-AD hexadecimal (base 16) notation (each “number” represents 4 bits) Link Layer

LAN addresses and ARP each adapter on LAN has unique LAN address 1A-2F-BB-76-09-AD LAN (wired or wireless) adapter 71-65-F7-2B-08-53 58-23-D7-FA-20-B0 0C-C4-11-6F-E3-98 Link Layer

LAN addresses (more) • MAC address allocation administered by IEEE • manufacturer buys portion of MAC address space (to assure uniqueness) • analogy: • MAC address: like Social Security Number • IP address: like postal address • MAC flat address ➜ portability • can move LAN card from one LAN to another • IP hierarchical address not portable • address depends on IP subnet to which node is attached Link Layer

Question: how to determine interface’s MAC address, knowing its IP address? ARP: address resolution protocol ARP table: each IP node (host, router) on LAN has table • IP/MAC address mappings for some LAN nodes: < IP address; MAC address; TTL> • TTL (Time To Live): time after which address mapping will be forgotten (typically 20 min) 137.196.7.78 1A-2F-BB-76-09-AD 137.196.7.23 137.196.7.14 LAN 71-65-F7-2B-08-53 58-23-D7-FA-20-B0 0C-C4-11-6F-E3-98 137.196.7.88 Link Layer

A wants to send datagram to B B’s MAC address not in A’s ARP table. A broadcasts ARP query packet, containing B's IP address dest MAC address = FF-FF-FF-FF-FF-FF all nodes on LAN receive ARP query B receives ARP packet, replies to A with its (B's) MAC address frame sent to A’s MAC address (unicast) A caches (saves) IP-to-MAC address pair in its ARP table until information becomes old (times out) soft state: information that times out (goes away) unless refreshed ARP protocol: same LAN Link Layer

111.111.111.110 E6-E9-00-17-BB-4B 222.222.222.222 49-BD-D2-C7-56-2A Addressing: routing to another LAN walkthrough: send datagram from A to B via R • focus on addressing – at IP (datagram) and MAC layer (frame) • assume A knows B’s IP address • assume A knows IP address of first hop router, R (how?) • DHCP • assume A knows R’s MAC address (how?) • ARP B A R 111.111.111.111 74-29-9C-E8-FF-55 222.222.222.220 1A-23-F9-CD-06-9B 222.222.222.221 111.111.111.112 88-B2-2F-54-1A-0F CC-49-DE-D0-AB-7D Link Layer

MAC src: 74-29-9C-E8-FF-55 MAC dest: E6-E9-00-17-BB-4B IP src: 111.111.111.111 IP dest: 222.222.222.222 IP Eth Phy 111.111.111.110 E6-E9-00-17-BB-4B 222.222.222.222 49-BD-D2-C7-56-2A Addressing: routing to another LAN • A creates IP datagram with IP source A, destination B • A creates link-layer frame with R's MAC address as dest, frame contains A-to-B IP datagram B A R 111.111.111.111 74-29-9C-E8-FF-55 222.222.222.220 1A-23-F9-CD-06-9B 222.222.222.221 111.111.111.112 88-B2-2F-54-1A-0F CC-49-DE-D0-AB-7D Link Layer

MAC src: 74-29-9C-E8-FF-55 MAC dest: E6-E9-00-17-BB-4B IP src: 111.111.111.111 IP dest: 222.222.222.222 IP Eth Phy IP src: 111.111.111.111 IP dest: 222.222.222.222 IP Eth Phy 111.111.111.110 E6-E9-00-17-BB-4B 222.222.222.222 49-BD-D2-C7-56-2A Addressing: routing to another LAN • frame sent from A to R • frame received at R, datagram removed, passed up to IP B A R 111.111.111.111 74-29-9C-E8-FF-55 222.222.222.220 1A-23-F9-CD-06-9B 222.222.222.221 111.111.111.112 88-B2-2F-54-1A-0F CC-49-DE-D0-AB-7D Link Layer

IP Eth Phy MAC src: 1A-23-F9-CD-06-9B MAC dest: 49-BD-D2-C7-56-2A IP Eth Phy IP src: 111.111.111.111 IP dest: 222.222.222.222 111.111.111.110 E6-E9-00-17-BB-4B 222.222.222.222 49-BD-D2-C7-56-2A Addressing: routing to another LAN • R forwards datagram with IP source A, destination B • R creates link-layer frame with B's MAC address as dest, frame contains A-to-B IP datagram B A R 111.111.111.111 74-29-9C-E8-FF-55 222.222.222.220 1A-23-F9-CD-06-9B 222.222.222.221 111.111.111.112 88-B2-2F-54-1A-0F CC-49-DE-D0-AB-7D Link Layer

IP Eth Phy MAC src: 1A-23-F9-CD-06-9B MAC dest: 49-BD-D2-C7-56-2A IP src: 111.111.111.111 IP dest: 222.222.222.222 111.111.111.110 E6-E9-00-17-BB-4B 222.222.222.222 49-BD-D2-C7-56-2A Addressing: routing to another LAN • R forwards datagram with IP source A, destination B • R creates link-layer frame with B's MAC address as dest, frame contains A-to-B IP datagram B A R 111.111.111.111 74-29-9C-E8-FF-55 222.222.222.220 1A-23-F9-CD-06-9B 222.222.222.221 111.111.111.112 88-B2-2F-54-1A-0F CC-49-DE-D0-AB-7D Link Layer

Ethernet switch • link-layer device: takes an active role • store, forward Ethernet frames • examine incoming frame’s MAC address, selectively forward frame to one-or-more outgoing links when frame is to be forwarded on segment, uses CSMA/CD to access segment • transparent • hosts are unaware of presence of switches • plug-and-play, self-learning • switches do not need to be configured Link Layer

Switch: multiple simultaneous transmissions • hosts have dedicated, direct connection to switch • switches buffer packets • Ethernet protocol used on each incoming link, but no collisions; full duplex • each link is its own collision domain • switching:A-to-A’ and B-to-B’ can transmit simultaneously, without collisions A B C’ 1 2 6 4 5 3 B’ C A’ switch with six interfaces (1,2,3,4,5,6) Link Layer

Switch forwarding table Q:how does switch know A’ reachable via interface 4, B’ reachable via interface 5? A B C’ • A:each switch has a switch table,each entry: • (MAC address of host, interface to reach host, time stamp) • looks like a routing table! 1 2 6 4 5 3 B’ C Q:how are entries created, maintained in switch table? • something like a routing protocol? A’ switch with six interfaces (1,2,3,4,5,6) Link Layer

Source: A Dest: A’ MAC addr interface TTL 60 1 A A A’ Switch: self-learning • switchlearnswhich hosts can be reached through which interfaces • when frame received, switch “learns” location of sender: incoming LAN segment • records sender/location pair in switch table A B C’ 1 2 6 4 5 3 B’ C A’ Switch table (initially empty) Link Layer

Source: A Dest: A’ A’ A MAC addr interface TTL 60 60 1 4 A A’ A A’ A A’ A A’ A A’ A A’ A A’ Self-learning, forwarding: example • frame destination, A’, locaton unknown: A flood B • destination A location known: C’ selectively send on just one link 1 2 6 4 5 3 B’ C A’ switch table (initially empty) Link Layer

Interconnecting switches • switches can be connected together S4 S1 S3 S2 A F I D C B H G E • Q: sending from A to G - how does S1 know to forward frame destined to F via S4 and S3? • A:self learning! (works exactly the same as in single-switch case!) Link Layer

Institutional network mail server to external network web server router IP subnet Link Layer

network link physical link physical datagram datagram frame frame frame Switches vs. routers application transport network link physical both are store-and-forward: • routers: network-layer devices (examine network-layer headers) • switches: link-layer devices (examine link-layer headers) both have forwarding tables: • routers: compute tables using routing algorithms, IP addresses • switches: learn forwarding table using flooding, learning, MAC addresses switch application transport network link physical They become more and more similar! Link Layer

Data center networks • 10’s to 100’s of thousands of hosts, often closely coupled, in close proximity: • e-business (e.g. Amazon) • content-servers (e.g., YouTube, Akamai, Apple, Microsoft) • search engines, data mining (e.g., Google) • challenges: • multiple applications, each serving massive numbers of clients • managing/balancing load, avoiding processing, networking, data bottlenecks Inside a 40-ft Microsoft container, Chicago data center Link Layer

Data center networks • load balancer: application-layer routing • receives external client requests • directs workload within data center • returns results to external client (hiding data center internals from client) Internet Border router Load balancer Load balancer Access router Tier-1 switches B A C Tier-2 switches TOR switches Server racks 7 6 5 4 8 3 2 1 Link Layer

A Scalable, Commodity Data Center Network Architecture Mohammad Al-Fares Alexander Loukissas Amin Vahdat

Oversubscription Ratio Upper Link Bandwidth(UB) B B B …………… Server n Server 2 Server 1 Oversubscription Ratio= B*n/UB

Current Data Center Topology • Edge hosts connect to 1G Top of Rack (ToR) switch • ToR switches connect to 10G End of Row (EoR) switches • Large clusters: EoR switches to 10G core switches Oversubscription of 2.5:1 to 8:1 typical in guidelines • No story for what happens as we move to 10G to the edge Key challenges: performance, cost, routing, energy, cabling

Design Goals • Scalable interconnection bandwidth Arbitrary host communication at full bandwidth • Economies of scale Commodity Switch • Backward compatibility Compatible with hosts running Ethernet and IP

Fat-Tree Topology K Pods k/2 Aggregation Switch in each pod k/2 Edge Switches in each pod k/2 servers in each Rack

Fat-tree Topology Equivalent

Routing (k/2)*(k/2) shortest path! IP needs extension here! Single-Path Routing VS Multi-Path Routing Static VS Dynamic

ECMP（Equal-Cost Multiple-Path Routing) Static Flow scheduling limited multiplicity of path to 8-16 Advantage: No packet reordering! Modern Switch support! Extract Source and Destination Address Hash Function(CRC16) Determine which region fall in 2 3 4 1 0 Hash-Threshold

Two-level Routing Table 192.168.1.2/24 • Routing Aggregation 192.168.1.10/24 0 192.168.1.45/24 192.168.1.89/24 1 192.168.2.3/24 192.168.2.8/24 192.168.2.10/24

Two-level Routing Table Addressing • Using 10.0.0.0/8 private IP address • Pod Switch: 10. pod. Switch.1. pod range is [0, k-1](left to right) switch range is [0, k-1] (left to right, bottom to top) • Core Switch: 10. k. i . j (i,j) is the point in (k/2)*(k/2) grid • Host: 10.pod. Switch.ID ID range is [2, k/2+1] (left to right)

10.0.0.1 10.0.1.1 10.0.2.1 10.4.1.1 Two-level Routing Table 10.4.1.2 10.4.2.1 10.4.2.2 10.2.0.3 10.2.0.2

Two-level Routing Table • Two-level Routing Table Structure • Two-level Routing Table implementation TCAM=Ternary Content-Addressable Memory Parallel searching Priority encoding

Two-level Routing Table---example • example

Two-Level Routing Table • Avoid Packet Reordering • traffic diffusion occurs in the first half of a packet journey • Centralized Protocol to Initialize the Routing Table

Flow Classification (Dynamic) • Soft State (Compatible with Two-Level Routing Table) • A flow=packet with the same source and destination IP address • Avoid Reordering of Flow • Balancing • Assignment and Updating

Flow Classification—Flow Assignment Hash(Src,Des) Have seen this hash value? N Record new flow record f Y Assign f to least-loaded port x Lookup previously assign port x Send packet on port x Send packet on port x

Flow Classification—Update

Flow Scheduling • distribution of transfer times and burst lengths of Internet traffic is long-tailed • Large flow dominating • Large flow should be specially handled • Path-level scheduling – will be discussed next class in the paper Hedera

Power and Heat

Experiment Description—hierarchical tree,click • four machines running four hosts each, and four machines each running four pod switches with one additional uplink • The four pod switches are connected to a 4-port core switch running on a dedicated machine. • 3.6:1 oversubscription on the uplinks from the pod switches to the core switch • Each host generates a constant 96Mbit/s of outgoing traffic

Result

Conclusion • Bandwidth is the scalability bottleneck in large scale clusters • Existing solutions are expensive and limit cluster size • Fat-tree topology with scalable routing and backward compatibility with TCP/IP and Ethernet

CMPE 252A : Computer Networks