1 / 123

Building Gigabit-rate Routers with the NetFPGA: NICTA Tutorial at UNSW

Building Gigabit-rate Routers with the NetFPGA: NICTA Tutorial at UNSW. Presented by: John W. Lockwood, Jad Naous, Glen Gibb (Stanford University) Hosted by: Lavy Libman (NICTA) and Philip Allen (UNSW) February 6, 2008: 9am-5pm Lab 343A, Electrical Engineering Building (G17)

wynter-barton
Download Presentation

Building Gigabit-rate Routers with the NetFPGA: NICTA Tutorial at UNSW

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building Gigabit-rate Routerswith the NetFPGA: NICTA Tutorial at UNSW Presented by: John W. Lockwood, Jad Naous, Glen Gibb (Stanford University) Hosted by: Lavy Libman (NICTA) and Philip Allen (UNSW) February 6, 2008: 9am-5pm Lab 343A, Electrical Engineering Building (G17) Kensington Campus, University of New South Wales Sydney, Australia http://NetFPGA.org

  2. What is the NetFPGA? PC with NetFPGA 1GE FPGA 1GE 1GE Memory 1GE NetFPGA Board NetworkingSoftware running on a standard PC CPU Memory PCI A hardware accelerator built with Field Programmable Gate Arraydriving Gigabit network links

  3. Introduction Who uses the NetFPGA • Teachers • Students • Researchers How they use the NetFPGA • To run the Router Kit • To build modular reference designs • IPv4 router • 4-port NIC • Ethernet switch, … • To create new systems

  4. Running the Router KitUser-space development, 4x1GE line-rate forwarding OSPF BGP My Protocol user kernel Routing Table “Mirror” 1GE FPGA Fwding Table Packet Buffer 1GE 1GE 1GE 1GE IPv4 Router 1GE Memory 1GE 1GE Usage #1 CPU Memory PCI

  5. Building Modular Router Modules Verilog EDA Tools (Xilinx, Mentor, etc.) NetFPGA Driver • Design • Simulate • Synthesize • Download 1GE L3 Parse L2 Parse In Q Mgmt 1GE 1GE IP Lookup Out Q Mgmt 1GE Verilog modules interconnected by FIFO interfaces Usage #2 PW-OSPF CPU Memory Java GUI Front Panel (Extensible) PCI 1GE FPGA 1GE 1GE My Block Memory 1GE

  6. Creating new systems Verilog EDA Tools (Xilinx, Mentor, etc.) • Design • Simulate • Synthesize • Download NetFPGA Driver 1GE My Design (1GE MAC is soft/replaceable) 1GE 1GE 1GE Usage #3 CPU Memory PCI 1GE FPGA 1GE 1GE Memory 1GE

  7. Tutorial Outline • Background • Basics of an IP Router • The NetFPGA Platform • The Stanford Base Reference Router • Demo1 : Reference Router running on the NetFPGA • Inside the NetFPGA hardware • Breakneck introduction to Verilog • Exercise 1: Build your own Reference Router • The Enhanced Reference Router • Motivation: Understanding buffer size requirements in a router • Demo 2: Observing and controlling the queue size • Using NetFPGA for research and teaching • Exercise 2: Enhancing the Reference Router • The Life of a Packet Through the NetFPGA

  8. Basic Operation of an IP Router D R3 R1 R4 D A B E R2 C R5 Destination Next Hop F D R3 E R3 F R5

  9. What does a router do? R3 R1 R4 D A 1 4 16 32 D Ver HLen T.Service Total Packet Length Fragment ID Flags Fragment Offset B E TTL Protocol Header Checksum 20 bytes Source Address R2 C R5 Destination Address Destination Next Hop F D R3 Options (if any) E R3 Data F R5

  10. What does a router do? R3 R1 R4 D A B E R2 C R5 F

  11. Basic Components of an IP Router Software Hardware Management & CLI Routing Protocols Control Plane Routing Table Datapath per-packet processing Forwarding Table Switching

  12. Per-packet processing in an IP Router 1. Accept packet arriving on an incoming link. 2. Lookup packet destination address in the forwarding table, to identify outgoing port(s). 3. Manipulate IP header: e.g., decrement TTL, update header checksum. 5. Buffer packet in the output queue. 6. Transmit packet onto outgoing link.

  13. Generic Datapath Architecture Data Hdr Data Hdr IP Address Next Hop Forwarding Table Buffer Memory Header Processing Lookup IP Address Update Header Queue Packet

  14. CIDR and Longest Prefix Matches 142.12/19 • The IP address space is broken into line segments. • Each line segment is described by a prefix. • A prefix is of the form x/y where x indicates the prefix of all addresses in the line segment, and y indicates the length of the segment. • e.g. The prefix 128.9/16 represents the line segment containing addresses in the range: 128.9.0.0 … 128.9.255.255. 128.9.0.0 65/8 128.9/16 0 232-1 216 128.9.16.14

  15. Classless Interdomain Routing (CIDR) 128.9.19/24 128.9.25/24 128.9.16/20 128.9.176/20 Most specific route = “longest matching prefix” 128.9/16 0 232-1 128.9.16.14

  16. Techniques for LPM in hardware • Linear search • Direct lookup • Currently requires too much memory • Updating a prefix leads to many changes • Tries • Deterministic lookup time • Easily pipelined • But requires multiple memories/references • TCAM (Ternary CAM) • Simple and widely used • But low-density, high-power • Gradually being replaced by new algorithms

  17. An IP Router on NetFPGA Software Hardware Management & CLI Linux user-level processes Routing Protocols Exception Processing Routing Table Verilog on NetFPGA PCI board Forwarding Table Switching

  18. NetFPGA Router Function 4 Gigabit Ethernet ports Fully programmable FPGA hardware Low cost Open-source FPGA hardware Verilog base design Open-souce Software Drivers in C and C++

  19. NetFPGA Platform Major Components • Interfaces • 4 Gigabit Ethernet Ports • PCI Host Interface • Memories • 36Mbits Static RAM • 512Mbits DDR2 Dynamic RAM • FPGA Resources • Block RAMs • Configurable Logic Block (CLBs) • Memory Mapped Registers

  20. NetFPGA System User Space Linux Kernel CAD Tools Monitor Software Web & VideoServer Browser & Video Client Packet Forwarding Table PCI-e PCI VI VI VI VI NIC NetFPGA RouterHardware GE GE GE GE GE GE (nf2c0 .. 3) (eth1 .. 2)

  21. NetFPGA Hardware

  22. NetFPGA System Implementation • NetFPGA Blocks • Virtex-2 Pro FPGA • 4.5MB ZBT SRAM • 64MB DDR2 DRAM • PCI Host Interface • 4 Gigabit Ethernet ports • Intranet Test Ports • Dual or Quad Gigabit Etherents on PCI-e • Internet • Gigabit Ethernet on Motherboard • Processor • Dual-Core CPU • Operating System • Linux CentOS 4.4

  23. NetFPGA Lab Setup CPU x2 Dual NIC Client Eth2 : Server PCI-e GE (eth1 .. 2) Eth1 : Local host GE Server Net-FPGA Nf2c3 : Adj. Server GE PCI NetFPGA Control SW Nf2c2 : Local Host Internet Router Hardware GE Nf2c1 : Adjacent GE Nf2c0 : Adjacent GE CAD Tools

  24. NetFPGA Hardware Set for Demo #1 CPU x2 NIC Video Server PCI-e PCI-e GE GE Net-FPGA GE PCI Internet Router Hardware GE GE Server deliversstreaming HD video through a chain of NetFPGA Routers GE Net-FPGA GE Internet Router Hardware GE GE GE … CPU x2 NIC PCI-e GE GE Net-FPGA GE Video Display PCI Internet Router Hardware GE GE CAD Tools GE

  25. Tutorial Outline Background Basics of an IP Router The NetFPGA Platform The Stanford Base Reference Router Demo1 : Reference Router running on the NetFPGA Inside the NetFPGA hardware Breakneck introduction to Verilog Exercise 1: Build your own Reference Router The Enhanced Reference Router Motivation: Understanding buffer size requirements in a router Demo 2: Observing and controlling the queue size Using NetFPGA for research and teaching Exercise 2: Enhancing the Reference Router The Life of a Packet Through the NetFPGA

  26. Topology of NetFPGA Routers Demo 1 Video Server HD Display

  27. Setup for the Reference Router Each NetFPGA card has four ports Port 2 connected to Client / Server Ports 0 and 3 connected to adjacent NetFPGA cards NetFPGA NetFPGA NetFPGA Demo 1 Video Server Video Client

  28. Demo 1: Logical Topology Demo 1 .1.1 .4.1 .7.1 .10.1 .13.1 .16.1 .1.2 .4.2 .7.2 .10.2 .13.2 .16.2 .3.1 .6.2 .9.2 .12.2 .15.2 .17.1 .3.2 .2.1 .6.1 .9.1 .12.1 .15.1 .30.2 .5.1 .8.1 .11.1 .14.1 .18.1 .30.1 .26.1 .23.1 .18.2 .27.2 .24.2 .21.2 .20.1 .29.1 .24.1 .21.1 .27.1 .28.2 .25.2 .22.2 .19.2 .28.1 .25.1 .22.1 .19.1 Video Server Video Client Shortest Path

  29. Working IP Router Objectives Become familiar with Stanford Reference Router Observe PW-OSPF re-routing traffic around a failure Demo 1

  30. Streaming Video through the NetFPGA Demo 1 • Video server • Source files /var/www/html/video • Network URL : http://192.168.Net.Host/Video • Video client • Windows Media Player • Linux mplayer • Video traffic • MPEG2 HDTV (35 Mbps) • MPEG2 TV (9 Mbps) • DVI (3 Mbps) • WMF (1.7 Mbps)

  31. Step 1 – Observe the Routing Tables The router is already configured and running on your machines The routing table has converged to the routing decisions with minimum number of hops Next, break a link … Demo 1

  32. Step 2 - Dynamic Re-routing Break the link between video server and video client Routers re-route traffic around the broken link and video continues playing Demo 1 .1.1 .4.1 .7.1 .10.1 .13.1 .16.1 .1.2 .4.2 .7.2 .10.2 .13.2 .16.2 .3.1 .6.2 .9.2 .12.2 .15.2 .17.1 .2.1 .3.2 .15.1 .6.1 .9.1 .12.1 .30.2 .5.1 .8.1 .11.1 .14.1 .18.1 .30.1 .26.1 .23.1 .18.2 .21.2 .27.2 .24.2 .20.1 .29.1 .24.1 .21.1 .27.1 .28.2 .25.2 .22.2 .19.2 .28.1 .25.1 .22.1 .19.1

  33. Tutorial Outline Background Basics of an IP Router The NetFPGA Platform The Stanford Base Reference Router Demo1 : Reference Router running on the NetFPGA Inside the NetFPGA hardware Breakneck introduction to Verilog Exercise 1: Build your own Reference Router The Enhanced Reference Router Motivation: Understanding buffer size requirements in a router Demo 2: Observing and controlling the queue size Using NetFPGA for research and teaching Exercise 2: Enhancing the Reference Router The Life of a Packet Through the NetFPGA

  34. Integrated Circuit Technology Full-custom Design Complementary Metal Oxide Semiconductor (CMOS) Semi-custom ASIC Design Gate array Standard cell Programmable Logic Device Programmable Array Logic Field Programmable Gate Arrays Processors

  35. Look-Up Tables Combinatorial logic is stored in Look-Up Tables (LUTs) Also called Function Generators (FGs) Capacity is limited only bynumber of inputs, not complexity Delay through the LUT is constant Combinatorial Logic A B Z C D Diagram From: Xilinx, Inc

  36. Each slice has four outputs Two registered outputs, two non-registered outputs Two BUFTs associated with each CLB, accessible by all 16 CLB outputs Carry logic run vertically Signals run upwards Two independent carry chains per CLB Xilinx CLB Structure LUT LUT PRE D Q CE CLR PRE D Carry Carry Q CE CLR Slice 0 Diagram From: Xilinx, Inc (Courtesy Jeff Weintraub)

  37. Field Programmable Gate Arrays CLB Primitive element of FPGA Routing Module Global routing Local interconnect Macro Blocks Block Memories Microprocessor I/O Block

  38. NetFPGA Block Diagram

  39. Details of NetFPGA Fits into Standard PCI slot Standard Bus : 32 bits, 33 MHz Provides Interfaces for processing network packets 4 Gigabit Ethernet Ports Allows hardware-accelerated processing Implemented with Field Programmable Gate Array (FPGA) Logic

  40. Tutorial Outline Background Basics of an IP Router The NetFPGA Platform The Stanford Base Reference Router Demo1 : Reference Router running on the NetFPGA Inside the NetFPGA hardware Breakneck introduction to Verilog Exercise 1: Build your own Reference Router The Enhanced Reference Router Motivation: Understanding buffer size requirements in a router Demo 2: Observing and controlling the queue size Using NetFPGA for research and teaching Exercise 2: Enhancing the Reference Router The Life of a Packet Through the NetFPGA

  41. Hardware Description Languages • Concurrent • By Default, Verilog statements evaluated concurrently • Express fine grain parallelism • Allows gate-level parallelism • Provides Precise Description • Eliminates ambiguity about operation • Synthesizable • Generates hardware from description

  42. Verilog Data Types reg [7:0] A; // 8-bit register, MSB to LSB // (Preferred bit order for NetFPGA) reg [0:15] B; // 16-bit register, LSB to MSB B = {A[7:0],A[0:7]}; // Assignment of bits reg [31:0] Mem [0:1023]; // 1K Word Memory integer Count; // simple signed 32-bit integer integer K[1:64]; // an array of 64 integers time Start, Stop; // Two 64-bit time variables From: CSCI 320 Computer Architecture Handbook on Verilog HDL, by Dr. Daniel C. Hyde : http://eesun.free.fr/DOC/VERILOG/verilog-manual.html

  43. Signal Multiplexers From:http://eesun.free.fr/DOC/VERILOG/synvlg.html • Two input multiplexer (using if / else) • reg y; • always @*   if (select)      y = a;   else      y = b; Two input multiplexer (using ternary operator ?:) wire t = (select ? a : b);

  44. Larger Multiplexers Three input multiplexer reg s; always @*   begin   case (select2)       2'b00: s = a;      2'b01: s = b;      default: s = c;    endcase   end

  45. Synchronous Storage Elements Din D Q Dout Clock 1 Clock Transition Clock 0 time t=0 t=1 t=2 A B C • Values change at times governed by clock • Clock • Input to circuit • Clock Event • Example: Rising edge Din t=0 • Flip/Flop • Transfers Value From Din to Dout on Clock event Clock Transition Dout A B S0 t=0

  46. Finite State Machines

  47. Synthesizable Verilog : Delay Flip/Flops From:http://eesun.free.fr/DOC/VERILOG/synvlg.html • D-type flip flop • reg q; • always @ (posedge clk)  q <= d; • D type flip flop with data enable • reg q; • always @ (posedge clk)   if (enable)     q <= d;

  48. Tutorial Outline Background Basics of an IP Router The NetFPGA Platform The Stanford Base Reference Router Demo1 : Reference Router running on the NetFPGA Inside the NetFPGA hardware Breakneck introduction to Verilog Exercise 1: Build your own Reference Router The Enhanced Reference Router Motivation: Understanding buffer size requirements in a router Demo 2: Observing and controlling the queue size Using NetFPGA for research and teaching Exercise 2: Enhancing the Reference Router The Life of a Packet Through the NetFPGA

  49. Reference Router Pipeline Exercise 1 MAC RxQ CPU RxQ MAC RxQ CPU RxQ MAC RxQ CPU RxQ MAC RxQ CPU RxQ • Five stages • Input • Input Arbitration • Routing Decision and packet modification • Output Queuing • Output • Packet-based module interface • Pluggable design Input Arbiter Output Port Lookup Output Queues MAC TxQ CPU TxQ MAC TxQ CPU TxQ MAC TxQ CPU TxQ MAC TxQ CPU TxQ

  50. Make your own router Exercise 1 Objectives: • Learn how to build hardware • Run the software • Explore router architecture Execution • Start synthesis • Rerun the GUI with the new hardware • Test connectivity and statistics with pings • Explore pipeline in the details page • Explore detailed statistics in the details page

More Related