1 / 43

IXP: The Bump in the Wire

Explore how Intel's Internet Exchange Processor (IXP) revolutionizes network processing, offloads host resources, and boosts network performance through specialized hardware. Discover the evolution of network processors, explosion of commercial products, features of Intel IXP1200 and IXP2400, and the architecture and memory hierarchy of IXP1200. Gain insights into traditional vs network processor systems, the main ideas behind network processors, and the impact on high-speed networks.

majore
Download Presentation

IXP: The Bump in the Wire

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. INF5062:Programming Asymmetric Multi-Core Processors IXP:The Bump in the Wire 7 January 2020

  2. Background and Motivation:Network traffic increase

  3. Software-Based Network System • Uses conventional, shared hardware (e.g., a PC) • Software • runs the entire system • allocates memory • controls I/O devices • performs all protocol processing • First generation network systems:

  4. user space kernel space Review of General Data Path on Conventional Computer Hardware Architectures sending: receiving: forwarding: application application application Checksumming Fragmentation Interrupts Copying communication system communication system communication system

  5. Question: • Which is growing faster? • network bandwidth • processing power • Note: if networkbandwidth is growing faster • CPU may be the bottleneck • need special-purpose hardware • Note: if processingpower is growing faster • no problems with processing • network/busses will be bottlenecks

  6. Growth Of Technologies Engineering rule: 1GHz general purpose CPU = 1Gbps network data rate Mbps Thus, software running on a general-purpose processor is insufficient to handle high-speed networks because the aggregate packet rate exceeds the capabilities of the CPU year

  7. Network Processors: The Idea in a Nutshell • Many designs through many generations (varying amount of HW & SW) • Include support for protocol processing and I/O on one chip • General-purpose processor(s) for control tasks • Special-purpose processor(s) for packet processing and table lookup • Include functional units for tasks such as checksum computation, hashing, … • Call the result a network processor

  8. Network Processors: Main Idea Traditional system: - slow - resource demanding - shared with other operations Network processors: - a computer within the computer - special, programmable hardware - offloads host resources

  9. Explosion of Commercial Products • 1990  2000: network processors transformed from interesting curiosity to mainstream product • reduction in both overall costs and time to market • 2002: over 30 vendors with a vide range of architectures • e.g., • Multi-Chip Pipeline (Agere) • Augmented RISC Processor (Alchemy) • Embedded Processor Plus Coprocessors (Applied Micro Circuit Corporation) • Pipeline of Homogeneous Processors (Cisco) • Pipeline of Heterogeneous Processors (EZchip) • Configurable Instruction Set Processors (Cognigine) • Extensive And Diverse Processors (IBM) • Flexible RISC Plus Coprocessors (Motorola) • Internet Exchange Processor (Intel) • …

  10. Intel IXP1200 / 2400:A Short Overview

  11. IXA is a broad term to describe the Intel network architecture (HW & SW, control- & data plane) IXP: Internet Exchange Processor processor that implements IXA IXP1200 is the first IXP chip (4 versions) IXP2xxx has now replaced the first version IXP1200 basic features 1 embedded 232 MHz StrongARM 6 packet 232 MHz µengines onboard memory 4 x 100 Mbps Ethernet ports multiple, independent busses low-speed serial interface interfaces for external memory and I/O busses … IXP2400 basic features 1 embedded 600 MHz XScale 8 packet 600 MHz µengines 3 x 1 Gbps Ethernet ports … IXA: Internet Exchange Architecture

  12. IXP1200 Architecture PCI bus: - allow IXP to connect to I/O devices - enable use of host CPU - rate 2.2 Gbps SRAM bus: - shared bus (several external units) - usually control rather than data - rate 3.71 Gbps Serial line: - connects to the RISC - intended for control and management - rate 38 Kbps SDRAM bus: - provide access to external SDRAM memory used to store packets - can also pass addresses, control/store operations, etc. - rate 7.42 Gbps IX (Intel eXchange) bus: • enable higher rates compared to PCI • form fast path (IXP and high-speed interfaces) - interface to other IXP cards - 4.4 Gbps

  13. IXP1200 Architecture RISC processor: - StrongARM running Linux - control, higher layer protocols and exceptions - 232 MHz Access units: - coordinate access to external units Scratchpad: - on-chip memory - used for IPC and synchronization Microengines: - low-level devices with limited set of instructions - transfers between memory devices - packet processing - 232 MHz

  14. IXP1200 Processor Hierarchy General-Purpose Processor: - used for control and management - running general applications RISC processor: - chip configuration interface (serial line) - control, higher layer protocols and exceptions I/O processors (microengines): - transfers between memory devices - packet processing Coprocessors: - real-time clock and timers • IX bus controller • hashing unit • ... Physical interface processors: - implement layer 1 & 2 processing

  15. IXP1200 Memory Hierarchy

  16. IXP1200 Memory Hierarchy Different memory types… • …are organized into different addressable data units (words or longwords) • …have different access times • …connected to different busses Therefore, to achieve optimal performance, programmers must understand the organization and allocate items from the appropriate type

  17. microengine 6 microengine 3 microengine 4 microengine 5 microengine 1 microengine 2 multiple independent internal buses IXP1200  IXP2400 PCI bus IXP1200 SRAM bus SRAM access PCI access Embedded RISK CPU (StrongARM) SRAM FLASH SCRATCH memory MEMORYMAPPEDI/O SDRAM access IX access DRAM DRAM bus IX bus

  18. microengine 3 microengine 4 microengine 5 microengine 1 microengine 2 multiple independent internal buses IXP2400 Architecture Coprocessors • hash unit • 4 timers • general purpose I/O pins • external JTAG connections (in-circuit tests) • several bulk cyphers (IXP2850 only) • checksum (IXP2850 only) • … PCI bus IXP2400 RISC processor: - StrongArm  XScale - 233 MHz  600 MHz SRAM bus SRAM access PCI access Embedded RISK CPU (XScale) SRAM coprocessor SCRATCH memory FLASH slowport access Slowport • shared inteface to external units • used for FlashRom during bootstrap Media Switch Fabric • forms fast path for transfers • interconnect for several IXP2xxx Microengines - 6  8 - 233 MHz  600 MHz … SDRAM access MSFaccess DRAM microengine 8 DRAM bus receive bus transmit bus Receive/transmit buses • shared bus  separate busses

  19. IXP2400 Architecture • Memory • generally more of everything • generally larger gap between CPUs and memory access in terms of cycles • local memory on each microengine • saving temporary results • private per packet processor • small (2560 bytes) • low latency (one cycle) • accessed through special registers

  20. microengine 8 microengine 3 microengine 4 microengine 5 microengine 1 microengine 2 multiple independent internal buses IXP2400 Basic Packet Processing PCI bus SRAM bus SRAM access PCI access Embedded RISK CPU (XScale) SRAM coprocessor SCRATCH memory FLASH slowport access … SDRAM access MSFaccess DRAM DRAM bus receive bus transmit bus

  21. Using IXP2400

  22. Intel IXP2400 Hardware Reference Manual

  23. core component process microblock Programming Model head tail logical mapping linked list metadata queue data buffers Scratch rings XScale microengines RX microblock TX microblock output ports input ports

  24. core component process microblock Programming Model Threads XScale microengines RX microblock TX microblock output ports input ports Hardware contexts

  25. Framework • uclo • Microengine loader • Necessary to load your microengine code into the microengines at runtime • hal • Hardware abstraction layer • Mapping of physical memory into XScale processes’ virtual address space • Functions starting with hal • ossl • Operating system service layer • Limited abstraction from hardware specifics • Functions starting with ix_ • rm • Resource manager • Layered on top of uclo and ossl • Memory and resource management • all memory types and their features • IPC, counters, hash • Functions starting with ix_

  26. Bump in the Wire

  27. Internet Bump in the Wire Count web packets, count ICMP packets

  28. Packet Headers and Encapsulation

  29. Ethernet 48 bit address configured to an interface on the NIC on the receiver 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Frame type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 48 bit address configured to an interface on the NIC on the sender describes content of ethernet frame, e.g., 0x0800 indicates an IP datagram, 0x0806 indicates an ARP packet

  30. Internet Protocol version 4 (IPv4) indication of the abstract parameters of thequality of service desired – somehow treat high precedence traffic as more important – tradeoff between low-delay, high-reliability, and high-throughput – NOT used, bits now reused for differential services code point indicates the format of the internet header, i.e., version 4 length of the internet header in 32 bit words, and thus points to the beginning of the data (minimum value of 5) datagram length (octets) includingheader and data - allows the lengthof 65,535 octets first zero, fragments allowed and last fragment 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of Service| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live | Protocol | Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Options | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ identifying value to aid assembly of fragments indicate where this fragment belongs in datagram disable a packet to circulate forever,decrease value by at least 1 in each node – discarded if 0 checksum on the header only – TCP, UDP over payload. Since some header fields change(TTL), this is recomputed and verified at each point indicates used transport layer protocol 32-bit address fields. May be configured differently from small to large networks options may extend the header – indicated by IHL. If the options do not end on a 32-bit boundary, the remaining fields are padded in the padding field (0’s)

  31. Internet Control Message Protocol (ICMPv4) Type of the control msg, including echo request (8) and echo reply (0) Checksum for the ICMP header only Refinement 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Code | header checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ type-specific arbitrary length data ICMP Echo Request 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 8 | 0 | header checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | identifier | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Optional sequence number, chosen by sender, echoed by receiver Optional identifier, chosen by sender, echoed by receiver

  32. UDP port to identify the sending application port to identify receiving application 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Length | Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ specifies the total length of the UDP datagram in octets contains a 1’s complement checksum over UDP packet and an IP pseudo header with source and destination address

  33. TCP code bits: urgent, ack, push, reset, syn, fin sequence number for data in payload acknowledgementfor data received port to identify the sending application port to identify receiving application 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Acknowledgment Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Header| |U|A|P|R|S|F| | | length| Reserved |R|C|S|S|Y|I| Window | | | |G|K|H|T|N|N| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum | Urgent Pointer | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Options | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ receiver’s buffer size foradditional data header length in 32 bit units pointer to urgent data in segment contains a 1’s complement checksum over UDP packet and an IP pseudo header with source and destination address options may extend the header. If the options do not end on a 32-bit boundary, the remaining fields are padded in the padding field (0’s)

  34. Encapsulation

  35. Identifying Web Packets These are the header fields you need for the web bumper: • Ethernet type 0x800 • IP type 6 • TCP port 80

  36. Lab Setup

  37. Internet Lab Setup IXP lab switch switch ssh connection … Student lab switch

  38. IXP lab switch switch … Lab Setup - Addresses

  39. IXP lab switch switch … Lab Setup - Addresses IXP lab 192.168.2.50 129.240.66.50 switch switch 192.168.2.51 129.240.66.51 192.168.2.11 192.168.2.52 129.240.66.52 … … … 192.168.2.55 129.240.66.55

  40. Lab Setup – Data Path SSH connection to IFI: 129.240.66.50 IXP lab PCI IO hub switch switch hub interface memory hub … 192.168.1.1 system bus 192.168.1.5 CPU RAM interface IXP2400 memory web bumper (counting web packets and forwarding all packets from one interface to another)

  41. Lab Setup – Data Path SSH connection to IFI: 129.240.66.50 IXP lab IO hub switch switch 192.168.2.50 192.168.2.11 memory hub … 192.168.1.1 192.168.1.5 CPU IXP2400 memory web bumper (counting web packets and forwarding all packets from one interface to another) SSH connection to IFI: 129.240.66.48

  42. IXP 2400 The Web Bumper the wwbump core components checks a packet forwarded by the wwbump microblock • count ping packet - add 1 to icmp counter • send back to wwbump microblock the wwbump microblock checks all packets from rx block: if it is a ping or web packet: • if web packet, add 1 to web counter and forward to tx block • if ping packet, forward to wwbump core component • if neither, forward to tx block The wwbump microblock forwards all packets from the wwbump core component to the tx block web bumper wwbump (core) XScale microengines input port output port rxmicroblock wwbump(microblock) tx microblock web bumper (counting web packets and forwarding all packets from one interface to another) rx block processing encompasses alloperations performed as packets arrive tx block processing encompasses all operations applied as packets depart

  43. Starting and Stopping • On the host machine • Location of the example: /root/ixa/wwpingbump • Rebooting the IXP card: make reset • Installing the example: make install • Telnet to the card: • telnet 192.168.1.5 • minicom  • On the card • To start the example: ./wwbump • To stop the example: CTRL-C • Let’s look at an example...

More Related