Intel® IXP2XXX Network Processor Architecture Overview

1. Intel® IXP2XXX Network Processor Architecture Overview John Morgan Infrastructure Processor Division September 2004

2. Agenda • IXP2400 External Features • IXP2800 External Features • Comparison of IXP2400 and IXP2800 • IXP2XXX Resource Overviews • MEv2 Overview • QDR SRAM Overview • DDR Overview • RDRAM Overview • PCI Overview • MSF Overview • Miscellaneous

3. IXP2400 External Features Host CPU (Optional) PCI 64-bit / 66 MHz External Interfaces • MSF Interface supports UTOPIA 1/2/3, SPI-3 (POS-PL3), and CSIX. • Four independent, configurable, 8-bit channels with the ability to aggregate channels for wider interfaces. • Media interface can support channelized media on RX and 32-bit connect to Switch Fabric over SPI-3 on TX (and vice versa) to support Switch Fabric option. • 2 Quad Data Rate SRAM channels. • A QDR SRAM channel can interface to Co-Processors. • 1 DDR SDRAM channel. • PCI 64/66 Host CPU interface. • Flash and PHY Mgmt interface. • Dedicated inter-IXP channel to communicate fabric flow control information from egress to ingress for dual chip solution. IXA SW QDR SRAM 1.6 GBs 64 M Byte Classification Accelerator IXP2400 (Ingress) DDR DRAM 2 GByte CoProc Bus Micro-Engine Clusters Customer ASICs IXP2400 (Egress) Flash Slow Port Utopia 1/2/3 or POS-PL2/3 Interface Flow Control Bus Utopia 1,2,3 SPI – 3 (POS-PL3) CSIX Switch Fabric Port Interface ATM / POS PHY or Ethernet MAC

4. Ingress Processor SAR’ing ClassificationMeteringPolicingInitial Congestion Management S D R A M Host CPU (IOP or iA) DDR SDRAM Packet Memory QDR SRAM Queues & Tables QDR QDR T C A M ClassificationAccelerator IXP2400 Ingress Processor IXF6048 Framer OC-48 OC48 SwitchFabricGasket OC48 OC48 IXP2400 Egress Processor 1x OC-48 or4x OC-12 T C A M Egress Processor ClassificationAccelerator Traffic Shaping Flexible Choices diff serve TM 4.0 … QDR SRAM Queues & Tables Q DR QDR DDR SDRAM Packet Memory IXP2400 Full-Duplex OC-48 System Implementation S D R A M

5. IXP2400 Chaining Glueless Interface between IXP2400 Devices using CSIX-L1 Control Plane Processor PCI 64/66 IXP2400 Processor IXP2400 Processor IXP2400 Processor 2.5Gbs SPI3 2.5Gbs CSIX-L1 2.5Gbs CSIX-L1 2.5 Gbs CSIX-L1 Q DR Q DR D R A M Q DR Q DR D R A M Q DR Q DR D R A M QDR SRAM Queues & Tables QDR SRAM Queues & Tables QDR SRAM Queues & Tables DDRPacket Memory DDRPacket Memory DDRPacket Memory

6. 72 IXP2400 MEv2 1 MEv2 2 DDRAM Rbuf 64 @ 128B S P I 3 or C S I X 32b MEv2 4 MEv2 3 Intel® XScale™ Core 32K IC 32K DC G A S K E T PCI (64b) 66 MHz Tbuf 64 @ 128B 32b 64b MEv2 5 MEv2 6 Hash 64/48/128 Scratch 16KB QDR SRAM 1 QDR SRAM 2 MEv2 8 MEv2 7 CSRs -Fast_wr -UART -Timers -GPIO -BootROM/Slow Port E/D Q E/D Q 18 18 18 18

7. IXP2400 Bandwidths • 600 MHz Operation 4.8+ GOPs • 2.5 Gb/s Full Duplex Media Interface • POS-PHY • Utopia • CSIX-L1 • 2.4 GBs DDR Memory Bandwidth at 300 MTs • 1.6 GBs QDR Memory Bandwidth with 200 MHz QDRII devices

8. IXP2400 Resources Summary • Half Duplex OC-48 / 2.5 Gb/sec Network Processor • (8) Multi-Threaded Microengines • Intel® XScale™ Core • Media / Switch Fabric Interface • PCI interface • 2 QDR SRAM interface controllers • 1 DDR SDRAM interface controller • 8 bit asynchronous port • Flash and CPU bus • Additional integrated feature • Hardware Hash Unit • 16 KByte Scratchpad Memory,Serial UART port • 8 general purpose I/O pins • Four 32-bit timers • JTAG Support

9. Agenda • IXP2400 External Features • IXP2800 External Features • Comparison of IXP2400 and IXP2800 • IXP2XXX Resource Overviews • MEv2 Overview • QDR SRAM Overview • DDR Overview • RDRAM Overview • PCI Overview • MSF Overview • Miscellaneous

10. Host CPU (Optional) PCI 64-bit / 66 MHz External Interfaces • Media Interface supports both SPI-4 and CSIX • 4 Quad Data Rate (QDR) SRAM channels • Each channel can interface to Co-processors • 3 RDRAM Channels • PCI 64/66 Host CPU interface • Flash and PHY Management interface • Dedicated inter-IXP channel to communicate fabric flow control information from egress to ingress for dual chip solution QDR SRAM 12.8 Gbps x 4 64 M Byte x 4 channels IXA SW Classification Accelerator IXP2800 (Ingress) RDR DRAM 50+Gbps 2 Gbyte total for 3 channels CoProc Bus Micro-Engine Clusters Customer ASICs IXP2800 (Egress) Flash Slow Port SPI-4 or CSIX-L1 Flow Control Bus SPI – 4, CSIX-L1 Switch Fabric Port Interface ATM / POS PHY or Ethernet MAC IXP2800 External Features

11. Ingress Processor SAR’ing ClassificationMeteringPolicingInitial Congestion Management Egress Processor Traffic Shaping Flexible Choices diff serve TM 4.1 … 10Gb/s SONET Line Card D R A M D R A M D R A M RDRPacket Memory Control Plane Processor QDR SRAM Queues & Tables Q D R Q D R Q DR Q DR PCI 64/66 IXP2800 Ingress Processor Fabric Interface Chip (FIC) IXF18101 CDR, DEMUX 15Gbs 10GbE OC-192c 10Gbs SPI I/F CSIX I/F Fabric Flow Ctl CDR, DEMUX 10Gbs 15Gbs IXP2800 Egress Processor 10 GbE WAN / PPP/ ATM/ OTN / SONET/ SDH QDR SRAM Queues & Tables Q DR Q DR Q DR Q DR D R A M D R A M D R A M RDR Packet Memory

12. IXP2800 System with SPI gasket D R A M D R A M D R A M Control Plane Processor QDR SRAM Queues & Tables RDRPacket Memory Q D R Q D R Q DR Q DR PCI 64/66 IXP2800 Ingress Processor 10Gbs 10Gbs Dual CSIX Utopia3 SPI4 2U3 x SPI 2U3 SPI4 2U3 x SPI gasket

13. IXP2800 Chaining • Glueless interface between IXP2800 devices using SPI-4.2 Control Plane Processor PCI 64/66 IXP2800 Processor IXP2800 Processor IXP2800 Processor 10Gbs SPI-4 10Gbs SPI-4 10Gbs SPI-4 10Gbs SPI-4 Q DR Q DR Q DR Q DR D R A M D R A M D R A M Q DR Q DR Q DR Q DR D R A M D R A M D R A M Q DR Q DR Q DR Q DR D R A M D R A M D R A M QDR SRAM Queues & Tables QDR SRAM Queues & Tables QDR SRAM Queues & Tables RDRPacket Memory RDRPacket Memory RDRPacket Memory

14. 18 18 18 IXP2800 Stripe RDRAM 1 RDRAM 2 RDRAM 3 MEv2 1 MEv2 2 MEv2 3 MEv2 4 Rbuf 64 @ 128B S P I 4 or C S I X 16b MEv2 8 MEv2 7 MEv2 6 MEv2 5 Intel® XScale™ Core 32K IC 32K DC G A S K E T PCI (64b) 66 MHz 64b Tbuf 64 @ 128B 16b MEv2 9 MEv2 10 MEv2 11 MEv2 12 Hash 48/64/128 Scratch 16KB QDR SRAM 1 QDR SRAM 2 QDR SRAM 3 QDR SRAM 4 MEv2 16 MEv2 15 MEv2 14 MEv2 13 CSRs -Fast_wr -UART -Timers -GPIO -BootROM/SlowPort E/D Q E/D Q E/D Q E/D Q 18 18 18 18 18 18 18 18

15. IXP2800 Bandwidths • 1.4 GHz Operation 20+ GOPs • 10Gbs Full Duplex Media Interface • SPI-4.2 • CSIX-L1 • 1.9 GB/s QDR SRAM Memory Bandwidth/Channel • 2.1 GB/s RDRAM Memory Bandwidth/Channel

16. IXP2800 Resources Summary • Half Duplex OC-192 / 10 Gb/sec Network Processor • (16) Multi-Threaded Microengines • Intel® XScale™ Core • Media / Switch Fabric Interface • PCI interface • 4 QDR SRAM Interface Controllers • 3 Rambus* DRAM Interface Controllers • 8 bit asynchronous port • Flash and CPU bus • Additional integrated features • Hardware Hash Unit for generating of 48-, 64-, or 128-bit adaptive polynomial hash keys • 16 KByte Scratchpad Memory • Serial UART port for debug • 8 general purpose I/O pins • Four 32-bit timers • JTAG Support

17. Agenda • IXP2400 External Features • IXP2800 External Features • Comparison of IXP2400 and IXP2800 • IXP2XXX Resource Overviews • MEv2 Overview • QDR SRAM Overview • DDR Overview • RDRAM Overview • PCI Overview • MSF Overview • Miscellaneous

18. IXP2800 and IXP2400 Comparison IXP2800 IXP2400 Frequency 1.4/1.0 GHz/ 650 MHz 600/400MHz DRAM Memory 3 channels RDRAM 800/1066MHz; Up to 2GB 1 channel DDR DRAM - 150MHz; Up to 2GB SRAM Memory 4 channels QDR (or co-processor) 2 channels QDR (or co-processor) Media Interface Separate 16 bit Tx & Rx configurable to SPI-4 P2 or CSIX_L1 Separate 32 bit Tx & Rx configurable to SPI-3, UTOPIA 3 or CSIX_L1 Number of MicroEngines 16 (MEv2) 8 (MEv2) Dual chip full duplex OC192 Dual chip full duplex OC48 Performance

19. Agenda • IXP2400 External Features • IXP2800 External Features • Comparison of IXP2400 and IXP2800 • IXP2XXX Resource Overviews • MEv2 Overview • QDR SRAM Overview • DDR Overview • RDRAM Overview • PCI Overview • MSF Overview • Miscellaneous

20. MicroEngine v2 D-Push Bus S-Push Bus From Next Neighbor Control Store 4K/8K Instructions Local Memory 640 words 128 GPR 128 GPR 128 Next Neighbor 128 D Xfer In 128 S Xfer In LM Addr 1 2 per CTX B_op A_op LM Addr 0 Prev B Prev A P-Random # B_Operand A_Operand CRC Unit Multiply TAGs 0-15 Lock 0-15 Status and LRU Logic (6-bit) 32-bit ExecutionData Path Find first bit CAM CRC remain Add, shift, logical Status Entry# OtherLocal CSRs ALU_Out To Next Neighbor Timers 128 D Xfer Out 128 S Xfer Out Timestamp D-Pull Bus S-Pull Bus

21. Microengine v2 Features – Part 1 • Clock Rates • IXP2400 – 600/400 MHz • IXP2800 - 1.4/1.0 GHz/ 650 MHz • Control Store • IXP2400 – 4K Instruction store • IXP2800 – 8K Instruction store • Configurable to 4 or 8 threads • Each thread has its own program counter, registers, signal and wakeup events • Generalized Thread Signaling (15 signals per thread) • Local Storage Options • 256 GPRs • 256 Transfer Registers • 128 Next Neighbor Registers • 640 - 32bit words of local memory

22. Microengine v2 Features – Part 2 • CAM (Content Addressable Memory) • Performs parallel lookup on 16 - 32bit entries • Reports a 9-bit lookup result • 4 State bits (software controlled, no impact to hardware) • Hit – entry number that hit; Miss – LRU entry • 4-bit index of Cam entry (Hit) or LRU (Miss) • Improves usage of multiple threads on same data • CRC hardware • IXP2400 - Provides CRC_16, CRC_32 • IXP2800 - Provides CRC_16, CRC_32, iSCSI, CRC_10 and CRC_5 • Accelerates CRC computation for ATM AAL/SAR, ATM OAM and Storage applications • Multiply hardware • Supports 8x24, 16x16 and 32x32 • Accelerates metering in QoS algorithms • DiffServ, MPLS • Pseudo Random Number generation • Accelerates RED, WRED algorithms • 64-bit Time-stamp and 16-bit Profile count

23. Intel® XScale™ Core Overview • High-performance, Low-power, 32-bit Embedded RISC processor • Clock rate • IXP2400 600 MHz • IXP2800 700/500/325 MHz • 32 Kbyte instruction cache • 32 Kbyte data cache • 2 Kbyte mini-data cache • Write buffer • Memory management unit

24. QDR SRAM Overview • Controller Configuration • IXP2400 - 2 channels • IXP2800 - 4 channels • Optional parity (support for x16 or x18 parts) • Address up to 64 Mbytes of SRAM per channel • Pin design supports up to 4 SRAM loads • Supports Burst of 2 QDR Devices • Supports byte parity bits [8], [17] for byte 0/1 • Parity can be enabled/disabled per channel in SRAM_control CSR

25. QDR SRAM Overview • Peak bandwidth of 1.6 GBytes/sec per channel • Using 200 MHz SRAMs • Specialized SRAM operations: • Atomic swap, bit set, bit clear, add, subtract • Hardware support for ring, queue and journal operations • 64 Q_Array registers per channel • Interface to QDR compatible TCAMs and CoProcessors • Network Processor Forum LA-1 Co-Processor Standard Compliant • “Clamshell” topology enables both Memory and Co-processor to share the same channel

26. IXP2400 DDR DRAM Overview • 1 64-bit (72-bit with ECC) SDRAM channel • DRAM sizes of 64Mb, 128Mb, 256Mb, 512Mb, or 1Gb • Max capacity is 2GB (using 1Gb parts) • Support x8 or x16 devices, DIMM or direct soldered • Support devices with 4 banks • Support 1 or 2 sided DIMM • Optional ECC • 200/300 MTS, 100MHz/150MHz respectively • Hardware Interleaving spreads contiguous addresses across multiple banks

27. IXP2800 RDRAM Overview • 3 Independent Rambus* DRAM Channels which operate concurrently • 1.6 GBytes/s (12.8Gbps) per channel at 800 MHz • Maximum total of 2 GBytes • 768 MBytes each if 3 channels are populated • 1 GBytes each if only 2 channels are populated • 2 GBytes if only 1 channel is populated • Supports 64Mb, 128Mb, 256Mb, 512Mb and 1 Gb devices • Supports RDRAMS with 1x16, 2x16 dependent and 4 independent Banks • Optional ECC and Parity Support • Interleaving implemented in HW provides balanced access across all channels • Interleave size is 128 bytes

28. PCI Interface Overview • PCI 2.2 compliant • PCI Bus Target • SRAM • DRAM • Control and Status Registers • PCI Bus Master to other devices • DMA channels • IXP2400 – 3 Channels • IXP2800 – 2 Channels • Doorbell and Mailbox Registers • Loads: • 4 loads at 66MHz • 8 loads at 33MHz

29. IXP2400 Media Switch Fabric Interface • Protocols • POS-PHY Levels 2 and 3 • Utopia Levels 1, 2 and 3 • CSIX-L1 for Switch Fabric Interface • LVTTL IO (3.3V) • 32-bit receive, 32-bit transmit • 25–133 MHz • 8KB receive buffer and 8KB transmit buffer

30. IXP2800 Media Switch Fabric Interface • Protocols • SPI-4 Phase 2 for Network Device • CSIX-L1 for Switch Fabric Interface • LVDS IO (IEEE 1596.3, ANSI/TIA/EIA-644) • 16-bit receive, 16-bit transmit • 311–500 MHz • 8KB receive buffer and 8KB transmit buffer

31. Miscellaneous • UART • Standard RS232 primarily for debugging • TIMER • 4 - 32 bit timers • Timer 4 can be used as Watchdog Timer • GPIO • 8 General Purpose IO pins • Can be used as interrupt source to XScale core or clock to timers • Interrupt Controller • Provides the ability to enable or mask interrupts from a number of chip wide sources like timers, PCI devices, DRAM ECC errors, etc. • Slow Port • Used for Flash ROM access and 8, 16, or 32-bit asynchronous device access • Allows XScale do read/ write data transfers to these slave devices

32. Backup

33. IXP2400 Target Application IXP2400 will provide IXP 1200 customers a performance upgrade for OC-12 applications and enable multi Gigabit Ethernet platforms up to OC-48 • WAN Edge/Access Aggregation • Includes IP Service Switches, Multiservice Switches, DSLAM, Cable Head End • Wireless Infrastructure • Layer 4-7 Switches • Includes Firewall, Server Offload, Content-Based Load Balancing

34. IXP2800 Target Application • Metropolitan Area Network (MAN) switches and routers • Internet core access switches and routers • Multi-service switches • 10 Gbs enterprise switches and routers supporting tomorrow’s data centers, • Storage area networks (SAN) • Content aware server off-load/web switches. • Security/VPN solutions • Wireless base stations • Digital Subscriber Line Access Multiplexers (DSLAMs).

35. Edge Multi-Service Switch - WAN/LAN Solutions IXP2400 IXP Oahu Quad Gig Phy Utopia3 Phy Interface OC-48c ATM SAR& TrafficManager Amazon IXF6048 OC-48c ATM & POS Framer Vallejo 4 x 1G Ethernet MAC 1 Gig LAN or Server Farm SPI-3 (Utopia3 Packet) SPI-3 (Utopia3 Packet) WAN Backbone (ATM, SONET) IXP2400 IXP Optical Ring 1 GigLAN Backbone or Server Farm OR CSIX Switch Fabric 80 Gig – 1+ TerabitSwitch Fabric

36. Edge Server Offload Host CPU (IOP or iA) PCI Bus IXP2400 IXP Oahu Quad Gig Phy Phy Interface Vallejo 4 x 1G Ethernet MAC CSIX Switch Fabric Server Farm CSIX Level 1 SPI-3 (Utopia3 Packet) 80 Gig – 1+ TerabitSwitch Fabric IXP2400 IXP 1 GigLAN Backbone or Server Farm

37. IXP2400 Media Configurations D D R D D R D D R Rx & Tx paths each have 2 separate clock domains for asynchronous traffic T C A M T C A M T C A M Q D R Q D R Q D R Q D R Q D R Q D R IXP2400 Xscale Xscale IXP2400 IXP2400 Xscale Rx Tx Rx Tx Rx Tx 32bit 32bit 32bit 16bit 8bit 32bit D D R T C A M Q D R Q D R Utopia 1/2/3 Or SPI-3 (POS-PHY 2/3) Or CSIX_L1B Utopia 1/2/3 Or SPI-3 (POS-PHY 2/3) IXP2400 Xscale Rx Tx Each Rx & Tx path may be configured to be single 32bit, quad 8bit, dual 16bit or combination of 8 & 16bit wide buses 16bit 8bit 16bit

38. Ingress Processor SAR’ing ClassificationMeteringPolicingInitial Congestion Management Egress Processor Traffic Shaping Flexible Choices diff serve TM 4.1 … 10Port 1Gb/s Ethernet Line Card D R A M D R A M D R A M RDRPacket Memory Control Plane Processor QDR SRAM Queues & Tables Q D R Q D R Q DR Q DR PCI 64/66 IXP2800 Ingress Processor Fabric Interface Chip (FIC) Ben Nevis 15Gbs 10x1GbE 10Gbs SPI I/F CSIX I/F Fabric Flow Ctl 10Gbs 15Gbs IXP2800 Egress Processor 10 x 1 GbE LAN QDR SRAM Queues & Tables Q DR Q DR Q DR Q DR D R A M D R A M D R A M RDR Packet Memory

39. 10Gb/s to Infiniband D R A M D R A M D R A M RDRPacket Memory Control Plane Processor QDR SRAM Queues & Tables Q D R Q D R Q DR Q DR PCI 64/66 IXP2800 Ingress Processor CalypsoBen NevisLoch Lomond InfinibandFabric 2.5Gbps 10GbE 10x1Gb OC-192c 15Gbs 10Gbs SPI I/F 2.5Gbps CSIX I/F Fabric Flow Ctl 2.5Gbps 10Gbs 15Gbs 2.5Gbps IXP2800 Egress Processor QDR SRAM Queues & Tables Q DR Q DR Q DR Q DR D R A M D R A M D R A M RDR Packet Memory

40. 10Gbs Ethernet to SONET D R A M D R A M D R A M RDRPacket Memory Control Plane Processor QDR SRAM Queues & Tables Q D R Q D R Q DR Q DR PCI 64/66 IXP2800 Ingress Processor Loch LomondBen Nevis Calypso Serveror DiskFarms Metro Or WAN 10GbE 10x1Gb 10Gbs 10Gbs SPI I/F SPI I/F OC-1924xOC48 Flow Ctl 10Gbs 10Gbs IXP2800 Egress Processor QDR SRAM Queues & Tables Q DR Q DR Q DR Q DR D R A M D R A M D R A M RDR Packet Memory

41. Media / Fabric Receive Logic: ThreadpushesID ontoFreelist AutoPushStatusto Thread Threadmovesdata 6 5 7 Rbuf 64/128 Elements 128/64B each Status Word Per element Rbuf Freelist Bit vector Thread Freelist 1 Receive State Machine CreateStatus Get Free element # 4 3 Assignthread # Media Switch Fabric Unit Idlebucket 2 Data Arrives SPI-4.2 Frame Discarded if idle packet Pkt ctrl Pkt payload a Pkt ctrl Cell payload Pkt ctrl Pkt payload b Media Device buffer Port A Port B ATM Cell packet