IXP2400 Protocol Offloading

1. IXP2400 Protocol Offloading Yan Luo (yluo@cs.ucr.edu) Chris Baron (cbaron@cs.ucr.edu

2. SDRAM (Packet buffer) SRAM (control structures) Bus Bus Output ports Input ports multi-threaded processing elements Co-processor Network Processor NP Overview

3. TCP/UDP/IP

4. VIA-UDP

5. IXP OS bypass User Kernel BigPhys SendBuf RcvBuf PCI IXP DRAM Downbuf/Upbuf

6. Protocol Overview • Customized protocol • Contains src/dest MAC, IP, and port • Unique packet ID and length of payload • Padding for ease of reading in blocks • Add own protocol header in the future

7. Socket Table • Socked id • Src/dstIP • Src/dstPort • User send/recv buff start • Phys send/recv buff start (for DMA) • Send/recv buffer size • Recv pointer (where data last written)

8. Theory of Operation (host) • Setup • Client application calls open() system call, kernel adds new entry to lookup table, returns unique socket descriptor • Client calls setup_connection(), a macro to ioctl calls, to add src/dst IP and requested ports for connection • Client calls ixpmalloc(), a macro to mmap() kernel memory into userspace, used for send and receive buffers

9. Theory of Operation (host) • Send • Macro to write() function for driver • If no space left on IXP, return error, else DMA/PIO to IXP, returning number of bytes copied • Currently uses PIO, for less latency • Receive • If no data available, return error • User must loop/poll for data

10. Theory of Operation (IXP)

11. (1) Host Send • User calls send data • Kernel assembles the packet header based on the socket table • Lookup the Downbuf tail/num in the bridgeCSR • Checks if downbuf is full • DMA/PIO data • Updates Downbuf tail in bridgeCSR

12. (2) Downbuf Manager • Polls bridgeCSR to get new tail of DOWNBUF • Examine protocol header • Fill in meta data (packet size, id etc) • Dispatch to worker ME • Examine the head of DOWNBUF, advance head if it is processed. tail head Just arrive Finished In-progress

13. (3) Downbuf Worker • Get a packet from Downbuf manager • Extract IP from packet • Lookup MAC and output port in IP forwarding table • Fill in MAC into packet • Fill in meta data (packet offset, len, output port) • Put the packet in TX queue

14. (4) XMIT ME • Get a packet from TX queue • Split packet into mpackets • Move mpackets to TBUF of MSF • Enable sending mpackets

15. (5) RCV ME • Waked up by MSF for new packets • Assemble and move mpackets to DRAM • Fill in meta data (packet offset, size, input port etc) • Put packet in RX queue

16. (6) Upbuf Manager • Get a packet from RX queue • Get meta data • Examine packet header (dstIP, destPort etc) • Look up socket table for up_rcv_ptr • Prepare DMA descriptor • Enable DMA • Update up_tail in bridge CSR

17. (7) Host RCV • User requests receive data • Kernel looks up the bridgeCSR from socket table • Returns the number of bytes arrived or error for no data available

18. Datapath Timing

19. Datapath Timing

20. Future additions • Use Interrupts • Use bridge registers more efficiently • Own protocol header

21. END