1 / 19

ONL NP Router Block Design Review: Lookup (Part of the PLC Block)

ONL NP Router Block Design Review: Lookup (Part of the PLC Block). John DeHart jdd@arl.wustl.edu http://www.arl.wustl.edu/projects/techX. Revision History. 4/12/07 (JDD): Started. ONL NP Router. SRAM Ring. xScale. xScale. Scratch Ring. TCAM. Assoc. Data ZBT-SRAM. SRAM. NN Ring.

rdaughtry
Download Presentation

ONL NP Router Block Design Review: Lookup (Part of the PLC Block)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ONL NP Router Block Design Review:Lookup (Part of the PLC Block) John DeHart jdd@arl.wustl.edu http://www.arl.wustl.edu/projects/techX

  2. Revision History • 4/12/07 (JDD): • Started

  3. ONL NP Router SRAM Ring xScale xScale Scratch Ring TCAM Assoc. Data ZBT-SRAM SRAM NN Ring NN 64KW Parse, Lookup, Copy (3 MEs) Rx (2 ME) Mux (1 ME) QM (1 ME) HdrFmt (1 ME) Tx (1 ME) NN Mostly Unchanged 64KW SRAM 64KW Each New NN NN NN NN Plugin3 Plugin4 Plugin0 Plugin1 Plugin2 SRAM xScale Needs A Lot Of Mod. Needs Some Mod. Tx, QM Parse Plugin XScale Stats (1 ME) FreeList Mgr (1 ME) QM Copy Plugins SRAM

  4. Contents • Overview • Design • Latency Analysis • Code Locations • Test Procedures • Implementation Status

  5. Overview • Initialization • Control Plane initializes TCAM and Route and Filter DBs • Runtime Updates • Control Plane updates to Route and Filter DBs • Design – in upcoming slides • Processing – in upcoming slides • Lookup will be written in C • There are many things about writing IXP code in “C” that I need to learn. Here are some of them: • Performing multiple memory operations in parallel and waiting on a set of signals (If needed for performance reasons) • Performing timestamp waits • Calling IDT microcode macros

  6. Lookup: Design -- Databases • Three Databases: • Route Lookup: • Unicast • Sorted by DAddr Prefix Length • Multicast • Exact match on DAddr and prefix of SAddr • Primary Filter • Filters should be sorted in the DB with higher priority filters first • Auxiliary Filter • Filters should be sorted in the DB with higher priority filters first • Priority between Primary Filter and Route Lookup • A priority will be stored with each Primary Filter • A priority will be assigned to RLs (all routes have same priority) • PF priority and RL priority compared after result is retrieved. • One of them will be selected based on this priority comparison. • Auxiliary Filters: • If matched, cause a copy of packet to be sent out according to the Aux Filter’s result.

  7. Lookup: Design -- Results • Use SRAM Bank 0 (2 MB per NPU) for Results • B0 Byte Address Range: 0x000000 – 0x1FFFFF • 21 bits • B0 Word Address Range: 0x000000 – 0x1FFFFC • 19 significant bits • 2 trailing 0’s • Store result in two parts: • 32-bit Associated Data SRAM result for Address of actual Result: • TCAM Control Bits (3b) • Done: 1b • Hit: 1b • MHit: 1b • Priority: 8b • Present for Primary Filters, for RL and Aux Filters should be 0 • SRAM B0 Word Address: 21b • 2 spare bitS if needed for anything else • 3 Words (<= 96 bits) of Result in SRAM Bank0 • Use Multi-Database Lookup (MDL) Indirect for searching all 3 DBs • Order of fields in Key is important. • Each thread will need one TCAM context

  8. Lookup Processing write KEY to TCAM use timestamp delay to wait appropriate time make delay long enough that we are as sure as possible that we will have to read the 1st word of the Results MB only once while !DoneBit // DONE Bit BUG Fix requires reading just first word read 1 word from Results Mailbox and check DoneBit done read words 2 and 3 from Results Mailbox If (PrimaryFilter AND RouteLookup results HIT) { PrimaryResult.Valid  TRUE compare priorities store higher priority result as Primary Result (read result from SRAM Bank0) } else if (PrimaryFilter results HIT) { PrimaryResult.Valid  TRUE PrimaryResults.*  PrimaryFilter.* (read result from SRAM Bank0) } else if (RouterLookup results HIT) { PrimaryResult.Valid  TRUE PrimaryResults.*  RouteLookup.* (read result from SRAM Bank0) } else PrimaryResult.Valid  False if (AuxiliaryFilter result HIT) { AuxiliaryResult.Valid  TRUE AuxiliaryResults.*  (read result from SRAM Bank0) } else AuxiliaryResult.Valid  FALSE

  9. D (1b) D (1b) D (1b) H (1b) H (1b) H (1b) M H (1b) M H (1b) M H (1b) Res (8b) Res (8b) Prio (8b) Address (21b) Address (21b) Address (21b) NH_MAC (48b) NH_MAC (48b) NH_MAC (48b) Res (16b) Res (16b) Res (16b) NH_IP (32b) NH_IP (32b) NH_IP (32b) Entry Valid (1b) NH IP Valid (1b) NH MAC Valid (1b) PPS (1b) IP MC Valid (1b) Multicast Copy Vector (11b) • If IP MC Valid = 1 Reserved (4b) D (1b) PPS (1b) UCast Out Port (3b) UCast Out Plugin (3b) • If IP MC Valid = 0 Lookup Key and Results Formats IP DAddr (32b) IP SAddr (32b) P Tag (5b) P (3b) Proto (8b) DPort (16b) SPort (16b) Exceptions (16b) TCP Flags (12b) 140 Bit Key: RL PF and AF 32 Bit Result in TCAM Assoc. Data SRAM: 96 Bit Result in QDR SRAM Bank0: V (4b) UCast MCast (12b) QID (16b) Stats Index (16b) PF V (4b) S B (2b) R e s (2b) Uni Cast (8b) QID (16b) Stats Index (16b) AF V (4b) UCast MCast (12b) QID (16b) Stats Index (16b) RL TCAM Ctrl Bits: D:Done H:HIT MH:Multi-Hit

  10. Exception Bits in Lookup Key IP DAddr (32b) IP SAddr (32b) P Tag (5b) P (3b) Proto (8b) DPort (16b) SPort (16b) Exceptions (16b) TCP Flags (12b) • Exception Bits: • TTL: TTL has expired. It was 0 or 1 on arriving packet • IP Opt: IP Packet contained Options • ARP: Ethertype field in ethernet header was ARP • Non-IP: Ethertype field in ethernet header was NOT IP • NOTE: An ARP packet will have ARP bit and Non-IP bit set 140 Bit Key: RL PF and AF Reserved (12b) Non-IP (1b) ARP (1b) IP Opt (1b) TTL (1b)

  11. Performance • What is our performance target? • To hit 5 Gb rate: • Minimum Ethernet frame: 76B • 64B frame + 12B InterFrame Spacing • 5 Gb/sec * 1B/8b * packet/76B = 8.22 Mpkt/sec • IXP ME processing: • 1.4Ghz clock rate • 1.4Gcycle/sec * 1 sec/ 8.22 Mp = 170.3 cycles per packet • compute budget: (MEs*170) • 1 ME: 170 cycles • 2 ME: 340 cycles • 3 ME: 510 cycles • 4 ME: 680 cycles • latency budget: (threads*170) • 1 ME: 8 threads: 1360 cycles • 2 ME: 16 threads: 2720 cycles • 3 ME: 24 threads: 4080 cycles • 4 ME: 32 threads: 5440 cycles • slide taken from ONL_NProuter.ppt

  12. Lookup Block Diagram mem access Latency Setup Lookup Key Write Lookup Key to TCAM SRAM Write: 5W TimeStamp Delay 315 cycles ctx_swap Read 1W Result from AD SRAM Read: 1W 150 cycles ctx_swap Check Done Bit Read 2W Result from AD SRAM Read: 2W 150 cycles ctx_swap SRAM Read: 3W 150 cycles Read 2 Full Results from QDR SRAM Read: 3W 150 cycles ctx_swap Setup Results for Copy TOTAL (No optimization) 915 cycles

  13. File locations (in …/ONL_Router/) • Code • src/applications/ONL_Router/src/plc/ONL/lookup.c • Include Paths • src/applications/ONL_Router/src/dispatch_loop/ONL/ • dl_source.h and dl_source.c • dl_source() and dl_sink() functions • src/IDT_NSE/data_place_IXP2XXX/include • IDT IIPC defines and macros • others?

  14. Test and Validation

  15. Implementation Status • Still in pseudo-code • Bugs • Untested • Optimizations:

  16. Extra Slides • The rest of the slides are either for extra support information or are old and will be deleted when I am convinced they are no longer needed

  17. Route Lookup • Route Lookup Key (72b) • Port (3b): Can be a wildcard (for Unicast, probably not for Multicast) • Value of 111b in Port field can be used to denote a packet that originated from the XScale • Value of 110b in Port field can be used to denots a packet that originated from a Plugin • Ports numbered 0-4 • PluginTag (5b): Can be a wildcard (for Unicast, probably not for Multicast) • Plugins numberd 0-4 • DAddr (32b) • Prefixed for Unicast • Exact Match for Multicast • SAddr (32b) • Unicast entries always have this and its mask set to 0 • Prefixed for Multicast • Route Lookup: Result (96b) • Unicast/Multicast Fields (determined by IP_MCast_Valid bit (1:MCast, 0:Unicast) (13b) • IP_MCast Valid (1b) • MulticastFields (12b) • Plugin/Port Selection Bit (1b): • 0: Send pkt to both Port and Plugin. Does it get the MCast CopyVector? • 1: Send pkt to all Plugin bits set, include MCast CopyVector in data going to plugins • MCast CopyVector (11b) • One bit for each of the 5 ports and 5 plugins and one bit for the XScale, to drop a MCast, set MCast CopyVector to all 0’s • UnicastFields (8b) • Drop Bit (1b) • 0: handle normally • 1: Drop Unicast pkt • Plugin/Port Selection Bit (1b): • 0: Send packet to port indicated by Unicast Output Port field • 1: Send packet to plugin indicated by Unicast Output Plugin field. Unicast Output Port, QID, Stats Index, and NH fields also get sent to plugin • Unicast Output Port (3b): Port or XScale • 0: Port0, 1: Port1, 2: Port2, 3: Port3, 4: Port4 • Unicast Output Plugin (3b): • 0: Plugin0, 1: Plugin1, 2: Plugin2, 3: Plugin3, 4: Plugin4 • 5: XScale (treated like a plugin) • QID (16b) • Stats Index (16b) • NH_IP/NH_MAC (48b): At most one of NH_IP or NH_MAC should be valid • Valid Bits (3b): At most one of the following three bits should be set • IP_MCast Valid (1b) (Also included above) • NH_IP_Valid (1b) • NH_MAC_Valid (1b)

  18. Primary Filter • Primary Filter Lookup Key (140b) • Port (3b): Can be a wildcard (for Unicast, probably not for Multicast) • Value of 111b in Port field to denote coming from the XScale • Ports numbered 0-4 • PluginTag (5b): Can be a wildcard (for Unicast, probably not for Multicast) • Plugins numberd 0-4 • DAddr (32b) • SAddr (32b) • Protocol (8b) • DPort (16b) • Sport (16b) • TCP Flags (12b) • Exception Bits (16b): Allow for directing of packets based on defined exceptions • Primary Filter Result (104b) • Unicast/Multicast Fields (determined by IP_MCast_Valid bit (1:MCast, 0:Unicast) (13b) • IP_MCast Valid (1b) • MulticastFields (12b) • Plugin/Port Selection Bit (1b): • 0: Send pkt to ports and plugins indicated by MCast Copy Vector. • 1: Send pkt to plugin(s) indicated by MCast Copy Vector but not ports and send Plugin(s) the MuticastFields bits • MCast CopyVector (11b) • One bit for each of the 5 ports and 5 plugins and one bit for the XScale, to drop a MCast, set MCast CopyVector to all 0’s • UnicastFields (8b) • Drop Bit (1b) • 0: handle normally • 1: Drop pkt • Plugin/Port Selection Bit (1b): • 0: Send packet to port indicated by Unicast Output Port field • 1: Send packet to plugin indicated by Unicast Output Plugin field. Unicast Output Port, QID, Stats Index, and NH fields also get sent to plugin • Unicast Output Port (3b): Port or XScale • 0: Port0, 1: Port1, 2: Port2, 3: Port3, 4: Port4 • Unicast Output Plugin (3b): • 0: Plugin0, 1: Plugin1, 2: Plugin2, 3: Plugin3, 4: Plugin4 • 5: XScale (treated like a plugin) • QID (16b) • Stats Index (16b) • NH IP(32b)/MAC(48b) (48b): At most one of NH_IP or NH_MAC should be valid • Valid Bits (3b): At most one of the following three bits should be set • IP_MCast Valid (1b) (also included above) • NH IP Valid (1b) • NH MAC Valid (1b) • Priority (8b)

  19. Auxiliary Filter • Auxiliary Filter Lookup Key (140b) • Port (3b): Can be a wildcard (for Unicast, probably not for Multicast) • Value of 111b in Port field to denote coming from the XScale • Ports numbered 0-4 • PluginTag (5b): Can be a wildcard (for Unicast, probably not for Multicast) • Plugins numberd 0-4 • DAddr (32b) • SAddr (32b) • Protocol (8b) • DPort (16b) • Sport (16b) • TCP Flags (12b) • Exception Bits (16b) • Allow for directing of packets based on defined exceptions • Can be wildcarded. • Auxiliary Filter Lookup Result (93b) • Unicast Fields (8b): (No Multicast fields) • Drop Bit (1b) (Should never actually be set by control software, but keep here for symmetry with other Unicast Fields) • 0: handle normally • 1: Drop pkt • Plugin/Port Selection Bit (1b): • 0: Send packet to port indicated by Unicast Output Port field • 1: Send packet to plugin indicated by Unicast Output Plugin field. Unicast Output Port, QID, Stats Index, and NH fields also get sent to plugin • Unicast Output Port (3b): Port or XScale • 0: Port0, 1: Port1, 2: Port2, 3: Port3, 4: Port4 • Unicast Output Plugin (3b): • 0: Plugin0, 1: Plugin1, 2: Plugin2, 3: Plugin3, 4: Plugin4 • 5: XScale • QID (16b) • Stats Index (16b) • NH IP(32b)/MAC(48b) (48b): At most one of NH_IP or NH_MAC should be valid • Valid Bits (3b): At most one of the following three bits should be set • NH IP Valid (1b) • NH MAC Valid (1b) • IP_MCast Valid (1b): Should always be 0 for AF Result • Sampling bits (2b) : For Aux Filters only • 00: “Sample All” • 01: Use Random Number generator 1 • 10: Use Random Number generator 2 • 11: Use Random Number generator 3

More Related