1 / 77

Design of a Diversified Router: Line Card

Design of a Diversified Router: Line Card. John DeHart jdd@arl.wustl.edu http://www.arl.wustl.edu/arl. Revision History. 5/22/06 (JDD): Updates to data passing from Block to Block. Buffer descriptor stuff probably needs updating. Outline. What is NOT covered in these slides

henleyt
Download Presentation

Design of a Diversified Router: Line Card

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Design of aDiversified Router: Line Card John DeHartjdd@arl.wustl.edu http://www.arl.wustl.edu/arl

  2. Revision History • 5/22/06 (JDD): • Updates to data passing from Block to Block. • Buffer descriptor stuff probably needs updating.

  3. Outline • What is NOT covered in these slides • JST’s original slides • Schedule • Model • Traffic Types • IP Addressing • Components • Switch • MetaLink Loopback Block • LC • Substrate Link Types • Packet Formats • LC Ingress/Egress Design Implementation • Common Router Framework (CRF) • Functional Blocks for implementing a Router

  4. Not Covered • Control • Installation • Configuration • Initialization • Monitoring • … • End Host • Substrate/Meta Model • Details about how PlanetLab works. • These are very important topics but for now we will talk about the Data Path in the Network.

  5. Outline • JST’s original slides • Schedule • Model • Traffic Types • IP Addressing • Components • Switch • MetaLink Loopback Block • LC • Substrate Link Types • Packet Formats • LC Ingress/Egress Design Implementation • Common Router Framework (CRF) • Functional Blocks for implementing a Router

  6. Substrate RX MR Substrate TX QM Rx DeMux Parse Lookup HeaderFormat Tx Design Implementation • We will now look at how our model might be implemented. • Again, the primary focus will be on wired Ethernet as the access technology. • We will look at RX and TX separately. • First we will focus on the Substrate Router functionality in the LC. • As we do we will also show the packet formats as they traverse other parts of the Substrate Router. • Some of this will apply only to the Shared NP case. • For the sake of simpler diagrams we will reduce the IXP PE functional blocks as shown below: • Then we will look at the implementation of a common router framework in the IXP PE.

  7. LC … Design Implementation Packet arriving On Port N Packet leaving On Port M LC Rxsubstrate MR Txsubstrate Switch Switch … IXP PE (Shared) RX TX

  8. Substrate Link Configuration • RX Rules for determining type of Frame/Substrate Link: • P2P-DC is pre-configured on an interface by interface basis. • ((EtherType = Substrate) AND (VLAN=VLAN0)) P2P-VLAN0 SL • VLAN0 is a predefined VLAN Id for use by the Substrate Network to connect peer substrate routers on a Multi-Access network • SL id = Senders Ethernet Address • Lookup MLI in SL Specific Table  MR:MI • ((EtherType = Substrate) & (VLAN ≠ VLAN0)) OR ((EtherType = ARP) and (ProtoType = Substrate))  Multi-Access • MLI is unique across a Multi-Access Substrate Link • MLI Lookup in Multi-Access-SL Table  MR:MI • ((EtherType = ARP) and (ProtoType ≠ Substrate)) OR ((EtherType ≠ Substrate) AND (EtherType ≠ ARP)) Tunnel or Legacy • Substrate Link in a Tunnel (e.g. IPv4) • ((EtherType = IP) AND (IP_Proto = Substrate))  Substrate Tunnel in IPv4 • Legacy (e.g. IPv4, ARP): • ((EtherType = IP) AND (IP_Proto = TCP))  Legacy IPv4 • ((EtherType = ARP) AND (ProtoType ≠ Substrate))  Legacy ARP • Etc.

  9. Substrate Link Configuration • TX: • Substrate Link uniquely identified by MR:MI tuple • MR:MI  SL • MetaLink uniquely identified by MR:MI tuple • MR:MI  MLI • Hence: • MR:MI  SL:MLI • Different Header formats may be defined per SL • i.e. IPv4 Tunnel header vs. P2P-VLAN0 header • Located with lookup based on MLI • P2P-VLAN0 • One or more MLI per MetaNet on the Multi-Access network • Multi-Access • One MLI per MetaNet on the Multi-Access network • Meta-Destination Address, to get Ethernet Address use ARP • Legacy: • Substrate Link in a Tunnel • IPv4 • Gateway • Legacy to/from MetaNet

  10. Tables and Lookups • The following slides have some tables and lookups. • They are not meant as the only way to do things, just one way. • Some if not all of the tables and lookups will almost certainly be implemented using the TCAM. • For different types of frames we might need different types of headers. • For this a lookup result will probably indicate what format/template to use and some/all of the info to use. • The Point: • These slides are meant to show that we have the right information at the right place to do the needed lookup. • There is still some design work for the actual implementer(s) to do to get it all right and make it efficient.

  11. LC LC Packet TX: All SL Types • MnFlags: define what type of Next Hop Address (NhAddr) is being given: • or perhaps what type is needed (maybe Substrate provides it for broadcast…) • Type (2b) • NhMnAddr: (01b) • We may need to use ARP to translate to MAC/Ethernet Address • MAC Address(10b) • Could be used for Broadcast/Multicast • NULL (00b): No Next Hop Address given or needed. • Size (6b): Length of NhAddr field in Bytes MR Txsubstrate Switch … MI (2B) Blade-to-Blade Ethernet Header (MR specific VLAN) MnFlags(1B) NhAddr (nB) LEN (2B) Meta Frame TxMI (2B) MnFlags (1B) NhAddr (nB) LEN (2B) Meta Frame PAD (nB) CRC (4B)

  12. Key Result VLANMNid MnID:MnAddr DAddr MnID:MnAddr DAddr LC MnId MnID:MnAddr DAddr MnID MnID:MnAddr DAddr . . . MnID MnID Blade-to-Blade Ethernet Header (MR specific VLAN) . . . MnID:MnAddr DAddr MnID TxMI (2B) Port MnFlags (1B) Port NhAddr (nB) SL/ML Table LEN (2B) Port Meta Frame Port MLI Hdr Info/Templates SL_Type PAD (nB) MLI Hdr Info/Templates SL_Type MLI Hdr Info/Templates SL_Type CRC (4B) Port MLI Hdr Info/Templates SL_Type Arp Cache . . . MLI Hdr Info/Templates SL_Type LC Packet TX: All SL Types MR Txsubstrate Switch … MI (2B) MnFlags(1B) NhAddr (nB) LEN (2B) Meta Frame If SL_TYPE=Multi-Access Lookup(MnID, MnAddr) VLAN:TxMI  SL:MLI

  13. Key Result MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr . . . MnID:MnAddr DAddr Port Port SL/ML Table Port Port MLI Hdr Info/Templates SL_Type Blade-to-Blade Ethernet Header MR specific VLAN MLI Hdr Info/Templates SL_Type MLI Hdr Info/Templates SL_Type Port MLI Hdr Info/Templates SL_Type Arp Cache . . . TxMI (2B) MnFlags (1B) MLI Hdr Info/Templates SL_Type NhMnAddr (nB) LEN (2B) Meta Frame PAD (nB) CRC (4B) LC Packet TX – P2P-DC • P2P-DC does not use ARP Cache. • DAddr is configured in the Hdr field of Per SL ML Tables DstAddr (6B) • P2P-DC should not need to give NhMnAddr • MnFlags would indicate NULL SrcAddr (6B) Type=802.1Q (2B) TCI (2B) Type=Substrate (2B) MLI (2B) LEN (2B) Meta Frame PAD (nB) Recalculate CRC (4B)

  14. Key Result MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr . . . MnID:MnAddr DAddr Port Port SL/ML Table DstAddr (6B) Port SrcAddr (6B) Port MLI Hdr Info/Templates SL_Type Blade-to-Blade Ethernet Header MR specific VLAN Type=802.1Q (2B) MLI Hdr Info/Templates SL_Type TCI ≠ VLAN0 (2B) MLI Hdr Info/Templates SL_Type Type=IP (2B) Port Ver/HLen/Tos/Len (4B) MLI Hdr Info/Templates SL_Type Arp Cache . . . ID/Flags/FragOff (4B) TTL (1B) TxMI (2B) Protocol=Substrate (1B) MnFlags (1B) Hdr Cksum (2B) MLI Hdr Info/Templates SL_Type NhMnAddr (nB) Src Addr (4B) LEN (2B) Dst Addr (4B) Meta Frame MLI (2B) LEN (2B) Meta Frame PAD (nB) PAD (nB) CRC (4B) CRC (4B) LC Packet TX – P2P-Tunnel (IPv4) • This seems to assume that we have a static first hop for the IP Tunnel. • Is that reasonable? • Probably, Yes. • P2P-Tunnel does not use ARP Cache. • DAddr is configured in the Hdr field of Per SL ML Tables • This may be an over-simplification. • The packet needs to be routed toward the other end of the Tunnel Recalculate

  15. Key Result MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr . . . MnID:MnAddr DAddr DstAddr (6B) Port SrcAddr (6B) Port SL/ML Table Type=802.1Q (2B) Port TCI=VLAN0 (2B) Port Type=Substrate (2B) MLI Hdr Info/Templates SL_Type MLI (2B) Blade-to-Blade Ethernet Header MR specific VLAN MLI Hdr Info/Templates SL_Type LEN (2B) Meta Frame MLI Hdr Info/Templates SL_Type Port MLI Hdr Info/Templates SL_Type Arp Cache . . . TxMI (2B) PAD (nB) MnFlags (1B) MLI Hdr Info/Templates SL_Type NhMnAddr (nB) CRC (4B) LEN (2B) Meta Frame PAD (nB) CRC (4B) LC Packet TX – P2P-VLAN0 • Do we want the P2P-VLAN0 to use the ARP Cache? • Or is the DAddr configured in the Hdr fields of Per SL ML Tables? Recalculate

  16. Key Result MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr . . . MnID:MnAddr DAddr DstAddr (6B) Port SrcAddr (6B) Port SL/ML Table Type=802.1Q (2B) Port TCI≠VLAN0 (2B) Port Type=Substrate (2B) MLI Hdr Info/Templates SL_Type MLI (2B) Blade-to-Blade Ethernet Header MR specific VLAN MLI Hdr Info/Templates SL_Type LEN (2B) Meta Frame MLI Hdr Info/Templates SL_Type Port MLI Hdr Info/Templates SL_Type Arp Cache . . . TxMI (2B) PAD (nB) MnFlags (1B) MLI Hdr Info/Templates SL_Type NhMnAddr (nB) CRC (4B) LEN (2B) Meta Frame PAD (nB) CRC (4B) LC Packet TX – Multi-Access • Multi-Access does use ARP Cache. • If DAddr entry for MnAddr is missing, send packet up to XScale for it to perform ARP Request. Recalculate

  17. DstAddr (6B) DstAddr (6B) SrcAddr (6B) SrcAddr (6B) Type=802.1Q (2B) Type=802.1Q (2B) TCI (2B) TCI ≠ VLAN0 (2B) DstAddr (6B) DstAddr (6B) Type=Substrate (2B) Type=IP (2B) MLI (2B) Ver/HLen/Tos/Len (4B) SrcAddr (6B) SrcAddr (6B) LEN (2B) ID/Flags/FragOff (4B) Type=802.1Q (2B) Meta Frame TTL (1B) Type=802.1Q (2B) TCI≠VLAN0 (2B) Protocol (1B) TCI=VLAN0 (2B) Hdr Cksum (2B) Type=Substrate (2B) Type=Substrate (2B) Src Addr (4B) MLI (2B) MLI (2B) LEN (2B) Dst Addr (4B) LEN (2B) PAD (nB) Meta Frame Meta Frame IP Payload CRC (4B) PAD (nB) PAD (nB) PAD (nB) CRC (4B) CRC (4B) CRC (4B) P2P-VLAN0 Multi-Access Legacy LC Packet RX: All SL Types VLAN-tagged formats • Blue Shading: Determine SL Type • Black Outline: Key Fields from pkt DstAddr (6B) SrcAddr (6B) Type=802.1Q (2B) TCI ≠ VLAN0 (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol=Substrate (1B) Hdr Cksum (2B) Src Addr (4B) Dst Addr (4B) MLI (2B) LEN (2B) Meta Frame PAD (nB) CRC (4B) P2P-DC (Configured) P2P-Tunnel

  18. LC Packet RX: All SL Types Non VLAN-tagged formats • Blue Shading: Determine SL Type • Black Outline: Key Fields from pkt DstAddr (6B) DstAddr (6B) SrcAddr (6B) SrcAddr (6B) Type=IP (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) Ver/HLen/Tos/Len (4B) DstAddr (6B) DstAddr (6B) ID/Flags/FragOff (4B) ID/Flags/FragOff (4B) TTL (1B) TTL (1B) SrcAddr (6B) SrcAddr (6B) Protocol=Substrate (1B) Protocol (1B) Hdr Cksum (2B) Hdr Cksum (2B) Type=Substrate (2B) Type=Substrate (2B) Src Addr (4B) Src Addr (4B) MLI (2B) MLI (2B) Dst Addr (4B) Dst Addr (4B) LEN (2B) LEN (2B) Meta Frame MLI (2B) Meta Frame IP Payload LEN (2B) Meta Frame PAD (nB) PAD (nB) PAD (nB) PAD (nB) CRC (4B) CRC (4B) CRC (4B) CRC (4B) P2P-DC (Configured) P2P-Tunnel Legacy Multi-Access

  19. LC Packet RX: All SL Types Optional Extension: GRE formats • Blue Shading: Determine SL Type • Black Outline: Key Fields from pkt DstAddr (6B) SrcAddr (6B) DstAddr (6B) Type=802.1Q (2B) SrcAddr (6B) TCI ≠ VLAN0 (2B) Type=IP (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) ID/Flags/FragOff (4B) TTL (1B) TTL (1B) Protocol=GRE (1B) Protocol=GRE (1B) Hdr Cksum (2B) Hdr Cksum (2B) Src Addr (4B) Src Addr (4B) ` Dst Addr (4B) Dst Addr (4B) Flags/recur/Ver (2B) Flags/recur/Ver (2B) Type=Substrate (2B) ` Type=Substrate (2B) ` Optional Fields (nB) Optional Fields (nB) MLI (2B) MLI (2B) ` LEN (2B) LEN (2B) Meta Frame MLI (2B) Meta Frame PAD (nB) PAD (nB) CRC (4B) CRC (4B) VLAN GRE GRE

  20. VLANMR RxMI Blade QID VLANMR VLANMR RxMI RxMI Blade Blade QID QID VLANMR VLANMR RxMI RxMI Blade Blade QID QID VLANMR VLANMR RxMI Blade Blade QID QID RxMI VLANMR Blade QID RxMI (Port N P2P-DC Table) (Port N P2P-DC Table) VLANMR RxMI Blade QID VLANMR RxMI Blade QID DstAddr (6B) Blade-to-Blade Ethernet Header MR Specific VLAN SrcAddr (6B) Type=802.1Q (2B) TCI (2B) Type=Substrate (2B) RxMI (2B) MLI (2B) LEN (2B) MI (2B) LEN (2B) Meta Frame Meta Frame LEN (2B) Meta Frame PAD (nB) PAD (nB) CRC (4B) CRC (4B) LC Packet RX: P2P-DC Per SL ML Tables VLANMR RxMI Blade QID Packet arriving On Port N VLANMR RxMI Blade QID VLANMR RxMI Blade QID VLANMR Blade QID RxMI (Port N P2P-DC Table) VLANMR RxMI Blade QID • Do we want to include Sender’s MnAddr in Frame going to MR? • No. • But we may need to give it the MAC address. • There is one P2P-DC SL Table per Port (Physical Interface). • Blade info in tables provides info so we can fill in templates for appropriate header(s) to get frame to necessary blade. • QID is used on LC Recalculate Packet arriving On Port N LC Rxsubstrate MR Switch …

  21. VLANMR RxMI Blade QID VLANMR VLANMR RxMI RxMI Blade Blade QID QID VLANMR VLANMR RxMI RxMI Blade Blade QID QID VLANMR VLANMR RxMI Blade Blade QID QID RxMI VLANMR Blade QID RxMI (Port N P2P-DC Table) (Port N P2P-DC Table) VLANMR RxMI Blade QID VLANMR RxMI Blade QID Blade-to-Blade Ethernet Header MR Specific VLAN DstAddr (6B) SrcAddr (6B) RxMI (2B) LEN (2B) Type=802.1Q (2B) MI (2B) Meta Frame TCI ≠ VLAN0 (2B) LEN (2B) Type=IP (2B) Meta Frame Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) PAD (nB) Protocol=Substrate (1B) Hdr Cksum (2B) CRC (4B) Src Addr (4B) Dst Addr (4B) MLI (2B) LEN (2B) Meta Frame PAD (nB) CRC (4B) LC Packet RX: P2P-Tunnel Packet arriving On Port N Per SL ML Tables VLANMR RxMI Blade QID VLANMR RxMI Blade QID Fct(PortN, IPv4, IP Src_Addr) VLANMR RxMI Blade QID VLANMR Blade QID RxMI (Port N IPv4 Tunnel(i) Table) VLANMR RxMI Blade QID • Do we want to include Sender’s MnAddr in Frame going to MR? • Probably not • IP Routers don’t • There is one P2P-Tunnel SL Table per tunnel SL. Recalculate Packet arriving On Port N LC Rxsubstrate MR Switch …

  22. VLANMR RxMI Blade QID VLANMR VLANMR RxMI RxMI Blade Blade QID QID VLANMR VLANMR RxMI RxMI Blade Blade QID QID VLANMR VLANMR RxMI Blade Blade QID QID RxMI VLANMR Blade QID RxMI (Port N P2P-DC Table) (Port N P2P-DC Table) VLANMR RxMI Blade QID VLANMR RxMI Blade QID DstAddr (6B) Blade-to-Blade Ethernet Header MR Specific VLAN SrcAddr (6B) Type=802.1Q (2B) TCI=VLAN0 (2B) Type=Substrate (2B) RxMI (2B) MLI (2B) LEN (2B) MI (2B) LEN (2B) Meta Frame Meta Frame LEN (2B) Meta Frame PAD (nB) PAD (nB) CRC (4B) CRC (4B) LC Packet RX: P2P-VLAN0 Per SL ML Tables Per SL ML Tables VLANMR RxMI Blade QID VLANMR RxMI Blade QID Packet arriving On Port N VLANMR RxMI Blade QID VLANMR Blade QID RxMI (Port N P2P-VLAN0 Table) VLANMR RxMI Blade QID • Do we want to include Sender’s MnAddr in Frame going to MR? • There is one P2P-VLAN0 SL Table per SL. • Located based on SrcAddr of sender Recalculate Packet arriving On Port N LC Rxsubstrate MR Switch …

  23. VLANMR RxMI Blade QID VLANMR VLANMR RxMI RxMI Blade Blade QID QID VLANMR VLANMR RxMI RxMI Blade Blade QID QID VLANMR VLANMR RxMI Blade Blade QID QID RxMI VLANMR Blade QID RxMI (Port N P2P-DC Table) (Port N P2P-DC Table) VLANMR RxMI Blade QID VLANMR RxMI Blade QID DstAddr (6B) Blade-to-Blade Ethernet Header MR Specific VLAN SrcAddr (6B) Type=802.1Q (2B) TCI≠VLAN0 (2B) Type=Substrate (2B) RxMI (2B) MLI (2B) LEN (2B) MI (2B) LEN (2B) Meta Frame Meta Frame LEN (2B) Meta Frame PAD (nB) PAD (nB) CRC (4B) CRC (4B) LC Packet RX: Multi-Access Per SL ML Tables VLANMR RxMI Blade QID Packet arriving On Port N VLANMR RxMI Blade QID VLANMR RxMI Blade QID VLANMR Blade QID RxMI (Port N Multi-Access Table) VLANMR RxMI Blade QID • Do we want to include Sender’s MnAddr in Frame going to MR? • There is one Multi-Access SL Table per port. • Contains the MLI entries for the Multi-Access SL Recalculate Packet arriving On Port N LC Rxsubstrate MR Switch …

  24. DstAddr (6B) GE GE LC LC SrcAddr (6B) GE GE Type=802.1Q (2B) GE GE TCI ≠ VLAN0 (2B) Type=IP (2B) GE GE Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) GE GE TTL (1B) Protocol ≠ Substrate (1B) GE GE Hdr Cksum (2B) Src Addr (4B) GE GE Dst Addr (4B) GE GE IP Packet Body GE GE PAD (nB) GE GE CRC (4B) Pure Legacy Traffic IXP PE Blade IXP PE Blade GP Blade GP Blade . . . IPv4 MR • LC: • If ((EtherType=IPv4) AND (IP Proto ≠ Substrate)) • then non-Tunnel Legacy • Send frame to pre-configured default IPv4 MR Switch Blade

  25. DstAddr (6B) VLANMR RxMI Blade QID SrcAddr (6B) VLANMR VLANMR RxMI RxMI Blade Blade QID QID VLANMR VLANMR RxMI RxMI Blade Blade QID QID Type=802.1Q (2B) TCI ≠ VLAN0 (2B) VLANMR VLANMR RxMI Blade Blade QID QID RxMI Type=IP (2B) VLANMR Blade QID RxMI (Port N P2P-DC Table) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) (Port N P2P-DC Table) TTL (1B) VLANMR RxMI Blade QID Protocol ≠ Substrate (1B) VLANMR RxMI Blade QID Hdr Cksum (2B) Src Addr (4B) Blade-to-Blade Ethernet Header MR Specific VLAN Dst Addr (4B) IP Packet Body PAD (nB) RxMI (2B) CRC (4B) LEN (2B) MI (2B) Meta Frame LEN (2B) Meta Frame PAD (nB) CRC (4B) LC Packet RX: Legacy IPv4 Packet arriving On Port N Per SL ML Tables Fct(PortN) VLANMR RxMI Blade QID VLANMR RxMI Blade QID VLANMR RxMI Blade QID Fct(IPv4) VLANMR Blade QID RxMI (Port N Legacy MR Table) VLANMR RxMI Blade QID • There is one Legacy MR Table per Port. • Entries point to Legacy MR for the specified Protocol. • There is at most one Legacy MR for each legacy protocol Recalculate Recalculate Packet arriving On Port N LC Rxsubstrate MR Switch …

  26. Key Result VLANMNid MnID:MnAddr DAddr MnID:MnAddr DAddr LC MnId MnID:MnAddr DAddr MnID MnID:MnAddr DAddr . . . MnID MnID Blade-to-Blade Ethernet Header (MR specific VLAN) . . . MnID:MnAddr DAddr MnID TxMI (2B) Port MnFlags (1B) Port NhAddr (nB) SL/ML Table LEN (2B) Port Meta Frame Port MLI Hdr Info/Templates SL_Type PAD (nB) MLI Hdr Info/Templates SL_Type MLI Hdr Info/Templates SL_Type CRC (4B) Port MLI Hdr Info/Templates SL_Type Arp Cache . . . MLI Hdr Info/Templates SL_Type LC Packet TX: All SL Types MR Txsubstrate Switch … MI (2B) MnFlags(1B) NhAddr (nB) LEN (2B) Meta Frame If SL_TYPE=Multi-Access Lookup(MnID, MnAddr) VLAN:TxMI  SL:MLI

  27. Key Result MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr MnID:MnAddr DAddr . . . Blade-to-Blade Ethernet Header (MR specific VLAN) MnID:MnAddr DAddr TxMI (2B) Port MnFlags (1B) DstAddr (6B) Port NhAddr (nB) SL/ML Table LEN (2B) Port SrcAddr (6B) Meta Frame Port Type=802.1Q (2B) MLI Hdr Info/Templates SL_Type TCI ≠ VLAN0 (2B) PAD (nB) MLI Hdr Info/Templates SL_Type Type=IP (2B) MLI Hdr Info/Templates SL_Type Ver/HLen/Tos/Len (4B) CRC (4B) Port ID/Flags/FragOff (4B) MLI Hdr Info/Templates SL_Type Arp Cache . . . TTL (1B) Protocol ≠ Substrate (1B) Hdr Cksum (2B) MLI Hdr Info/Templates SL_Type Src Addr (4B) Dst Addr (4B) IP Packet Body PAD (nB) CRC (4B) LC Packet TX – Legacy IPv4 • Legacy IPv4 does use ARP Cache. • If DAddr entry for MnAddr is missing, send packet up to XScale for it to perform ARP Request. Recalculate

  28. GE GE LC LC GE GE GE GE GE GE GE GE GE GE GE GE GE GE DstAddr (6B) GE GE SrcAddr (6B) GE GE Type=802.1Q (2B) TCI ≠ VLAN0 (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol ≠ Substrate (1B) Hdr Cksum (2B) Src Addr (4B) Dst Addr (4B) IP Packet Body PAD (nB) CRC (4B) Pure Legacy Traffic IXP PE Blade IXP PE Blade GP Blade GP Blade . . . IPv4 MR Switch Blade

  29. Substrate/MetaNet Model MetaLink 1 Substrate Link MetaLink 2 MetaLink 3 • To the Substrate, some Meta Links “Pass Through” • Pass through a Substrate Router without visiting a Meta Router MR_A MR_A MR_A MI MI MI MI MI MI MR_B MR_B Pass Through Meta Link MI MI MI MI MR_C MR_C MR_C MI MI MI MI MI MI Substrate Router X Substrate Router Z Substrate Router Y

  30. Blade-to-Blade Ethernet Header (MR specific VLAN) Blade-to-Blade Ethernet Header (MR specific VLAN) DstAddr (6B) RxMI (2B) TxMI (2B) SrcAddr (6B) MnFlags (1B) MnFlags (1B) NhAddr (nB) NhAddr (nB) Type=802.1Q (2B) LEN (2B) LEN (2B) TCI=VLAN0 (2B) Meta Frame Meta Frame Type=Substrate (2B) MLI (2B) PAD (nB) PAD (nB) LEN (2B) Meta Frame CRC (4B) CRC (4B) PAD (nB) CRC (4B) Pass-Through MetaLink Packet arriving On Port N • For Pass-Through ML, we need to include MnFlags and NhMnAddr so Tx can handle Multi-Access case • Allocate special set of MR/MI for use by Substrate when handling Pass-Through MetaLinks Packet arriving At LC Y Packet arriving On Port N Of LC X Ethernet Switch LC Y LC X …

  31. Per SL ML Tables MR RxMI Blade QID MR RxMI Blade QID MR RxMI Blade QID MR Blade QID RxMI (Port N P2P-DC Table) MR RxMI Blade QID Blade-to-Blade Ethernet Header (MR specific VLAN) DstAddr (6B) RxMI (2B) SrcAddr (6B) MnFlags (1B) NhAddr (nB) Type=802.1Q (2B) LEN (2B) TCI (2B) Meta Frame Type=Substrate (2B) MLI (2B) PAD (nB) LEN (2B) Meta Frame CRC (4B) PAD (nB) CRC (4B) MR MR RxMI RxMI Blade Blade QID QID MR MR RxMI RxMI Blade Blade QID QID MR MR RxMI RxMI Blade Blade QID QID MR MR Blade Blade QID QID RxMI RxMI . . . . . . MR MR RxMI RxMI Blade Blade QID QID Rx: Pass-Through MetaLink Packet arriving On Port N Recalculate Packet arriving On Port N Of LC X Ethernet Switch LC Y LC X …

  32. Key Result VLANMNid MnID:MnAddr DAddr MnID:MnAddr DAddr MnId MnID:MnAddr DAddr MnID MnID:MnAddr DAddr . . . MnID MnID Blade-to-Blade Ethernet Header (MR specific VLAN) . . . MnID:MnAddr DAddr MnID TxMI (2B) Port MnFlags (1B) Port NhAddr (nB) SL/ML Table LEN (2B) Port Meta Frame Port MLI Hdr Info/Templates SL_Type PAD (nB) MLI Hdr Info/Templates SL_Type MLI Hdr Info/Templates SL_Type CRC (4B) Port MLI Hdr Info/Templates SL_Type Arp Cache . . . MLI Hdr Info/Templates SL_Type Tx: Pass-Through MetaLink Ethernet Switch LC Y LC X … If SL_TYPE=Multi-Access Lookup(MnID, MnAddr) VLAN:TxMI  SL:MLI • This is exactly like the common case shown for all SL Types. • After this things proceed as previously shown for the type of SL that is being used for the Transmit.

  33. LC: Functional Blocks Lookup (1 ME) Hdr Format (1 ME) QM/Schd (1 ME) Switch Tx (3 ME) S W I T C H Phy Int Rx (2 ME) Key Extract (1 ME) Phy Int Tx (3 ME) QM/Schd (1 ME) Hdr Format (1 ME) Lookup (1 ME) Key Extract (1 ME) Switch Rx (2 ME) Rate Monitor (1 ME) • ME counts are my first guesses for number needed. • For each block I’ll show: • Function: • What this block does. • Memory Accesses: • SRAM: • DRAM: • Buffer Descriptor Accesses: • Which fields in Buffer Descriptor the block needs to read and/or write. • Notes: • Additional thoughts about this block. • Next: • Analyze the SRAM Accesses and try to map them on to the available SRAM Channels. • Analyze the SRAM and DRAM accesses and calculate packet processing rates.

  34. LC: Functional Blocks • Ingress (Physical Interface  Switch): • PhyInt Rx • KeyExtractor • Lookup • Hdr Format • QM/Scheduler • Switch Tx • Egress (Switch  Physical Interface): • Switch Rx • KeyExtractor • This is only extracting the VLAN and the MI. Combine with previous or following block? • But it involves a DRAM Read so we probably want to leave it separate and use all 8 threads. • RateMonitor • Before or after the Lookup? • Is this different than what QM/Scheduler will/could do? • Lookup • Hdr Format • QM/Scheduler • PhyInt Tx

  35. Buffer Descriptor Buffer_Next Buffer_Size Offset Free_List Packet_Size MR_ID TxMI VLAN Packet_Next LC: Buffer Descriptor • Hopefully we can use the same buffer descriptor for the LC and the CRF Processing Engine. • There might be some fields that are used on one and not on the other but that’s ok (MR_ID, TxMI, VLAN not needed on LC) • PE Buffer Descriptor: • LW0: buffer_next 32 bits Next Buffer Pointer (in a chain of buffers) • LW1: offset 16 bits Offset to start of data in bytes • LW1: BufferSize 16 bits Length of data in the current buffer in bytes • LW2: reserved 8 bits reserved/unused • LW2: reserved 4 bits reserved/unused • LW2: free_list 4 bits Freelist ID • LW2: packet_size 16 bits (Total packet size across multiple buffers) • LW3: MR_ID 16 bits Meta Router ID • LW3: TxMI 16 bits Transmit Meta Interface • LW4: VLAN 16 bits VLAN • LW4: reserved 16 bits reserved/unused • LW5: reserved 32 bits reserved/unused • LW6: reserved 32 bits reserved/unused • LW7: packet_next 32 bits pointer to next packet (unused in cell mode) • Leave multi-buffer fields there as a template for the dedicated blade implementation of a jumbo-frame MR. • Also reduces changes to Rx, Tx, and QM and reduces potential problems. • So, far I haven’t found anything extra that we need on LC.

  36. Buffer Descriptor Buffer_Next Buffer_Size Offset Free_List Packet_Size MR_ID TxMI QM/Schd VLAN Packet_Next LC Ingress: Functional Blocks Lookup Hdr Format Switch Tx S W I T C H Phy Int Rx Key Extract Buf Handle(32b) RBUF CRC(32b) Port(8b) • Rx: • Function: • Coordinate transfer of packets from RBUF to DRAM • Memory Accesses: • SRAM: • Write Buffer Descriptor • Free List? • DRAM: Transfer from RBUF • Buffer Descriptor Accesses: • Write/Initialize: Buffer_Next, Buffer_Size, Offset, Free_List, Packet_Size • Monitoring: • Per Physical Interface • Pkt Counter • Byte Counter • Notes: • Buffer Handle: • contains the SRAM address of the buffer descriptor. • from the SRAM address of the descriptor we can calculate the DRAM address of the buffer data. • Offset of where packet starts should be a constant.

  37. Buffer Descriptor Buffer_Next Buffer_Size Offset Free_List Packet_Size MR_ID TxMI QM/Schd VLAN Packet_Next LC Ingress: Functional Blocks Lookup Hdr Format Switch Tx S W I T C H Phy Int Rx Key Extract Buf Handle(32b) Buf Handle(32b) CRC(32b) Lookup Key(9B) Port(8b) • Key_Extract (2 Microengines): • Function: • Extracts lookup key based on type of frame that was received. • Peel ARP packets off and send to XScale • Compare CRC from Rx to CRC at end of Frame. • Drop if fail. • Memory Accesses: • DRAM: • Read as much of header as is necessary to extract key • May vary depending on type of Substrate Link • SRAM: None • Buffer Descriptor Accesses: None • Monitoring: • ARP Pkt Counter • Notes: • Calculates DRAM Address based on SRAM descriptor address in buffer handle and the Offset passed to it by RX. • Frame offset in buffer is a constant and does not need to be read from Buffer Descriptor

  38. LC Ingress: Functional Blocks SL Type (SL Type ID) (size) P2P-DC(0x0) (24 bits) SL(4b) 0100 Port (4b) MLI(16b) P2P-Tunnel - IPv4(0x1) (72 bits) EtherType (16b) 0x0800 IP SAddr (32b) MLI (16b) SL(4b) 0100 Port (4b) P2P-VLAN0(0x2) (72 bits) Ethernet SAddr (48b) MLI (16b) SL(4b) 0100 Port (4b) MA(0x3) (24 bits) SL(4b) 0100 Port (4b) MLI(16b) • Substrate Link Type: • Will be used as Database ID by Lookup Block • Lookup Keys: Legacy IPv4(0x4) (24 bits) SL(4b) 0100 Port (4b) EtherType (16b) 0x0800 Physical Interface # Substrate Link Type

  39. QM/Schd LC Ingress: Functional Blocks Lookup Hdr Format Switch Tx S W I T C H Phy Int Rx Key Extract Buf Handle(32b) Buf Handle(32b) Lookup Result (10B) Lookup Key(9B) • Lookup: • Function: • Performs Lookup and passes result on to Hdr Format. • Memory Accesses: • DRAM: None • SRAM: • TCAM Lookup • Write Lookup command • Read Lookup Result (1-5 words from TCAM SRAM Controller or other SRAM Controller) • Buffer Descriptor Accesses: None • Monitoring: • Notes: • Lookup does no processing on the lookup result. • Need to decide how lookup result will be stored and retrieved. • See notes on TCAM for information about the issues involved.

  40. Buf Handle(32b) Buf Handle(32b) Lookup Result (8B) Size (16b) Buffer Descriptor QID(24b) Buffer_Next Buffer_Size Port(8b) Offset Free_List Packet_Size MR_ID TxMI QM/Schd VLAN Packet_Next LC Ingress: Functional Blocks Lookup Hdr Format Switch Tx S W I T C H Phy Int Rx Key Extract • Hdr Format: • Function: • From lookup result: • re-writes headers in DRAM to make frame ready to transmit. • Extract QID to pass on to QM/Scheduler • Memory Accesses: • DRAM: • SRAM: • Read Descriptor and Re-Write Descriptor • OR • Atomic Increment/Decrement some fields in Descriptor • Buffer Descriptor Accesses: • Update Size and Offset fields • Monitoring: • Notes: • Pass Size on to QM/Scheduler so it does not have to read buffer descriptor for Enqueue to update Q Length. • Buffer Offset should be a constant and should not need to be read from Buffer Descriptor

  41. Buf Handle(32b) Buf Handle(32b) Size (16b) Port(8b) QID(24b) Port(8b) QM/Schd LC Ingress: Functional Blocks Lookup Hdr Format Switch Tx S W I T C H Phy Int Rx Key Extract • QM/Scheduler (See Sailesh’s slides for more details) • Function: • Enqueue and Dequeue from queues • Scheduling algorithm • Drop Policy • Memory Accesses: • DRAM: None • SRAM: • Q-Array Reads and Writes • Scheduling Data Structure Reads and Writes • QLength Data Structure Reads and Writes • Dequeue: Read Buffer Descriptor to retrieve Packet Size • Buffer Descriptor Accesses: Read packet size • Monitoring: • Queue Lengths • Drops • Notes:

  42. Buf Handle(32b) Port(8b) QM/Schd LC Ingress: Functional Blocks Lookup Hdr Format Switch Tx S W I T C H Phy Int Rx Key Extract TBUF • Switch TX: • Function: • Coordinate transfer of packets from DRAM to TBUF • Memory Accesses: • SRAM: Read Buffer Descriptor • DRAM: Transfer to TBUF • Buffer Descriptor Accesses: • Read Size and Offset • Monitoring: • Per Physical Interface • Pkt Counter • Byte Counter • Notes: • Calculate DRAM address based on SRAM Descriptor address in buffer handle

  43. Buffer Descriptor Buffer_Next Buffer_Size Offset Free_List Packet_Size MR_ID TxMI QM/Schd VLAN Packet_Next LC Egress: Functional Blocks Phy Int Tx Lookup Hdr Format S W I T C H Rate Monitor Key Extract Switch Rx Buf Handle (32b) RBUF CRC (32b) • Rx: • Function: • Coordinate transfer of packets from RBUF to DRAM • Memory Accesses: • SRAM: Write Buffer Descriptor • DRAM: Transfer from RBUF • Buffer Descriptor Accesses: • Write/Initialize: Buffer_Next, Buffer_Size, Offset, Free_List, Packet_Size • Notes: • Buffer Handle: • contains the SRAM address of the buffer descriptor. • from the SRAM address of the descriptor we can calculate the DRAM address of the buffer data. • Passing the offset of where the packet starts in memory will save the next block from having to read the buffer descriptor. • Perhaps we should just pass the actual DRAM Buffer Pointer?

  44. QM/Schd LC Egress: Functional Blocks Phy Int Tx Lookup Hdr Format S W I T C H Rate Monitor Key Extract Switch Rx Buf Handle(32b) Buf Handle (32b) VLAN (16b) Lookup Key TxMI (16b) • Key_Extract: • Function: • Extracts lookup key based on type of frame that was received. • Memory Accesses: • DRAM: • Read VLAN and TxMI from Frame • SRAM: None • Buffer Descriptor Accesses: None • Notes: • Calculates DRAM Address based on SRAM descriptor address in buffer handle and the Offset passed to it by RX.

  45. QM/Schd LC Egress: Functional Blocks Phy Int Tx Lookup Hdr Format S W I T C H Rate Monitor Key Extract Switch Rx Buf Handle(32b) Buf Handle(32b) VLAN (16b) VLAN (16b) TxMI (16b) TxMI (16b) • Rate Monitor: • Function: • Ensures that MR/MI’s behave according to their Rate Specs. • Does this need to be a separate function from the QM/Scheduler? • Memory Accesses: Unknown at this point • DRAM: • SRAM: • Buffer Descriptor Accesses: Unknown at this point • Notes:

  46. QM/Schd LC Egress: Functional Blocks Phy Int Tx Lookup Hdr Format S W I T C H Rate Monitor Key Extract Switch Rx Buf Handle (32b) Buf Handle(32b) Lookup Result (20B) VLAN (16b) TxMI (16b) • Lookup: • Function: • Performs Lookup and passes result on to Hdr Format. • Memory Accesses: • DRAM: None • SRAM: • TCAM Lookup • Write Lookup command • Read Lookup Result (1-5 words from TCAM SRAM Controller or other SRAM Controller) • Buffer Descriptor Accesses: None • Notes: • Lookup does no processing on the lookup result. • Need to decide how lookup result will be stored and retrieved. • See notes on TCAM for information about the issues involved.

  47. Buffer Descriptor Buffer_Next Buffer_Size Offset Free_List Packet_Size MR_ID TxMI QM/Schd VLAN Packet_Next LC Egress: Functional Blocks Phy Int Tx Lookup Hdr Format S W I T C H Rate Monitor Key Extract Switch Rx Buf Handle(32b) Buf Handle (32b) Size (16b) Lookup Result (16B) QID(24b) Port(8b) • Hdr Format: • Function: • From lookup result: • re-writes headers in DRAM to make frame ready to transmit. • Extract QID to pass on to QM/Scheduler • Memory Accesses: • DRAM: • SRAM: • Read Descriptor and Re-Write Descriptor • OR • Atomic Increment/Decrement some fields in Descriptor • Buffer Descriptor Accesses: • Update Size and Offset fields • Notes: • Pass Size on to QM/Scheduler so it does not have to read buffer descriptor for Enqueue to update Q Length.

  48. Buffer Descriptor Buffer_Next Buffer_Size Offset Free_List Packet_Size MR_ID TxMI QM/Schd VLAN Packet_Next LC Egress: Functional Blocks Phy Int Tx Lookup Hdr Format S W I T C H Rate Monitor Key Extract Switch Rx Buf Handle(32b) Buf Handle(32b) Port(8b) Size (16b) QID(24b) Port(8b) • QM/Scheduler (See Sailesh’s slides for more details) • Function: • Enqueue and Dequeue from queues • Scheduling algorithm • Drop Policy • Memory Accesses: • DRAM: None • SRAM: • Q-Array Reads and Writes • Scheduling Data Structure Reads and Writes • QLength Data Structure Reads and Writes • Dequeue: Read Buffer Descriptor to retrieve Packet Size • Buffer Descriptor Accesses: Read packet size • Notes:

  49. Buf Handle(32b) Buffer Descriptor Port(8b) Buffer_Next Buffer_Size Offset Free_List Packet_Size MR_ID TxMI QM/Schd VLAN Packet_Next LC Egress: Functional Blocks Phy Int Tx Lookup Hdr Format S W I T C H Rate Monitor Key Extract Switch Rx TBUF • Switch TX: • Function: • Coordinate transfer of packets from DRAM to TBUF • Memory Accesses: • SRAM: Read Buffer Descriptor • DRAM: Transfer to TBUF • Buffer Descriptor Accesses: • Read Size and Offset • Notes: • Calculate DRAM address based on SRAM Descriptor address in buffer handle

  50. QM/Schd ML Loopback: Functional Blocks Lookup Hdr Format Switch Tx S W I T C H Phy Int Rx Key Extract Loopback Hdr Re-Format Phy Int Tx QM/Schd Hdr Format Lookup Rate Monitor Key Extract Switch Rx • Loopback path should be able to re-use some of the blocks implemented for the LC • Loopback Hdr Re-Format: • Needs to be able to strip off the previous MR’s Header. • For Plain-IP  IPv4 MR  MR Y • When frame arrives at Loopback (between IPv4 MR and MR Y) it will still have IP Header which should be stripped off before frame is sent to MR Y. • Lookup Result should probably include a length field or a Buffer offset field that indicates where new Meta Frame should start.

More Related