Substrate Control: Common Type Definitions

1. Substrate Control:Common Type Definitions Fred Kuhns fredk@arl.wustl.edu Applied Research Laboratory Washington University in St. Louis

2. Simple Types • bw_t: unsigned 32-bit; Bandwidth units are Kb/s • cnt_t : unsigned 32-bit; item count or system counter value • len_t : unsigned 16-bit count • tStamp_t : unsigned 32-bit time stamp. • dwX_t: Defines a generic X-Byte data word of integral type • Assume two’s complement representation of integer values, NBO on the wire. • plabID_t : unsigned 32-bit, globally unique slice ID assigned by PLC • name_t { len_t len; uchar[len] } • string representation on the wire • fpID_t: unsigned 16-bit, Fast path identifier (fpID) • Slice fpid range [0, N) • Substrate range [1, M) (0 represents the global namespace) • ctxID_t : unsigned 16-bit, Message context ID. • 0 used for substrate/ privileged context; otherwise fpid • retCode_t: signed 32-bit return code returned by server methods. • 0 must always represent success (see wuspp::retCode_t)

3. Simple Types • qid_t: unsigned 16-bit, Slice relative qid • Ranges: Slice [0, N); Substrate [1, M) • fid_t: unsigned 32-bit; filter identifier • Range: Slice and Substrate [0, N) Address/Network related • ipAddr_t: unsigned 32-bit; IP address • always in network byte order (NBO) • ipPort_t : unsigned 16-bit; UDP/TCP Port number or ICMP ID • always in NBO • ipProto_t : unsigned 8-bit; IP header protocol field • TCP=6,UDP=17, ICMP=1 • hwaddr_t : array of 6 uint8_t; Ethernet MAC Address • vlanID_t: unsigned 16-bit; VLAN identifier

4. Interfaces, Meta-Interfaces and Endpoints • ifn_t : unsigned 16-bit; Logical interface number (aka link id). • no correspondence to physical interface number; used as interface index in NPE TCAM filters • miID_t: unsigned 16-bit; fast path relative Meta-Interface number • Ranges: Slice relative only, [0, N) • ifType_t: unsigned 16-bit; Interface type • 0 = Public Internet; 1 = SPP Peering Endpoints • struct epoint_t {ipAddr_t; ipPort_t; ipProto_t}; • struct epInfo_t { bw_t bw; epoint_t ep; } • struct ifAttr_t {ifn_t ifn; ifType_t type; ipAddr_t ipaddr; bw_t link; bw_t avail;} • if_list = {ifAttr_t, ...}

5. FastPath Related Types Specify fastpath NP resources • fpParams_t {copt_t, bwspec_t, rcnts_t, mem_t} copt_t: unsigned 16-bit code option identifier • Defined code options {Invalid = 0, IPv4 = 1, i3 = 2} struct bwspec_t { bw_t firm; bw_t soft;} • Aggregate BW requirements for fastpath across all interfaces • bw-firm: Expresses the required aggregate processing resources in terms of BW, units are Kbps. • bw-soft: A hint as to the expected max required processing resources. Resources are not actually allocated to the fast-path, rather the system uses this number for load balancing best-effort fast-paths across available NPEs. struct rcnts_t { cnt_t fltrs; cnt_t queues; cnt_t buffers; cnt_t stats;} • Max resource counts struct mem_t { cnt_t sram; cnt_t dram;} • Requested size in Bytes (may be fixed by code option) Data describing an instantiated fastpath’s NP resources, returned to slice • struct fpInfo_t { fpID_t fpid; // Slice relative fast-path id, starts at 0ipAddr_t npeIP; // FP addr on the NPEepoint_t ldAddr; // Local Delivery UDP Portepoint_t exAddr;} // Exception traffic UDP Port

6. Reservations • rdate_t { date_t start; date_t end; } • date_t := “YYYYMMDDHHMMSS” • Always 14 Bytes • Not null terminated on wire Interface BW requirements • ifParams_t { cnt_t, ifRec_t[] } • ifRec_t {bw_t, ipAddr_t} Reservation Records • fpRSpec_t { fpParams_t, ifParams_t, name_t fpName } • Defines resources required for one fastpath instance • plRSpec_t { ifParams_t } • rsvRecord_t { rdate_t; cnt_t fpCnt; fpRSpec_t[fpCnt]; plRSpec;} • List all resource allocations for a given interval of time.

7. Filters • typedef dw1_t fltrKey_t[COPT_KEY_SIZE]; // 14 Bytes • struct fpKeyWrap_t { uchar type; // 0 = Normal, 1 = substrate only (i.e. tunnel)miID_t rxmi; // Receiving meta-interface IDfltrKey_t coptKey; // Code option specific lookup key} • struct fpFltrResult_t { dw1_t actions; // 0 forward, 1 local, 2 drop, 3 drop and local??ipAddr_t daddr; // UDP tunnel’s destination IP ipPort_t dport; // UDP tunnel’s destination portqid_t qid; // Slice relative queue IDstatID_t sindx;}; // stats index • struct fpFltr_t { fpKeyWrap_t keyWrap; fltrKey_t coptMask; fpFltrResult_t result; }

8. Misc Types • sid_t: unsigned 16-bit; (Absolute) Scheduler Id • There are a total of 20 schedulers split across 4 queue managers • range [0, 20) • 5 schedulers per QM; 4 QMs • Queue Manager ID = qmID_t = sid / 5 • QM’s scheduler ID = schedID_t = sid % 5 • qmID_t: unsigned Byte; 2-bit QM identifier • schID_t: unsigned Byte; 3-bit qm relative scheduler identifier • sched_map_t { qmID_t qmID; schID_t schID; qid_t qid; statID_t sindx; vlanID_t vid;}

9. !TCP 00 RSV proto TCP 01 00 flags 2 2 12 IPv4 Filter Formats (Slice’s view) • RMP translations (see next page): • rx miID to ‘rxPort’ and input IP address • result qid to ingress meta-interface ID • SCD translations: • input IP address to ifindx • Slice relative IDs to global IDs (qid, sindx, fid) • fastpath ID (message context ID) to vlan • TX miID to corresponding scheduler and QM IDs Defined by the IPv4 Code Option, 14 Bytes Substrate rx miID type daddr saddr sport dport tcp/proto 8 16 32 32 16 16 16 Type: 0 => Normal Lookup 1 => Substrate only lookup 2 6 8 Result TX IP daddr TX dport qid D L sindx 8 32 16 16 16 actions Destination Address of UDP egress tunnel D: Drop packet L: Local delivery

10. !TCP 00 RSV proto TCP 01 00 flags 2 2 12 IPv4 TCAM Filter Formats (on NPE) Defined by the IPv4 Code Option, 112bits Substrate defined T if vlan RX port daddr saddr sport dport tcp/proto 1 4 11 16 32 32 16 16 16 Represents input meta-interface 2 6 8 T = 0: Normal Lookup T = 1; substrate only lookup Result, 128 bits sindx TX IP daddr TX dport TX sport rsv QM Sch qid rsv D L rsv 16 32 16 16 12 2 3 15 3 1 1 11 global stats index (SCD maps slice’s sindx to global value) TX sport represents the output meta-interface. The TX IP addres and dport is provided by the slice. (RMP maps miid to tx tunnel params, use dport provided by slice) 20-bit internal qid (SCD maps slice’s miid to QM and Sch. SCD Also maps slice’s qid to global qid value) D: Drop packet L: Local delivery Slice parameters: Key: Input miid, IPv4 fltr {daddr, saddr, sport, dport, tcp/proto} Result: Flags {Drop, GPE}, sindx, Output miID, QID

11. X X X X X T L W Flags: W : {0 - Bytes, 1 - Packts} L : {0 - PreQ, 1 - PostQ} T : {0 - Push, 1 - Pull} X : Don’t Care 7 6 5 4 3 2 1 0 Stats (accessing FP instance specific counters) • struct statRec_t { cnt_t cnt; tStamp_t tstamp;}; • flags : dw1_t, bit vector. • see figure to the right • handle : unsigned 32-bit • opaque reference to periodic event • statID_t: unsigned 32-bit; Stats Index • Ranges: Slice [0, N), Substrate [0, M)