1 / 15

Managing SPP Resources: System Resource Manager (SRM)

Managing SPP Resources: System Resource Manager (SRM). Fred Kuhns fredk@arl.wustl.edu Applied Research Laboratory Washington University in St. Louis. Common Definitions and Types.

hova
Download Presentation

Managing SPP Resources: System Resource Manager (SRM)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Managing SPP Resources:System Resource Manager (SRM) Fred Kuhns fredk@arl.wustl.edu Applied Research Laboratory Washington University in St. Louis

  2. Common Definitions and Types • For all commands the message context ID is used to identify the context within which a command is to be executed. • The special value of 0 (cid = 0) indicates a privileged operation performed by the substrate. • Otherwise the context ID is an identifier indicating a user specific context in which the command is to be interpreted and executing. In most cases this is the Global Fastpath ID. • See presentation titled Types.ppt

  3. NATD Interface • [egress_map, ingress_map] get_sched_map(ipAddr_t link, hwaddr_t board) • Message context ID = 0. sched_map_t { qmID_t qmID; // 2-bit QM identifier schID_t schID; // 3-bit qm relative scheduler identifier qid_t qid; // 16-bit queue identifier statID_t sindx; // Stats Index vlanID_t vid; // VLAN identifier } Return: egress_map: sched_map_t ingress_map: sched_map_t Arguments: link: Interface IP address (aka external IP) board: Board/Device MAC Address

  4. Reserving Fastpath Resources See Types.ppt for type definition and mapping to substrate API, message context ID = 0. Reserve resources for 0..N fastpaths (on NPEs) and 0..1 GPEs • retCode_t make_reservation(plabID, rsvRecord_t) Update an existing reservation, note because all operations need to be idempotent the update argument includes all required resources. • retCode_t update_reservation(plabID_t, rsvRecord_t) Cancel any reservation overlapping date_t • retCode_t cancel_reservation(plabID_t, date_t) Return any reservation (belonging to slice) that overlaps with date_t • rsvRecord_t? get_reservation(plabID_t, date_t) Return a list of all reservations associated with calling slice • rsvRecord_t* get_reservations(plabID_t)

  5. Allocating Fast-path Instance • fastpath_t alloc_fastpath(plabID_t, copt, bwspec, rcnts, mem) • Message context ID = 0 plabID_t: GPE/PlanetLab slice identifier uint16_t copt: code option type. {IPv4=1,I3=2} struct bwspec {bw_t bw-firm, bw-soft;} struct rcnts {cnt_t fltrs, queues, buffers, stats;} struct mem {cnt_t sram, dram;}// Requested size of SRAM and DRAM in Bytes struct fastpath_t { fpid_t fpid;// fastpath id assigned by srm, used as context ID in future messagesvlan_t vlan;// VLAN id to associate with fp’s logical interfaceipAddr_t fpIP;// IP address of logical fp’s interface on GPEipAddr_t npeIP;// IP address for fp’s data interface on NPEipAddr_t fpNM;// Netmask of fp’s logical interfaceipAddr_t scdIP;// IP address for fp’s assigned SCDipPort_t scdPort;// UDP Port number for fp’s assigned SCDhwaddr_t npeMAC;}// NPE MAC address • bw-firm: Expresses the required aggregate processing resources in terms of BW, units are Kbps. • bw-soft: This is just a hint as to the expected maximum processing resources required. Resources are not actually allocated to the fastpath, rather the system uses this number for load balancing best-effort fastpaths across the available NPEs.. • Actions required to allocate code option instance and resources: • Select NPE • Load balance across available NPEs • Must have sufficient resources (SRAM, TCAM space, queues, etc) • Either allocates requested resources or returns error • Keeps memory map of SRAM so can perform allocation, though the absolute starting address is not required. • If compaction is necessary then must communicate with SCD directly. • Allocate VLAN and configure switch. • Send commands to SCD on NPE • set_fastpath(fpid, copt, VLAN, TParams, Mem[]) • set_sched_params(sid(gpe) GPE_MIID, GPE_BW_MAX*, GPE_BW_MIN) • set_fpmi_bw(fpid, sid(gpe), GPE_MIID, GPE_BW_MIN) • set_encap_gpe(fpid, gpeIP(fpid), npeIP(fpid)) Deprecated

  6. Releasing a fast-path instance • retCode_t free_fastpath() • Message context ID = Global fastpath ID (fpid) • Send command to SCD on NPE • set_sched_params(sid(gpe),GPE_MIID, GPE_BW_MAX*, GPE_BW_MIN) • retCode_t set_fpmi_bw(fpid, sid(gpe), GPE_MIID, 0) • set_encap_gpe(fpid, 0, 0) • rem_fastpath(fpid) Deprecated

  7. Instantiate Fastpath and/or Slowpath reservations • All resource allocation methods operate on the currently active reservation (i.e. must be used during the interval of time when a reservation is valid). • If free is not called then all resources are automatically freed when reservation expires. allocate any GPE/Slice resources reserved in the current interval • retCode_t alloc_plspec(plabID_t) instantiate the “current” fastpath reservation with name fpName • fpInfo_t alloc_fpspec(plabID_t, name_t fpName=“”) • if only one fastpath then the fpName parameter can be omitted. free any GPE specific resources currently allocated to slice. • retCode_t free_plspec(plabID_t) Destroys fastpath instance, releasing meta-interfaces and NPE resources. • retCode_t free_fpspec() • Message context ID set to the Global fastpath ID

  8. Managing Interfaces Reserve Aggregate Interface BW for slice (traffic to/from GPE). • retcode resrv_slice_ifbw(plabID, bw, ifn) Release slice interface BW reservation for GPE • retcode reles_slice_ifbw(plabID, bw, ifn) Reserve aggregate Interface BW for indicated fastpath • retcode resrv_fpath_ifbw(bw, ifn) • message context ID = fpid Release Aggregate Interface BW allocation for indicated fastpath • retcode reles_fpath_ifbw(bw, ifn) • message context ID = fpid Deprecated

  9. System Interfaces, Addresses and Ports Get list of available interfaces • if_list get_interfaces(plabID) • Message context ID = 0. • if_list = {ifAttr_t, ...} ifAttr_t { ifn_t ifn; // Interface number ifType_t type; // {Multi-Access = 0, Point-to-Point = 1} ipAddr_t ipaddr; // Interface/Link IP address ipAddr_t linkBW; // Raw (Gross) interface (link) BW bw_t availBW; // Available (Net) interface (link) BW } • if type = point-to-point then may call get_ifpeer() Map interface IP address to interface number. • ifn_t get_ifn(plabID_t, ipAddr_t) Get updated interface attributes • ifAttr_t get_ifattrs(plabID_t, ifn_t) Get the IP address of point-to-point link’s peer node (SPP peer). • ipAddr_t get_ifpeer(plabID_t, ifn_t)

  10. Managing Externally Visible Address for GPE Traffic • Message context ID set to 0, plabID used to identify Slice making request • Managing bandwidth: • SRM: verifies slice meta-interfaces and queues do not exceed interface allocations • SRM: Notifies SCD of per Scheduler slice limits Allocate a local endpoint for slice (linecard directs matching traffic to the corresponding GPE) • epInfo_t alloc_endpoint(plabID_t, epInfo_t) • Message context ID = 0. Release allocated endpoint (linecard filters removed and address added bac kto available pool of addresses tuples). • retCode_t free_endpoint(plabID_t, epoint_t) • Message context ID = 0

  11. Internal Interfaces for NAT map{bw_t bw; # Kbps ipAddr_t ipaddr; # local interface IP address ipPort_t port; # if TCP/UDP and 0 then allocate ipProto_t proto; # TCP/UDP,ICMP dw1_t smac}; # low order 8 bits of source mac • {fid, port} alloc_epmap(map) • Message context ID = 0. • Allocate endpoint and add LC TCAM filter to send pkts to GPE. • retCode_t free_epmap(fid) • Message context ID = 0. • Allocate endpoint and add LC TCAM filter to send pkts to GPE. Deprecated

  12. Allocating Meta-Interfaces (tunnel endpoint) • Common Types ep {uint32_t bw, uint32_t ipaddr; uint16_t port; uint8_t proto} dw2_t mi: Meta-interface number for an endpoint. Interface numbers start at 0. • [mi, ep] alloc_tunnel(bw, ipaddr, tunnel_type, params) Not implemented • [mi, ep] alloc_udp_tunnel(bw, ipaddr, port) Message context Id = fpid uint8_t tunnel_type : must be one of {UDP = 0} uint32_t bw: (Minimum guaranteed) BW assigned to endpoint, Kbps uint32_t ipaddr: External IP address of endpoint/tunnel. There is a 1-to-1 relationship between links and ip addresses. uint16_t port: External UDP port number • Actions required to allocate code option instance and resources: • verify requested address is available, assign port if necessary (port == 0) • Allocate new meta-interface ID • allocate and assign filter id for ingress and egress TCAM • Lookup existing or allocate qid for ingress and/or egress • Send command to SCD on Line Card Ingress • set_sched_params(sid(fabric,NPE), ifn(fabric), BW*, BWmin) • set_queue_sched(qid, sid(fabric,NPE)) • set_queue_params(qid, threshold, bw) • write_fltr(dbid_ingress, fid, key_ingress, mask, result_ingress) • write_fltr(dbid_egress, fid, key_egress, mask, result_egress) • Send commands to Line Card Egress • set_sched_params(sid(link), ifn(link), BW*, BWmin) • set_queue_sched(qid, sid(link)) • set_queue_params(qid, threshold, bw) • Send command to SCD on NPE • set_sched_params(sid(link, miid), ifn(fabric), BW*, BWmin) • set_fpmi_bw(fpid, sid(link, miid), miid, bw*) • ep get_endpoint(mi) Message context Id = fpid Deprecated

  13. Releasing Meta-Interfaces • Common Types ep {uint32_t bw, uint32_t ipaddr; uint16_t port; uint8_t proto} uint_t mi: Meta-interface number for an endpoint. Interface numbers start at 0. • retCode_t free_tunnel(ipaddr, tunnel_type, params)retCode_t free_udp_tunnel(ipaddr, port) • Message context Id = fpid. uint32_t ipaddr: External IP address of endpoint/tunnel uint16_t port: External UDP port number • Actions required to allocate code option instance and resources: • Send command to SCD on Line Card Ingress • set_sched_params(sid(fabric,NPE), ifn(fabric), BW*, BWmin) • set_queue_params(qid, threshold, bw) • rem_fltr_byfid(dbid_ingress, fid) • rem_fltr.fid(dbid_egress) • Send commands to Line Card Egress • set_sched_params(sid(link), ifn(link), BW*, BWmin) • set_queue_params(qid, threshold, bw) • Send command to SCD on NPE • set_sched_params(sid(link, miid), ifn(fabric), BW*, BWmin) • set_fpmi_bw(fpid, sid(link, miid), miid, bw*) Deprecated

  14. Allocating Meta-Interfaces (tunnel endpoint) --- Simplified interface, see steps from deprecated slides --- Allocate local Address for Slice’s fastpath (MI) and forward to NPE, will also reserve sufficient aggregate BW on corresponding internal and external interfaces. • [miID_t, epInfo_t] alloc_udp_tunnel(bw_t, ipAddr_t, ipPort_t) • Message context ID set to the global fastpath ID • It is an error if the sum of requested BW exceeds the reserved fastpath BW. • Adds one end of a UDP tunnel, treated as a meta-interface for a fast path instance. • if port = 0 then one is allocated from the free pool • Returns the meta-interface number representing the new endpoint Release BW and Address associated with corresponding meta-interface • retCode_t free_udp_tunnel(ipAddr_t, ipPort_t) Map meta-interface ID to corresponding externally visible address tuple • epInfo_t get_endpoint(miID_t) • mi must be a valid meta-interface ID returned from a call to alloc_udp_tunnel

  15. Interfaces VLAN maps System tables ifn:{type,ipaddr,linkBW,availBW} range:{start,end} ... free {...} endpoint (port) maps resvMap availMap usedMaps xsidMap GPE controlIP Slices BWmaps endpoint references servMap resvMap Per Slice Tables vlan xsid plabID NPE Table id:{addr,BW/Port,copts,fltrs,sram,Qs} ... NPE (allocated) meta-ifaces sram {start,size} #flts mi:endpoint ... board ID #Qs BW #Stats gpe board id BW plab sliceID Classes and Tables Slice Table xsid plabID Mappings: plabID to Entry xsid to Enry Entry: {plabID, xsid, gpe, npe, MI} GPE: {GPE object reference, BW, EPs, ??} NPE: {NPE object reference, vlan bw, sram, fltrs, qs, stats} MI: {EP object reference, ID}

More Related