1 / 8

Making queues a first class citizen : Abstracting queues in the UApi

This document discusses the problem statement of ethtool using queue_id bind in the UApi for NIC specific queues, and proposes two high-level solutions for abstracting queues to make them a first class citizen in the UApi.

dballard
Download Presentation

Making queues a first class citizen : Abstracting queues in the UApi

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Making queues a first class citizen : Abstracting queues in the UApi Magnus Karlsson & Björn Töpel, Intel

  2. Problem Statement Ethtool using queue_id bind(af_xdp_fd, netdev, queue_id) UAPI NIC specific queues AF_XDP Tx & Rx queues Netdev queues XDP Tx queues • Drivers decide what queue ids to use for what. No standard • Only possible to use pre-configured queues in the driver. No dynamic creation • XDP and AF_XDP Tx queues invisible to user space How to get rid of HW queue ids in user space and still be able to create an AF_XDP socket and direct traffic to it?

  3. Basic Requirements Rx: Create a dedicated Rx queue and steer traffic to it • Bridge the gap with a getsockopt to get queue ids for use in ethtool • Tx: Multiple users (XDP Tx and AF_XDP Tx) but no traffic steering • Traffic shaping and QoS can be present though • Requirements: • UAPI: create an AF_XDP socket without specifying a HW queue id • Kernel: common module/functionality for allocating, freeing and naming/enumeration of queues • Drivers: No change to drivers that do not use this feature • UAPI: configuring traffic-steering and QoS to and from this socket • By NIC HW steering or XDP programs Will focus on 1 – 3 only in this presentation

  4. High-Level Proposal: Option 1 – Hide Queues Ethtool using fd (or opaque queue_id) fd = bind(af_xdp_fd, netdev) UAPI • Driver registers queues with QM (non-netdev vs all?) • Components that need queues can alloc, free and lookup in QM • QM uses new ndo to perform allocs and frees in the driver Xsk Ethtool XDP Queue module (QM) Alloc, free & lookup fd -> HW qid Register, alloc& free Driver AF_XDP Tx & Rx queues XDP Tx queues

  5. High-Level Proposal: Option 2 – Expose Queues Devlink to expose and manipulate queues fd = bind(af_xdp_fd, netdev, queue_id) UAPI • Expose queues and required info to user space to make a decision • New queues can be created through ethtool or devlink, if needed • Feed the desired queue id into bind • Flow objects could be added to control steering Xsk Devlink XDP Queue module (QM) Alloc, free & lookup fd -> HW qid Register, alloc& free Driver AF_XDP Tx & Rx queues XDP Tx queues

  6. Reflections / Questions • Is the queue manager uneccesary and a subdev should be used? • Switchdev / tc offload to steer traffic • Extend __rx and __tx structs to implement a queue manager? • Socket, abstracted queue id or extended queue id in user space? • Should netdev use the queue manager too? • XDP Tx and AF_XDP Tx two separate paths. Unify? • AF_XDP ToDo list on xdp-project repo • Need infra in user space to list queues

  7. Feedback from the Meeting • Option 2 preferred • Provides advanced configuration and control • Simplified user interface can be provided with libbpf • Start with the queue object, then the flow object • Use devlink to expose and manipulate queues and flows • Keep the current interface to bind() • Queue id can be treated as an opaque

More Related