Pipelined van Emde Boas Tree: Algorithms, Analysis, and Applications

1. Pipelined van Emde Boas Tree: Algorithms, Analysis, and Applications Hao Wang and Bill Lin University of California, San Diego

2. Introduction • Priority queues used many network applications • Per-flow weighted fair queueing • Management of per-flow packet buffers using DRAM • Maintenance of exact statistics counters for real-time network measurements • Items in priority queue sorted at all times(e.g. smallest key first) • Challenge: Need to operate at high speeds (e.g. 10+ Gb/s)

3. Introduction • Binary heap common data structure for priority queues • O(lg n) time complexity, where n is # items • e.g. in fine-grained per-flow weighted fair queueing, ncan be very large (e.g. 1 million) • O(lg n) may be too slow for high line rates • Pipeline heaps[Bhagwan, Lin 2000][Ioannou, Katevenis 2001] • Reduced time complexity to O(1) • At the expense of O(lg n) pipeline stages

4. This Talk • Instead of pipelining binary heaps, we present new approach based on pipelining van Emde Boas trees • van Emde Boas (vEB) trees introduced in 1975 • Instead of maintaining a priority queue of sorted items, maintain a sorted dictionary of keys • In many applications, since keys are represented by a w-bit integer, possible keys can only be from fixed universeofu = 2wvalues • Only O(lg lg u) complexity vs. O(lg n) for heaps • Main result: pipelined vEB with O(1) operation andO(lg lg u) pipeline stages

5. van Emde Boas (vEB) Trees • Goal: Maintain a sorted subset S from universeU = {0, 1, …, u – 1} of size u = 2w, subject to INSERT, DELETE, EXTRACTMIN, SUCCESSOR, PREDECESSOR. e.g. • INSERT – Inserts new item into the queue • EXTRACTMIN– Removes item with smallest key

6. vEB Trees • Size u vs. n depends on application. e.g. • If u is only polynomial in n, i.e. u = O(nc), thenO(lg lg u) = O(lg lg n), exponential speedup over O(lg n) Application w u = 2w n compare O(lg lg u) O(lg n) Per-flow Fair Queues 24 bits 4 × 106 106 flows u≈ n 5 20 Statistics Counters 8 bits 512 106 counters 512 groups u« n u ≈ n 3 20 8

7. vEB Trees w/2 bits H and L[0], …,L[ – 1] recursivelydefined as vEB treeswith w/2 bits H w/2 bits . . . L[0] L[1] L[ -1] sub-trees • Conceptually think of the universe of w-bits,U = {0, 1, …, 2w – 1}, as a binary tree with height w • Think of top part H of w/2bits,and bottom part of sub-trees ofw/2bits, L[0], L[1], …, L[ – 1]

8. vEB Trees w/2 bits H w/2 bits . . . L[0] L[1] L[ -1] sub-trees • Suppose w = 8 bits, consider e.g. a key x = 31 • Split x into high and low parts: • Consider INSERT( x, S ) • If xh H, recursively call • INSERT( xl , L[xh]) • Else, recursively call • INSERT( xh , H ) • INSERT( xl , L[xh]) • Can avoid 2nd recursionby storing min[S] 0 0 0 1 1 1 1 1 xh xl

9. Representing vEB Trees • H L[0] L[1] L[ – 1] • min[S] : minimum key in S • n[S] : number of elements in S • H, L’s : just pointers to corresponding vEB sub-trees min[S] n[S]

10. INSERT Operation // if S empty set min to x // if x is smaller swap them // recursive call to eitherL[xh] or H // increment size of S Only One Recursive Call

11. INSERT Operation Supposew = 8 x = INSERT( x, S ) • If xh H, recursively call • INSERT( xl , L[xh]) • Else, recursively call • INSERT( xh , H ) • min( L[xh]) ←xl 0 0 0 1 1 1 1 1 xh xl H 4 bits 4 bits . . . L[0] L[1] L[255]

12. INSERT Operation Supposew = 8 x = INSERT( x, S ) • If xh H, recursively call • INSERT( xl , L[xh]) • Else, recursively call • INSERT( xh , H ) • min( L[xh]) ←xl 0 0 0 1 1 1 1 1 xh xl H 4 bits 4 bits . . . L[0] L[1] L[255]

13. INSERT Operation Noww = 4 x = 1 1 1 1 xh xl INSERT( x, S ) • If xh H, recursively call • INSERT( xl , L[xh]) • Else, recursively call • INSERT( xh , H ) • min( L[xh]) ←xl L[1] 4 bits

14. INSERT Operation Noww = 4 x = 1 1 1 1 xh xl INSERT( x, S ) • If xh H, recursively call • INSERT( xl , L[xh]) • Else, recursively call • INSERT( xh , H ) • min( L[xh]) ←xl H 2 bits 2 bits . . . L[0] L[1] L[255]

15. INSERT Operation Noww = 4 x = 1 1 1 1 xh xl INSERT( x, S ) • If xh H, recursively call • INSERT( xl , L[xh]) • Else, recursively call • INSERT( xh , H ) • min( L[xh]) ←xl H 2 bits 2 bits . . . L[0] L[1] L[255]

16. INSERT Operation Noww = 4 x = 1 1 1 1 xh xl INSERT( x, S ) • If xh H, recursively call • INSERT( xl , L[xh]) • Else, recursively call • INSERT( xh , H ) • min( L[xh]) ←xl H 2 bits 2 bits . . . L[0] L[1] L[255] Overall O(lg w) or O(lg lg u) time

17. EXTRACTMIN Operation // if S empty, nothing to return // set return to current min // recursive call to either H or L[mh] // decrement size of S // set new minimum and return value

18. EXTRACTMIN Operation // if S empty, nothing to return // set return to current min // recursive call to either H or L[mh] // decrement size of S // set new minimum and return value Again Only One Recursive Call

19. Basic Idea of Pipelining H w/2 bits w/2 bits . . . L[0] L[1] L[ -1] sub-trees • vEB operations are recursively defined • Each operation only makes one recursive call at each level of recursion • Each call goes down to a sub-tree with universe defined by w/2 bits (then w/4 bits,w/8 bits, etc) • Therefore, intuitively canunroll into lg lg u steps

20. Pipeline Structure min min min min n n n n H H H H L0 L0 L0 L0 L1 L1 L1 L1 … … … … input output op1 arg1 stage 1 M1 = pointers to all subtrees of width w/2 op2 arg2 stage 2 M2 = pointers to all subtrees of width w/4 op3 arg3 stage 3 M3 = pointers to all subtrees of width w/8 op4 arg4 stage 4 M4 = pointers to all subtrees of width w/16 : :

21. Main Issues • Want to initiate new operation at every pipeline cycle(e.g. initiate an INSERT or EXTRACTMIN) • Even if throughput is O(1), • Don’t want to wait lg lg u pipeline cycles to retrieve data (e.g. want data in same pipeline cycle for EXTRACTMIN) • But previous INSERTand EXTRACTMIN operation(s) may still be in pipeline • Two pipeline stages may need to access same data • Need to resolve memory access orders

22. Basic Idea min min min min n n n n H H H H L0 L0 L0 L0 L1 L1 L1 L1 … … … … • Operations proceed from top to bottom • min[S] for the vEB tree rooted at that level is updated immediately as operations flow thru pipeline • e.g. INSERT at Stage 1 will update min[S] immediately so next EXTRACTMIN at Stage 1 will retrieve correct min[S] input output op1 arg1 stage 1 op2 arg2 stage 2 op3 arg3 stage 3 op4 arg4 stage 4 : :

23. Other Operations • In addition to INSERTand EXTRACTMIN, paper describes • SUCCESSOR( x, S ) – Returns next element in S larger than x in the universe U • Can also define in a similar pipeline fashion • PREDECESSOR( x, S ) – Returns next element in Ssmaller than x in the universe U • EXTRACTMAX( S )– Removes item with largest key • To facilitate these operations, we also • Store max[S] for corresponding S at each pipeline stage • Provide another pipeline memory structure for tracking “in-flight” SUCCESSOR and PREDECESSOR operations

24. Conclusions • Pipelined van Emde Boas trees can achieve O(1) operations with O(lg lg u) pipelined stages • Same O(1) time complexity as pipelined heaps, but exponentially fewer pipeline stages than O(lg n) required for pipelined heaps when u is a polynomial of n • Can simultaneously support EXTRACTMIN and EXTRACTMAX, which is harder to do with heaps • Can support other operations like SUCCESSOR and PREDECESSOR, which have potential application to various network problem instances

25. Thank You