Priority Queues

1. Priority Queues For all the important things in life!

2. Priority Queue • Useful for assigning “priorities” to shared resources. • Operating System: scheduler can use a priority queue to determine which task to execute next. Need not be executed in the order that the tasks were inserted. • Shared Printers: scheduler can print small jobs before big ones. This increases the average delay for output.

3. remove() returns the highest priority item insert(priority, data) Priority Queue Priority Queue • A collection of data that is accessed by “priority” • The data is “ordered” only by it’s priority • Multiple identical priorities are allowed • Supports the following fundamental methods • void insert(Comparable priority, Object data) • Inserts data into the queue using the specified priority • Object remove() • Removes and returns the data having the “greatest” priority • An error occurs if the queue is empty. • Object minElement() • Returns the data having the greatest priority but doesn’t remove it from the queue. • An error occurs if the queue is empty. • Object minKey() • Returns the key having the greatest priority.

4. Priority Queue ADT Method Output Queue insert(5, A) none ((5,A)) insert(9, C) none ((5,A), (9,C)) insert(4, B) none ((5,A), (9,C), (4, B)) insert(5, D) none ((5,A), (9,C), (4, B), (5, D)) minElement() B ((5,A), (9,C), (4, B), (5, D)) minElement() B ((5,A), (9,C), (4, B), (5, D)) remove() B ((5,A), (9,C), (5,D)) remove() A or D ((9,C), (5,D)) or ((5,A), (9,C)) remove() A or D ((9,C))

5. Simple Priority Queue Implementations • Use an un-ordered array of values • Similar to a Vector • What is the run time of insert/remove? • Maintain an ordered array of values • Ordered by priority (or key) • What is the run time of insert/remove?

6. Heaps • A binary-tree-based implementation of a priority queue • A complete binary tree with a heap property For every node V of the heap, the priority (key) stored at V is greater than or equal to the priority (key) of any of V’s children. “Greater priority” often means “lower key value”.

7. 4 4 4 18 9 18 9 4 4 28 19 11 28 11 19 4 4 4 Heaps Nope! (not a complete binary tree) Yes Indeed! Yes Indeed!

8. 4 5 6 15 9 7 20 16 25 14 12 11 13 2 Heap Insertion Example How to insert the key “2” into the heap?

9. Heap Insertion (UpHeapBubble) 4 5 6 15 9 7 2 16 25 14 12 11 13 20 Restore the heap ordering property by “bubbling” the new item into its proper location. If the parent is bigger than the bubbling item, then swap the parent with the “bubbler”.

10. Heap Insertion (UpHeapBubble) 4 5 2 15 9 7 6 16 25 14 12 11 13 20 The “up heap bubbling” restores the heap property. A node is only replaced by a “smaller” node. Therefore all the nodes children are still greater-than-or-equal to the new node value.

11. Heap Insertion (UpHeapBubble) 2 5 4 15 9 7 6 16 25 14 12 11 13 20 A Heap!

12. Heap Removal Example 2 3 4 15 9 7 6 16 25 14 12 11 13 20 How to remove an item from the heap?

13. Heap Removal Example 2 ? 3 4 15 9 7 6 16 25 14 12 11 13 20 The item to remove is always at the root and is found in O(1). How to restore the heap?

14. Heap Removal Example 20 3 4 15 9 7 6 16 25 14 12 11 13 20 Remove the “last” item in the “last” row and place it at the root. How to restore the heap ordering property?

15. Heap Removal (DownHeapBubble) 3 20 4 15 9 7 6 16 25 14 12 11 13 Perform a “down heap bubble”! If either child is smaller than the bubbler, recursively swap the bubbler with its smallest child.

16. Heap Removal Example 3 9 4 15 20 7 6 16 25 14 12 11 13

17. Heap Removal Example 3 9 4 15 12 7 6 16 25 14 20 11 13 A Heap!

18. 6 12 11 15 Example Perform the following operations on an initially empty heap: insert(3) insert(12) insert(2) remove() insert(3) insert(15) insert(6) insert(11) insert(2) insert(2) remove() remove() remove() remove()

19. Heap Question • Does a heap guarantee the first-in-first-out principle is applied to identically-valued keys? • Consider the following sequence: • insert(1) • insert(2) • insert(2) • remove() • remove() !

20. Heap Question Given a linked heap representation, find the location of the next insertion point! algorithm nextNode(TreeNode t) Input: Root of a binary tree t. Output: None. New node is inserted into t if t == null t = new TreeNode() else if height(t.left) != height(t.right) if t.left is full t.right = nextLocation(t.right) else t.left = nextLocation(t.left) else if t.right is full t.left = nextLocation(t.left) else t.right = nextLocation(t.right) return t;

21. Priority Queue Performance Method Sorted Array Unsorted Array Heap insert O(N) O(1) O(Log N) remove O(1) O(N) O(Log N) minElement O(1) O(N) O(1) minKey O(1) O(N) O(1)

22. Heap Implementation • Which way to implement heaps? • sequential (array based) binary tree representation? • linked binary tree representation? • The sequential representation is probably the “best”! • Since a heap is a complete binary tree, all nodes are in contiguous array locations. • No “wasted” slots. Oooh!

23. 0 1 2 3 4 5 6 7 8 9 10 11 n 3 9 5 12 11 7 9 14 13 n n Example Given the following “heap”, perform the listed operations: insert(4) insert(1) remove() remove()

24. Sorting Digression • How can we use a Priority Queue to sort? Algorithm queueSort(Vector v) Input: Vector v of comparable values to sort. Output: None. Vector v is sorted. Let Q be a Priority Queue while v is not empty Data = v.remove(0) Q.insert(Data, Data) while Q is not empty v.add(Q.remove())

25. Sorting Digression • What is the run-time of queueSort if we use an unsorted-array-based queue? • N insertions, each of which is O(1) • N removals, each of which is O(n) • Phase 1 is actually N*O(1)or O(N) • Phase 2 is actually O(N) + O(N-1) + … + O(2) + O(1) • which can be written O(Sum[i, {i,1,N}]) or O(N2) • This is also known as “selection sort”

26. Sorting Digression • What is the run-time of queueSort if we use a sorted-array-based queue? • N insertions, each of which is O(n) • N removals, each of which is O(1) • Phase 1 is actually O(N) + O(N-1) + … + O(2) + O(1) • Phase 2 is actually N*O(1) or O(N) • This is also known as “insertion sort”

27. Sorting Digression • What is the run-time of queueSort if we use a heap? • N insertions, each of which is O(Log N) • N removals, each of which is O(Log N) • Phase 1 is actually: • O(Log 1)+O(Log 2)+O(Log 3)+…+O(Log (N-1))+O(Log N) • Since each Log term (except the last) is less than O(Log N) we can say that Phase 1 is O(N Log N) • Phase 2 is also O(N Log N) by similar reasoning. Therefore this algorithm is O(N Log N) • This is also known as “heap sort”

28. Building A Heap Goal: How can we quickly construct a heap given an unordered array?

29. Bottom-Up Heap Construction • build (n + 1)/2 trivial one-element heaps • now build three-element heaps on top of them

30. Bottom-Up Heap Construction • downheap to preserve the order property • now form seven-element heaps

31. Bottom-Up Heap Construction (cont.)

32. Bottom-Up Heap Construction (cont.)

33. Original Array Final Array Build Heap Trace 1 2 3 4 5 6 7 8 9 10 -------------------------------------- 40 12 89 34 16 78 65 21 11 09 40 12 89 34 09 78 65 21 11 16 40 12 89 11 09 78 65 21 34 16 40 12 65 11 09 78 89 21 34 16 40 09 65 11 12 78 89 21 34 16 09 11 65 21 12 78 89 40 34 16

34. Gasp! You can’t be serious! Build Heap Performance • Consider: • The buildHeap method does exactly (n/2)bubbleDowns. • Each bubbleDown is O(h) • h is O(Log n) • buildHeap method is therefore O (n/2 * Log n) or O(n Log n) • But Wait: • O(n Log n) is not a “tight upper bound” just an “upper bound” • The buildHeap method is actually Linear!

35. Build Heap Performance • Now Consider: • The buildHeap method does exactly (n/2) bubbleDowns. • Each bubbleDown is O(h) where h is the height of each sub-heap! • h is O(Log n) where n is the size of each sub-heap • Find the height of each internal node of a complete binary tree to determine the build heap performance. build Heap is therefore O(n)!

36. Analysis of Bottom-Up Heap Construction • Proposition: Bottom-up heap construction with n keys takes O(n) time. • Insert (n + 1)/2 nodes • Insert (n + 1)/4 nodes and downheap them • Insert (n + 1)/8 nodes and downheap them • ... • visual analysis: n inserts, n/2 upheaps with total O(n) running time

37. Heaps: Complexity of building Sizes of the levels: n/4, n/8, n/16, ..., 1 where n = # nodes. Times for bubbling down node: at most 1, 2, 3, ..., (log n) - 1. So the total time is at most approximately: 1 ·n/4 + 2 ·n/8 + 3 ·n/16 + ... + (log n - 1) · 1 < n/4 · (1 + 2 · 1/2 + 3 · 1/4 + 4 · 1/8 + ...) Now, 1 + 2 · 1/2 + 3 · 1/4 + 4 · 1/8 + ... = 1 + 1/2 + 1/4 + 1/8 + ... + 1/2 + 1/4 + 1/8 + ... + 1/4 + 1/8 + ... + ... = 2 + 1 + 1/2 + 1/4 + ... = 4. So our heap building algorithm takes O(n) time.

38. Sorting Digression Revisited • What is the run-time of queueSort if we use a heap combined with buildHeap? • Execute buildHeap on the array which is O(N) • N removals, each of which is O(Log N) • Phase 1 is linear • Phase 2 is O(N Log N) • This algorithm is still O(N Log N) • Performance is slightly better that the insert/remove version

39. Ouch! My head is spinning! Huffman Code(a cool application of Priority Queues) • What is the huffman code? • A data compression scheme • Used in many compression formats (JPG, MPG, MP3, etc…) • Uses a variable-length encoding scheme (vs. fixed length) • Uses a prefix encoding scheme (no code is a prefix of another code) • Based on an analysis of the symbol frequencies in the input file • Utilizes a priority queue of binary trees to construct the translation table!

40. Fixed vs. Variable Length Codes Total Bits: Fixed: 3 bits * 100,000 = 300,000 bits Variable: 45k*1+13k*3+12k*3+16k*3+9k*4+5k*4 = 224,000 bits 76k bits / 300k bits is a 25% compression ratio

41. Fixed Length Codes Decode the following fragment: 001100000011 algorithm fixedLengthDecode(InputStream) for every 3-bit-byte B in the InputStream look up the symbol corresponding to B and print it out 001100000011 b e a d

42. Variable Length Codes Decode the following fragment: 10111010111 algorithm variableLengthDecode(InputStream) W is a vector of “BITS” for every bit B in the InputStream append B to W look up the symbol S corresponding to W print S if found and clear W 10111010111 b ea d

43. a c b d f e Variable Length Codes Imagine a binary tree where each leaf represents a symbol and the path from root to leaf represents its variable-length binary encoding! No path would be a prefix of another symbol! Decode the following fragment 10111010111 algorithm variableLengthDecode(InputStream) N = the root node of the “encoding tree” for every bit B in the InputStream if B is zero then N = left child of N else N = right child of N if N is a leaf then print the leaf value set N = root node of encoding tree

44. Huffman Tree • The huffman tree serves as the key to both encoding (compressing) and decoding (uncompressing) a file. How is the huffman tree constructed? algorithm Huffman(V) Input: A vector V of characters with frequency f Output: The huffman tree Let Q be a priority queue while V is not empty C = v.remove(0) f = frequency of C Q.insert(f, Binary tree rooted at C with no children) while Q contains more than 1 item f1 = Q.minKey() T1 = Q.remove() f2 = Q.minKey() T2 = Q.remove() Q.insert(f1+f2, Binary tree with left child of t1 and right child of t2) return Q.remove()

45. Huffman Compression Compression • Scan the input file and construct a frequency table • Construct a huffman encoding tree from the frequency table • Save the huffman tree to the output file • Scan the input file and for every symbol output its huffman representation (which will be 1 or more bits) to the output file Decompression • Read the huffman tree and then process every bit, outputting the proper symbol when a leaf node is encountered.

46. Unique identifier, often referred to as a “magic number” that indicates this file was generated by your compression program. The huffman tree (or enough information to re-construct the huffman tree) The encoded file. Note that since symbols are not necessarily 8-bits, the file may “end” on a non-byte boundary. This means that a spurious “EOF” symbol with frequency 1 should always be included in the output. Huffman Compression Most compressed files are structured as shown below