CS 261 – Data Structures

1. CS 261 – Data Structures Priority Queues & Heaps

2. Priority Queues • Not really a FIFO queue – misnomer!! • Associates a “priority” with each object: • First element has the highest priority • A data structure that supports the PQueue interface: public void add(Object val); public Object getFirst(); public void removeFirst(); • Examples of priority queues: • To do list with priorities • Active processes in an OS

3. Use of Priority Queues in Simulations • The original, and one of the most important uses of Priority queues • Discrete Event Driven Simulation • Actions are represented by “events” - things that have (or will) happen at a given time • The PQ maintains list of pending events. Highest priority is next event. • Event is pulled from list, executed, usually spawns more events, which are then inserted into the PQ. • Loop until everything happens, or until fixed time is reached.

4. Priority Queue Applications • Discrete event-driven simulation – i.e., a simulation of a process where events occur • Example: Ice cream store • People arrive • People order • People leave • Simulation Algorithm: • Determine the times of each event using a random number generator with some distribution: • Put all events in a priority queue based on when it happens • Simulation framework pulls the minimum (next to happen) and executes the event

5. Priority Queue Applications • Many types of events … but they all have a time when they occur • Can we store various types of events in a priority queue? • Heterogeneous collection • Keep a pointer to action, or a flag to tell what action to perform • So, have your comparison based on the time

6. Priority Queues: Implementations We can definitely do better than these!!!

7. Priority Queues: Heaps • Heap: has 2 completely different meanings • Classic data structure used to implement priority queues • Memory space used for dynamic allocation We will study the data structure (not dynamic memory allocation) • Heap data structure: a complete binary tree in which every node’s value is less than or equal to the values of its children • Review: a complete binary tree is a tree in which • Every node has at most two children (binary) • The tree is entirely filled except for the bottom level which is filled from left to right (complete) • Longest path is ceiling(log n) for n nodes

8. Heap: Example Root = Smallest element 2 3 5 9 10 7 8 14 12 11 16 Next open spot Last filled position (not necessarily the last object added)

9. Maintaining the Heap: Addition 2 Add element: 4 3 5 9 10 7 8 New element in next open spot. 4 14 12 11 16 Place new element in next available position, then fix it by “percolating up”

10. Maintaining the Heap: Addition (cont.) 2 3 5 2 9 10 4 8 3 4 14 12 11 16 7 After first iteration (swapped with 7) 9 10 5 8 14 12 11 16 7 After second iteration (swapped with 5) New value not less than parent  Done Percolating up: while new value is less than parent, swap value with parent

11. Maintaining the Heap: Removal • Since each node’s value is less than or equal to the values of its children, the root is always the smallest element • Thus, the PQueue methods getFirst and removeFirst access and remove the root node, respectively • Heap removal (removeFirst): • Replace root with the element in the last filled position • Fix heap by “percolating down”

12. Maintaining the Heap: Removal • removeFirst: • Move value in last • element into root • 2. Percolate down Root = Smallest element 2 3 5 9 10 7 8 14 12 11 16 Last filled position

13. Maintaining the Heap: Removal (cont.) 16 3 5 3 9 10 7 8 16 5 14 12 11 Root object removed (16 copied to root and last node removed) 9 10 7 8 14 12 11 After first iteration (swapped with 3) Percolating down: while greater than smallest child swap with smallest child

14. Maintaining the Heap: Removal (cont.) 3 9 5 3 16 10 7 8 8 9 5 14 12 11 After second iteration (moved 9 up) 12 10 7 8 14 16 11 After third iteration (moved 12 up) Reached leaf node  Stop percolating Percolating down: while greater than smallest child swap with smallest child

15. Maintaining the Heap: Removal (cont.) Root = New smallest element 3 9 5 12 10 7 8 14 16 11 New last filled position

16. Heap Representation • Recall that a complete binary tree can be efficiently represented by an array or a vector: • Children of node at index i are stored at indices • Parent of node at index i is at index 2i + 1 and 2i + 2 floor((i - 1) / 2) 2 3 5 9 10 7 8 14 12 11 16 3 9 5 7 7 14 8 12 9 11 10 16 0 2 1 3 2 5 4 10 6 8

17. Heap Implementation: add (cont.) void addElement(DyArray da, EleType val) { int pos = dyArraySize(da); // Get index of next open spot. dyArrayAdd(da, val); // add element at end int up = (pos – 1) / 2; // Get parent index (up the tree). ... } Example:add 4 to the heap Prior to addition, size = 11 Parent position (up) Next open spot (pos) 3 9 5 7 7 14 8 12 9 11 10 16 11 4 0 2 1 3 2 5 4 10 6 8

18. A couple of Useful Routines void swap (dyArray * da, int i, in j) { // swap elements at I and j EleType temp = dyArrayGet(da, i); dyArrayPut(da, i, dyArrayGet(da, j)); dyArrayPut(da, j, temp); } int indexSmallest (dyArray * da, int i, int j) { // return index of smallest element if (LT(dyArrayGet(da, i), dyArrayGet(da, j))) return i; return j; }

19. Heap Implementation: add (cont.) public void add(dyArray * da, EleType val) { ... // While not at root and new value is less than parent. while ((pos != 0) && (indexSmallest(da, pos, up) == pos)) { swap(da, pos, up); // swap values of parent and child pos = up; up = (pos - 1) / 2; // Move position and compute parent idx. ... } After first iteration:“swapped” new value (4) with parent (7) New parent value: 5 up pos 3 9 5 4 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 5 4 10 6 8

20. Heap Implementation: add (cont.) public void add(dyArray * da, EleType val) { ... // While not at root and new value is less than parent. while ((pos != 0) && (indexSmallest(da, pos, up) == pos)) { swap(da, pos, up); // swap values of parent and child pos = up; up = (pos - 1) / 2; // Move position and compute parent idx. ... } After second iteration:“swapped” new value (4) with parent (5) New parent value: 2 up pos 3 9 5 5 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 4 4 10 6 8

21. Heap Implementation: add (cont.) public void add(Dyarray * da, EleType val) { ... // While not at root and new value is less than parent. while ((pos != 0) && (indexSmallest(da, pos, up) == pos)) { . . . }// End while: reached root or parent is smaller than new value. } // Done. Second part of while test fails:stop iteration and add new value up pos 3 9 5 5 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 4 4 10 6 8

22. Heap Implementation: add (cont.) 2 3 5 9 10 7 8 up 4 14 12 11 16 pos 3 9 5 7 7 14 8 12 9 11 10 16 11 4 0 2 1 3 2 5 4 10 6 8

23. Heap Implementation: add (cont.) 2 3 5 up 9 10 4 8 pos 7 14 12 11 16 3 9 5 4 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 5 4 10 6 8

24. Heap Implementation: add (cont.) 2 3 4 up 9 10 5 pos 8 7 14 12 11 16 3 9 5 5 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 4 4 10 6 8

25. Heap Implementation: add (cont.) 2 3 4 9 10 5 pos 8 7 14 12 11 16 3 9 5 5 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 4 4 10 6 8

26. Heap Implementation: removeFirst void removeFirst(dyArray * da) { int last = dyArraySize(da) – 1; if (last != 0) // Copy the last element to dyArrayPut(da,0, dyArrayGet(da, last)); // the first position. dyArrayRemoveAt(da, last); // Remove last element. adjustHeap(dyArraySize(da), 0); // Rebuild heap property. } First element (elementAt(0)) Last element (last) 3 9 5 5 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 4 4 10 6 8 3 9 5 5 7 14 8 12 9 11 10 16 11 0 7 1 3 2 4 4 10 6 8

27. Heap Implementation: adjustHeap void adjustHeap(EleType * data, int max, int idx) { What’s next:val>min  Copy min to parent spot (idx) Move idx to child child (left child smallest) idx max min = 3 3 9 5 5 7 14 8 12 9 11 10 16 11 0 7 1 3 2 4 4 10 6 8 val = 7

28. Heap Implementation: adjustHeap (cont.) 2 3 4 9 10 5 8 7 14 12 11 16 last 3 9 5 5 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 4 4 10 6 8

29. Heap Implementation: adjustHeap (cont.) 7 3 4 9 10 5 8 New root 14 12 11 16 max 3 9 5 5 7 14 8 12 9 11 10 16 11 0 7 1 3 2 4 4 10 6 8

30. Heap Implementation: adjustHeap (cont.) 7 val>min  Copy min to parent spot (idx) Move idx to child 3 4 val = 7 9 10 5 8 Smallest child (min = 3) 14 12 11 16 max 3 9 5 5 7 14 8 12 9 11 10 16 11 0 7 1 3 2 4 4 10 6 8

31. Heap Implementation: adjustHeap (cont.) 3 val>min  Copy min to parent spot (idx) Move idx to child 7 4 val = 7 9 10 5 8 idx 14 12 11 16 max 3 9 5 5 7 14 8 12 9 11 10 16 11 0 3 1 7 2 4 4 10 6 8

32. Heap Implementation: adjustHeap (cont.) 3 If larger than both children Done 7 4 val = 7 9 10 5 8 idx 14 12 11 16 max Smallest child (min = 9) 3 9 5 5 7 14 8 12 9 11 10 16 11 0 3 1 7 2 4 4 10 6 8

33. AdjustHeap - Write recursively void adjustHeap (EleType * data, int max, int pos) { int leftChild = pos * 2 + 1; int rightChild = pos * 2 + 2; if (rightChild < max) { // have two children // get index of smallest // compare smallest child to pos // if necessary, swap and call adjustHeap(max, child) } else if (leftChild < max) { // have only one child // compare child to parent // if necessary, swap and call adjustHeap(max, child) } // else no children, we are at bottom }

34. Priority Queues: Performance Evaluation So, which is the best implementation of a priority queue?

35. Priority Queues: Performance Evaluation • Remember that a priority queue’s main purpose is rapidly accessing and removing the smallest element! • Consider a case where you will insert (and ultimately remove)n elements: • ReverseSortedVector and SortedList: Insertions: n * n = n2 Removals: n * 1 = n Total time: n2 + n = O(n2) • Heap: Insertions: n * log n Removals: n * log n Total time: n * log n + n * log n = 2n log n = O(n log n)