Priority Queues and Heaps

1. Klaus Sutner January 29, 2004 Priority Queues and Heaps 15-211 Fundamental Data Structures and Algorithms

2. Plan Today: Demo: intelligent math handbook Priority Queues Binary heaps Implementation questions Reading: Chapter 20 in MAW. See also 6.7 and 6.8.

3. A Simulation We want to simulate a chain of events Ei. Each event Ei is associated with a time ti. No problem: sort the times, and run through them in order. t1 < t2 < t3 < ... < tn But what if an event can cause the creation of some new events in the future?

4. A Simulation We can still sort the original events by time, but not the additional ones: we don't yet know what/when they are. We need to be able to dynamically - insert an event into a data structure, and - extract the next event. And, of course, the operations should be cheap.

5. An Example: Street Crossing Suppose a robot wants to cross a street. No jaywalking, it can only move straight forward or backward. Maximum speed of the robot is c. Cars are coming from either side, but the robot has complete information about when and where they cross your line. Problem: Find a way for the robot to get across without getting hit.

6. More Street Crossing How do we model the given information? In space-time a car is just a rectangular box. Movement of the robot is constrained to a light-cone. space time

7. More Street Crossing Hugging an obstacle (not that unrealistic, just inflate the obstacles to account for safety buffer).

8. A Death Trap Note how obstacles may be “fused” together.

9. Street Crossing The times when obstacles appear/disappear are clearly events. But there are more: The times when the robot touches an obstacle. Or when the light cones from both ends of a disappearing obstacle meet. Note that these events have to be computed as we go along, there is no way to compute them all at the outset.

10. Time Travel Note how the first event happens before the first obstacle arrives! We need to go backwards in time, starting at the end of the last obstacle.

11. Towards An Algorithm How does this translate into a data structure? What are the crucial operations? What would be a simple (not necessarily efficient) implementation? Where is room for improvement?

12. Priority Queue Interface Crucial Operations insert find-min delete-min Think about just inserting times (real numbers), but in reality there is associated information.

13. Some Applications Event simulations Shortest path computation Huffman coding Sorting (sort of) Not a universal data structure (like hash tables) but irreplaceable in some cases.

14. PQ Interface Requires total order on the elements of the universe. Could be done with any traversable container: just run through and find the min. We would like much better performance. Binary search trees come to mind: min element is in the leftmost position.

15. Possible priority queue implementations • Linked list • deleteMinO(1) O(N) • insertO(N) O(1) • Search trees • All operations O(log N) • Heapsavg (assume random)worst • deleteMinO(log N) O(log N) • insert 2.6 O(log N) special case: • buildheapO(N) O(N)i.e.,insert*N or

16. Possible priority queue implementations • Linked list • deleteMin O(1) O(N) • insert O(N) O(1) • Search trees • All operations O(log N) • Heapsavg (assume random)worst • deleteMinO(log N) O(log N) • insert 2.6 O(log N) special case: • buildheapO(N) O(N)i.e.,insert*N or

17. Possible priority queue implementations • Linked list • deleteMin O(1) O(N) • insert O(N) O(1) • Search trees • All operations O(log N) • Heapsavg (assume random)worst • deleteMinO(log N) O(log N) • insert 2.6 O(log N) • buildheapO(N) O(N) N inserts or

18. A Digression: Perfect and Complete Binary Trees

19. Perfect binary trees 13 21 16 24 31 19 68 65 26 32 26 65 26 65 26

20. 65 65 65 65 65 65 65 65 26 26 26 26 26 26 26 26 Perfect binary trees 13 21 16 24 31 19 68 65 26 32 26 65 26 65 26

21. 65 65 65 65 65 65 65 65 26 26 26 26 26 26 26 26 65 65 65 65 65 65 65 65 65 65 65 65 65 65 65 65 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 Perfect binary trees 13 21 16 24 31 19 68 65 26 32 26 65 26 65 26

22. 13 21 16 24 31 19 68 65 26 32 26 65 26 65 26 Perfect binary trees • How many nodes? h=3N=15

23. 13 21 16 24 31 19 68 65 26 32 26 65 26 65 26 Perfect binary trees • How many nodes? • In general: N = = 2h+1 - 1 • Most of the nodes are leaves h=3 h=3N=15

24. A Serious Proof Define PBT by structural induction: • nil is a PBT and H(nil) = 0 • (a,L,R) is a PBT whenever L and R are PBTs and H(L) = H(R). • Moreover, H( (a,L,R) ) = H(L) + 1. • size(nil) = 0 • size( (a,L,R) ) = size(L) + size(R) + 1. • Then size(T) = 2H(T) -1.

25. Quiz Break

26. Red-green quiz • In a perfect binary tree, what is the sum of the heights of the nodes? • Give a mathematical characterization, the prettier the better. • Give a tight upper bound (in big-O terms)

27. 13 21 16 24 31 19 68 65 26 32 26 65 26 65 26 Perfect binary trees • What is the sum of the heights? S =< N = O(N) h=3N=15

28. Complete binary trees 13 21 16 24 31 19 68 65 26 32

29. Complete binary trees 1 2 3 4 5 6 7 8 9 10

30. 1 2 3 4 5 6 7 8 9 10 Representing complete binary trees • Linked structures?

31. 1 2 3 4 5 6 7 8 9 10 Representing complete binary trees • Linked structures? No! • Instead, use arrays!

32. 1 2 3 4 5 6 7 8 9 10 Representing complete binary trees • Arrays • Parent at positioni in the array • Children at positions 2i and 2i+1

33. Representing complete binary trees • Arrays (1-based) • Parent at positioni • Children at 2i and 2i+1. 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

34. Representing complete binary trees • Arrays (1-based) • Parent at positioni • Children at 2i and 2i+1. 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

35. Representing complete binary trees • Arrays (1-based) • Parent at positioni • Children at 2i(and 2i+1). 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

36. Representing complete binary trees • Arrays (1-based) • Parent at positioni • Children at 2i and 2i+1. public class BinaryHeap { private Comparable[] heap; private int size; public BinaryHeap(int capacity) { size=0; heap = new Comparable[capacity+1]; } . . .

37. Representing complete binary trees • Arrays • Parent at positioni • Children at 2i and 2i+1. • Example: find the leftmost child int left=1;for(; left<size; left*=2);return heap[left/2]; • Example: find the rightmost child int right=1;for(; right<size; right=right*2+1);return heap[(right-1)/2];

38. Implementing Priority Queues with Binary Heaps

39. Binary heaps: the invariant • Representation invariants 1. Structure property • Complete binary tree (i.e. the elements of the heap are stored in positions 1…size of the array) 2. Heap order property • Parent keys less than children keys

40. Heaps • Representation invariant 1. Structure property • Complete binary tree • Hence: efficient compact representation 2. Heap order property • Parent keys less than children keys • Hence: rapid insert, findMin, and deleteMin • O(log(N)) for insert and deleteMin • O(1) for findMin

41. The heap order property • Each parent is less than each of its children. • Hence: Root is less than every other node. • (obvious proof by induction) 13 21 16 24 31 19 68 65 26 32

42. Operating with heaps Representation invariant: • All methods must: 1.Produce complete binary trees 2. Guarantee the heap order property • All methods may assume 1. The tree is initially complete binary 2. The heap order property holds

43. Constructor method • All methods must: 1. Produce complete binary trees • Trivially true 2. Guarantee the heap order property • Also trivially true • This is the base case

44. findMin () • The code public boolean isEmpty() { return size == 0; } public Comparable findMin() { if(isEmpty()) return null; return heap[1]; } • Does not change the tree • Trivially preserves the invariant

45. insert (Comparable x) • Process 1. Create a “hole” at the next tree cell for x.heap[size+1] This preserves the completeness of the tree. 2. Percolate the hole up the tree until the heap order property is satisfied. This assures the heap order property is satisfied.

46. insert (Comparable x) • Process 1. Create a “hole” at the next tree cell for x.heap[size+1] This preserves the completeness of the treeassuming it was complete to begin with. 2. Percolate the hole up the tree until the heap order property is satisfied. This assures the heap order property is satisfiedassuming it held at the outset.

47. Percolation up public void insert(Comparable x) throws Overflow { if(isFull()) throw new Overflow(); int hole = ++size; for(; hole>1 && x.compareTo(heap[hole/2])<0; hole/=2) heap[hole] = heap[hole/2]; heap[hole] = x; }

48. Percolation up • Bubble the hole up the tree until the heap order property is satisfied. hole = 11 HOP false 13 21 16 24 31 19 68 Not really there... 65 26 32 14

49. Percolation up • Bubble the hole up the tree until the heap order property is satisfied. hole = 11 hole = 5 HOP false HOP false 13 13 21 16 21 16 24 31 19 68 24 14 19 68 65 26 32 14 65 26 32 31

50. Percolation up • Bubble the hole up the tree until the heap order property is satisfied. hole = 5 hole = 2 HOP false HOP true done 13 13 21 16 14 16 24 14 19 68 24 21 19 68 65 26 32 31 65 26 32 31