Heap Sort

1. Heap Sort

2. A Heap is a Binary Tree Height of tree = longest path from root to leaf = Q(lgn) A heap is a binary tree satisfying the heap condition: At every node in a heap, the node value is >= all the values in its subtrees. A heap with heap_size elements can be represented as an array segment: A[1..heap_size]

3. Ordering of Nodes 1 16 Each node is filled in order from left to right. All rows are filled except, possibly, the last row. 3 2 14 10 5 6 7 4 3 9 7 8 8 9 10 2 4 1 Address of left child of node i => 2i Address of right child of node i => 2i+1 Address of parent of node i => Example: Left(4) = 8 Right(4) = 9 Parent(4) = 2

4. Heap properties At every node in the heap, that node's value is greater than or equal to all the values in the subtrees of that node. Heaps can be represented as an array. The elements in the array occur in the order of their heap addresses. A= 16 14 10 8 7 9 3 2 4 1 Note that the array is not sorted. Can guarantee that A[1] is the maximum value in the array.

5. Building a Heap • To perform a heap sort, we must first build a heap out of our unsorted array. • To build a heap, we will use the function heapify, which creates a heap from a node i and two subtrees that are already heaps.

6. Heapify 1 Example: i = 2 Want to move the value, 3, to a position consistent with the heap property. 16 3 i=2 3 14 5 6 7 4 11 9 6 15 8 9 10 7 8 2 In English: Compare the value at node i with the values at each child node. Swap the value at node i with the largest of those values. Repeat with the subtree that ith value was swapped into. Note: If the value at i is in the correct place, it will be swapped with itself and the algorithm is done.

7. Pseudocode for Heapify Heapify (A, i) lft <- Left(i) rt <- Right(i) if lft <= heap-size[A] and A[lft] > A[i] then largest <- lft else largest <- i if rt <= heap-size[A] and A[rt] > A[largest] then largest <- rt if largest != i then Swap( A, i, largest ); Heapify (A, largest);

8. Final Tree from example 1 16 3 i=2 15 14 5 6 7 4 11 9 6 8 8 9 10 7 3 2 Can show that heapify runs in O(lgn) time. (Why?)

9. Building a Heap Each leaf of a binary tree is already a heap of size 1. Build-Heap will start with the highest addressed, non-leaf node and heapify it. It then repeats with each node addressed 1 less than the previous node. In a binary tree, the leaf nodes have addresses: We want to heapify starting with node: Build-Heap(A) heap-size <- length[A] for i <- length[A] / 2 downto 1 do Heapify(A, i)

10. Example A= 4 7 16 3 27 9 10 34 1 25 1 4 3 2 7 16 5 6 7 4 10 9 27 3 8 9 10 34 1 25 We will work this out in class.

11. Heap Sort • In English: • We know that the maximum value is at A[1] in a heap. • We put that value at the end of the array, by swapping it with the last element of the heap. • We then reduce the size of the heap by 1, so that the sorted element is no longer part of the heap. • Heapify the new, smaller heap (A[1] is not necessarily in the correct position for a heap). This will put a new maximum at the top. • Repeat this process until all the nodes have been sorted.

12. Pseudocode for Heapsort • Heapsort(A) • Build-Heap(A) • for i <- length[A] downto 2 • do Swap(A, 1, i) • heap-size[A] <- heap-size[A] - 1 • Heapify(A, 1) • What is the running time of Heapsort? • Must find running times of Build-Heap and Heapify first.

13. Recall Heapify Heapify( ) is a function that creates a heap from a node i and two subtrees that are already heaps. 1 16 Example: i = 2 3 i=2 3 14 5 6 7 4 11 9 6 15 8 9 10 7 8 2 Swap the value at node i with the largest of the values at child nodes. If value at node i is larger than those of children, stop. Otherwise, continue with the next level.

14. Running time of Heapify Maximum number of swaps in Heapify is the height of the heap. 1 Height of n element heap is: 3 2 5 6 7 4 8 9 10 Therefore, the running time of heapify is O(lgn)

15. Running time of Build heap Recall that Build-Heap starts with the last element and repeatedly calls Heapify to create heaps from the bottom up. If the height of a node is given by the longest distance from a leaf to that node, then the number of nodes at a given height h is at most: Why? Build-Heap calls heapify for each node at a given height: T(n) = ? number of nodes at height h cost of Heapify at height h

16. Cost of Heap-Sort • Heapsort(A) • Build-Heap(A) • for i <- length[A] downto 2 do • Swap(A, 1, i) • heap-size[A] <- heap-size[A] - 1 • Heapify(A, 1) O(n) n n*const n*const n*O(lgn) T(n) = O(n) + n*O(lgn) = O(nlgn)

17. Using a Heap in Priority Queues • A priority queue is a queue in which the element with the highest value is retrieved first. WIPO: Whatever in, Priority out. • Priority Queue Operations: • Insert(S, x) • Inserts element x into set S • Maximum(S) • Returns the element of S with the largest value (or key) • Extract-Max(S) • Removes and returns the element of S with the largest value. • Heaps are useful data structures for priority queue functions.

18. Extracting the Maximum Idea: Extract the maximum element, reduce the heap size by 1, then heapify. • Heap-Extract-Max(A) • if heap_size[A] < 1 then • error "heap underflow" • max <- A[1] { root value is max } • A[1] <- A[heap_size[A]] { put A[heap_size[A]] in root} • heap_size[A] <- heap_size[A] - 1 { decrease heap size } • Heapify(A, 1) { enforce heap property } • return max

19. Example 1 16 We will work this through in class. 3 12 8 5 4 2 3

20. Heap-Insert Idea: To insert an element at the proper place in the heap, traverse the tree from leaf to root and find the correct place. HeapInsert(A, key) heap_size[A] <- heap_size[A] + 1 { increase size of heap } i <- heap-size[A] while i > 1 and A[Parent(i)] < key do A[i] <- A[Parent(i)] i <- Parent(i) A[i] <- key

21. Example Insert the number, 15. We will work through this in class. 1 16 3 14 10 5 6 7 4 3 9 7 8 8 9 10 2 4 1

22. Binary Heap: Definition • Binary heap. • Almost complete binary tree. • filled on all levels, except last, where filled from left to right • Min-heap ordered. • every child greater than (or equal to) parent 06 14 45 78 18 47 53 91 77 83 84 81 99 64

23. Binary Heap: Properties • Properties. • Min element is in root. • Heap with N elements has height = log2 N. 06 N = 14Height = 3 14 45 78 18 47 53 91 77 83 84 81 99 64

24. Binary Heaps: Array Implementation • Implementing binary heaps. • Use an array: no need for explicit parent or child pointers. • Parent(i) = i/2 • Left(i) = 2i • Right(i) = 2i + 1 06 1 14 45 2 3 78 18 47 53 4 5 6 7 91 77 83 84 81 99 64 8 9 10 12 14 11 13

25. next free slot 42 Binary Heap: Insertion • Insert element x into heap. • Insert into next available slot. • Bubble up until it's heap ordered. • Peter principle: nodes rise to level of incompetence 06 14 45 78 18 47 53 91 77 83 84 81 99 64

26. 42 42 Binary Heap: Insertion • Insert element x into heap. • Insert into next available slot. • Bubble up until it's heap ordered. • Peter principle: nodes rise to level of incompetence swap with parent 06 14 45 78 18 47 53 91 77 83 84 81 99 64

27. 42 Binary Heap: Insertion • Insert element x into heap. • Insert into next available slot. • Bubble up until it's heap ordered. • Peter principle: nodes rise to level of incompetence swap with parent 06 14 45 78 18 47 42 91 77 83 84 81 99 64 53

28. Binary Heap: Insertion • Insert element x into heap. • Insert into next available slot. • Bubble up until it's heap ordered. • Peter principle: nodes rise to level of incompetence • O(log N) operations. stop: heap ordered 06 14 42 78 18 47 45 91 77 83 84 81 99 64 53

29. Binary Heap: Decrease Key • Decrease key of element x to k. • Bubble up until it's heap ordered. • O(log N) operations. 06 14 42 78 18 47 45 91 77 83 84 81 99 64 53

30. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. • power struggle principle: better subordinate is promoted 06 14 42 78 18 47 45 91 77 83 84 81 99 64 53

31. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. • power struggle principle: better subordinate is promoted 53 14 42 78 18 47 45 91 77 83 84 81 99 64 06

32. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. • power struggle principle: better subordinate is promoted exchange with left child 53 14 42 78 18 47 45 91 77 83 84 81 99 64

33. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. • power struggle principle: better subordinate is promoted exchange with right child 14 53 42 78 18 47 45 91 77 83 84 81 99 64

34. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. • power struggle principle: better subordinate is promoted • O(log N) operations. stop: heap ordered 14 18 42 78 53 47 45 91 77 83 84 81 99 64

35. Binary Heap: Heapsort • Heapsort. • Insert N items into binary heap. • Perform N delete-min operations. • O(N log N) sort. • No extra storage.