RAIK 283: Data Structures & Algorithms

1. RAIK 283: Data Structures & Algorithms Transform and Conquer (Heaps and Heapsort) Dr. Ying Lu ylu@cse.unl.edu Design and Analysis of Algorithms – Chapter 6

2. RAIK 283: Data Structures & Algorithms • Giving credit where credit is due: • Most of the lecture notes are based on the slides from the Textbook’s companion website • http://www.aw-bc.com/info/levitin • Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope College and Dr. Ellen Walker from Hiram College • I have modified many of their slides and added new slides. Design and Analysis of Algorithms – Chapter 6

3. Heapsort Definition: A heap is a binary tree with the following conditions: • it is essentially complete: (all levels are full except possibly the last level, where only some rightmost leaves may be missing.) • the key at each node is ≥ keys at its children --- parental dominance Design and Analysis of Algorithms – Chapter 6

4. 9 9 5 7 5 7 2 1 6 2 1 9 5 7 4 2 5 Heaps (or not)? tree 1 tree 2 tree 3 Design and Analysis of Algorithms – Chapter 6

5. Definition implies: • There exists a uniquely structured binary tree with n nodes that is essentially complete, with h= lgn • The root has the largest key • The subtree rooted at any node of a heap is also a heap • Partial order tree property Design and Analysis of Algorithms – Chapter 6

6. Priority queues • A priority queue is the abstract data type (ADT) of an ordered set with the operations: • find element with highest priority • delete element with highest priority • insert element with assigned priority • Heaps are very good for implementing priority queues Design and Analysis of Algorithms – Chapter 6

7. Bottom-up vs. Top-down heap construction • Bottom-up: Put everything in and then fix it • Top down: Heaps can be constructed by successively inserting elements into an (initially) empty heap Design and Analysis of Algorithms – Chapter 6

8. Bottom-up construction • Insert elements in the order given breadth-first in a binary tree • Starting with the last (rightmost) parental node, fix the heap rooted at it, if it does not satisfy the heap condition: • exchange it with its largest child • fix the subtree rooted at it (now in the child’s position) Example: 2 9 7 6 5 8 10 Efficiency: Design and Analysis of Algorithms – Chapter 6

9. h-1 Σ 2(h-i) 2i = 2 ( n – lg (n + 1)) = Θ(n) i=0 # nodes at level i Bottom-up heap construction analysis • For parental node at level i it does 2(h-i) comparisons in the worst case • Total: Design and Analysis of Algorithms – Chapter 6

10. Top-down heap construction • Top down: Heaps can be constructed by successively inserting elements into an (initially) empty heap Design and Analysis of Algorithms – Chapter 6

11. Top-down construction: Insertion of a new element • Insert element at last position in heap • Compare with its parent and if it violates heap condition exchange them • Continue comparing the new element with nodes up the tree until the heap condition is satisfied Example: 2 9 7 6 5 8 10 Efficiency: Design and Analysis of Algorithms – Chapter 6

12. Top-down construction: Insertion of a new element • Insert element at last position in heap. • Compare with its parent and if it violates heap condition exchange them • Continue comparing the new element with nodes up the tree until the heap condition is satisfied Efficiency:  log(i)  n log(n) Design and Analysis of Algorithms – Chapter 6

13. Bottom-up vs. Top-down heap construction • Bottom-up: Put everything in and then fix it • Top down: Heaps can be constructed by successively inserting elements into an (initially) empty heap • Which one is better? Design and Analysis of Algorithms – Chapter 6

14. In-class exercise • Page 233 Exercise 6.4.1 • a. Construct a heap for the list 1, 8, 6, 5, 3, 7, 4 by the bottom-up algorithm. • b. Construct a heap for the list 1, 8, 6, 5, 3, 7, 4 by successive key insertions (top-down algorithm). • c. Is it always true that the bottom-up and top-down algorithms yield the same heap for the same input? Design and Analysis of Algorithms – Chapter 6

15. Heapsort Strategy • If the elements to be sorted are arranged in a heap, how can we build a sorted sequence from it? Design and Analysis of Algorithms – Chapter 6

16. Heapsort Strategy • If the elements to be sorted are arranged in a heap, we can build a sorted sequence in reverse order by • repeatedly removing the element from the root, • rearranging the remaining elements to reestablish the partial order tree property, • and so on. Design and Analysis of Algorithms – Chapter 6

17. Heapsort Algorithm: • Build heap • Remove root –exchange with last (rightmost) leaf • Fix up heap (excluding last leaf) Repeat 2, 3 until heap contains just one node. Design and Analysis of Algorithms – Chapter 6

18. Root deletion and Heap fix up The root of a heap can be deleted and the heap fixed up as follows: • exchange the root with the last leaf • compare the new root (formerly the leaf) with each of its children and, if one of them is larger than the root, exchange it with the larger of the two. • continue the comparison/exchange with its current children until it reaches a level of the tree where it is larger than both its children Design and Analysis of Algorithms – Chapter 6

19. deleteMax()in action Design and Analysis of Algorithms – Chapter 6

20. Analysis of Heapsort (continued) Recall algorithm: • Build heap • Remove root –exchange with last (rightmost) leaf • Fix up heap (excluding last leaf) Repeat 2, 3 until heap contains just one node. Design and Analysis of Algorithms – Chapter 6

21. k=n – 1, n-2, … 1 Analysis of Heapsort (continued) Recall algorithm: • Build heap • Remove root –exchange with last (rightmost) leaf • Fix up heap (excluding last leaf) Repeat 2, 3 until heap contains just one node. Θ(n) Θ(log k) Total:Θ(n) + Θ( n log n) = Θ(n log n) • Note: this is the worst case. Average case also Θ(n log n). Design and Analysis of Algorithms – Chapter 6

22. Analysis of Heapsort (continued) • Is HeapSort an in-place sorting algorithm? • How to store/represent a Heap in an array? Design and Analysis of Algorithms – Chapter 6

23. 9 1 2 3 4 5 6 5 3 9 5 3 1 4 2 1 4 2 Representation • Use an array to store breadth-first traversal of heap tree: • Example: • Left child of node j is at 2j • Right child of node j is at 2j+1 • Parent of node j is at j/2 • Parental nodes are represented in the first n/2 locations • Using array: simpler and efficient implementation of heaps Design and Analysis of Algorithms – Chapter 6

24. Bottom-up heap construction algorithm Design and Analysis of Algorithms – Chapter 6

25. Heapsort example • Sorting the array 2 9 7 6 5 8 10 Design and Analysis of Algorithms – Chapter 6

26. In-class exercises (I) • Exercise 6.4.6 (a) (c) • Sort the following lists by heapsort by using the array representation of heaps: • a) 1, 2, 3, 4, 5 (in increasing order) • c) S, O, R, T, I, N, G (in alphabetical order) • Design a heap construction algorithm by applying divide and conquer strategy Design and Analysis of Algorithms – Chapter 6

27. Heap construction Strategy (divide and conquer) • base case is a tree consisting of one node

28. Construct Heap Outline • Input: A heap structure H that does not necessarily have the partial order tree property • Output: H with the same nodes rearranged to satisfy the partial order tree property • void constructHeap(H) // Outline • if (H is not a leaf) • constructHeap (left subtree of H); • constructHeap (right subtree of H); • Element K = root(H); • fixHeap(H, K); • return; • W(n) = W(n-r-1) + W(r) + 2 lg(n) for n > 1 • W(n)  (n); heap is constructed in linear time.