UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2001

1. UMass Lowell Computer Science 91.503Analysis of AlgorithmsProf. Karen DanielsFall, 2001 Lecture 11 Tuesday, 12/4/01 Advanced Data Structures Chapters 20-21

2. Relevant Sections of Chapters You’re responsible for material in this chapter that we discuss in lecture. (Note that this includes all sections.) Ch20 Binomial Heaps Ch7 HeapSort You’re responsible for material in this chapter that we discuss in lecture. (Note that this includes all sections.) Ch21 Fibonacci Heaps Note that Chapter 22 has been removed.

3. Overview • Chapter 7: Heap Review • Chapter 20: Binomial Heaps • Chapter 21: Fibonacci Heaps

4. Chapter 7 Heap Review

5. 16 14 10 8 7 9 3 2 4 1 16 14 10 8 7 9 3 2 4 1 1 2 3 4 5 6 7 8 9 10 Review of Heap Basics • Structure: • Nearly complete binary tree • Convenient array representation • HEAP Property: (for MAX HEAP) • Parent’s label not less than that of each child

6. Operations on a Heap assuming array representation • HEAPIFY: • for a given node that is the root of a subtree, if both subtrees of that node are already HEAPs, HEAPIFY enforces the HEAP PROPERTY via “downward swaps” so that the node together with its subtrees form a HEAP • BUILD-HEAP: • builds a HEAP from scratch using HEAPIFY • HEAPSORT: • sorts an array by first using BUILD-HEAP then repeatedly swapping out root and calling HEAPIFY

7. O(n) + nO(lgn) O(lgn) Operations on a Heap assuming array representation T(n) = T(2n/3) + Q(1) is in O(lgn) using Master Theorem • HEAPIFY: • BUILD-HEAP: • HEAPSORT: Asymptotic worst-case running time is in O(lg n). For a node at height h, time is in O(h). Asymptotic worst-case running time is in O(n lg n). However this is a loose bound! Time is also in O(n). Asymptotic worst-case running time is in O(n lg n).

8. Operations on a Heap assuming array representation • PRIORITY QUEUE SUPPORT: • HEAP-INSERT • adds new leaf to the tree and then “swaps up” to restore HEAP PROPERTY • HEAP- MAXIMUM • HEAP PROPERTY guarantees that maximum is at the root of a MAX HEAP • HEAP- EXTRACT-MAX • removes the maximum value from the root by swapping it out • restores HEAP PROPERTY using HEAPIFY Applications: Job Scheduling, Event Scheduling

9. Operations on a Heap assuming array representation • PRIORITY QUEUE SUPPORT: • HEAP-INSERT • HEAP- MAXIMUM • HEAP- EXTRACT-MAX Asymptotic worst-case running time is in O(lg n). For a node at height h, time is in O(h). Asymptotic worst-case running time is in O(1). Asymptotic worst-case running time is in O(lg n). O(1) For a node at height h, time is in O(h). + O(lgn)

10. 16 14 10 8 7 9 3 2 4 1 Building a Heap usingHEAPIFY vs. HEAP-INSERT • HEAP-INSERT • swaps up • compares parent with one child before each swap • HEAPIFY • swaps down • compares parent with both children before each swap maximum number of swaps = length of path from this level up to root number of levels number of levels maximum number of swaps = length of path from this level down to leaf number of nodes in this level number of nodes in this level O(n) as in HEAPIFY Asymptotic worst-case running time of BUILD-HEAP using HEAPIFY is in O(n). However, using HEAP-INSERT the time would only be in O(n lg n).

11. Chapter 20 Binomial Heaps

12. Mergeable Heap Operations If UNION not needed, binary heap (Ch7) suffices Ch20/21 Goal: Mergable Heaps supporting fast UNION source: 91.503 textbook Cormen et al.

13. swap up if heap property violation add to root list; update min pointer UNION search root list use min pointer swap with root; swap down if heap property violation orphan, reverse, UNION orphan, consolidate = degree roots add to root list; update min pointer orphan, consolidate = degree roots swap up if heap property violation cascading cut if heap property violation swap up if heap property violation DECREASE-KEY, EXTRACT-MIN DECREASE-KEY, EXTRACT-MIN Ch20 Ch21 (inefficient SEARCH) Mergeable Heap Operations Ch20/21 Goal: Mergable Heaps supporting fast UNION source: 91.503 textbook Cormen et al.

14. Binomial Tree Bk is an ordered tree defined recursively (children are ordered) Binomial Tree Definition Binomial Heap is a collection of Binomial Trees source: 91.503 textbook Cormen et al.

15. Binomial Tree Properties source: 91.503 textbook Cormen et al.

16. Binomial Tree Properties source: 91.503 textbook Cormen et al.

17. Binomial Tree Properties Proof: (continued) source: 91.503 textbook Cormen et al.

18. Binomial Heap Definition root degrees increase along root list source: 91.503 textbook Cormen et al.

19. ( assume no keys with value ) Binomial Heap Operations:MAKE-HEAP, MINIMUM MAKE-HEAP worst-case running time is in Q(1) HEAP-MINIMUM worst-case running time is in O(lgn) Minimum key must be in a root node due to heap-ordering. There are at most roots to check. search root list source: 91.503 textbook Cormen et al.

20. Binomial Heap Operations:UNION Merge root lists into single linked list sorted by nondecreasing degree Links 2 binomial heaps whose roots have same degree Link roots of equal degree until at most one root remains of each degree Pointers into root list: HEAP-UNION worst-case running time is in O(lgn) source: 91.503 textbook Cormen et al. orphan, consolidate = degree roots

21. Binomial Heap Operations:UNION (continued) 3 roots of same degree source: 91.503 textbook Cormen et al.

22. Binomial Heap Operations:UNION (continued) source: 91.503 textbook Cormen et al.

23. Binomial Heap Operations:UNION (continued) source: 91.503 textbook Cormen et al.

24. Binomial Heap Operations:INSERT HEAP-INSERT worst-case running time is in O(lgn) UNION source: 91.503 textbook Cormen et al.

25. Binomial Heap Operations:EXTRACT-MIN HEAP-EXTRACT-MIN worst-case running time is in Q(lgn) orphan, reverse, UNION source: 91.503 textbook Cormen et al.

26. Binomial Heap Operations:DECREASE-KEY If violate heap property, swap up. No change in structure. HEAP-DECREASE-KEY worst-case running time is in Q(lgn) swap up if heap property violation source: 91.503 textbook Cormen et al.

27. Binomial Heap Operations: DELETE HEAP-DELETE worst-case running time is in Q(lgn) DECREASE-KEY, EXTRACT-MIN source: 91.503 textbook Cormen et al.

28. Chapter 21 Fibonacci Heaps

29. swap up if heap property violation add to root list; update min pointer UNION search root list use min pointer swap with root; swap down if heap property violation O(D(n)) orphan, reverse, UNION orphan, consolidate = degree roots add to root list; update min pointer orphan, consolidate = degree roots swap up if heap property violation cascading cut if heap property violation swap up if heap property violation O(D(n)) DECREASE-KEY, EXTRACT-MIN DECREASE-KEY, EXTRACT-MIN Ch20 Ch21 (inefficient SEARCH) Mergeable Heap Operations Ch20/21 Goal: Mergable Heaps supporting fast UNION source: 91.503 textbook Cormen et al.

30. Heap Potential Function Fibonacci Heap Basics For asymptotically fast MST, shortest paths Collection of trees Relaxed structure Lazy: delay work Amortized (potential) cost Minimum root t(H) = #trees in root list m(H) = #marked nodes Circular linked lists source: 91.503 textbook Cormen et al.

31. Potential Method (review) • Potential Method • amortized cost can differ across operations (as in accounting method) • overcharge some operations early in sequence (as in accounting method) • store overcharge as “potential energy” of data structure as a whole • (unlike accounting method) • Let ci be actual cost of ith operation • Let Di be data structure after applying ith operation • Let F(Di ) be potential associated with Di • Amortized cost of ith operation: • Total amortized cost of n operations: • Must have: to “pay in advance” terms telescope

32. Unordered Binomial Tree Properties DECREASE-KEY Fibonacci Heap operation may violate Unordered Binomial Tree properties. DIFFERENCE source: 91.503 textbook Cormen et al.

33. t (H) = #trees in root list m(H) = #marked nodes Heap Potential Function Fibonacci Heap Operations:INSERT add to root list; update min pointer Increase in potential = Actual cost is in O(1) Amortized cost is in O(1) source: 91.503 textbook Cormen et al.

34. t (H) = #trees in root list m(H) = #marked nodes Heap Potential Function Fibonacci Heap Operations:UNION add to root list; update min pointer source: 91.503 textbook Cormen et al.

35. Fibonacci Heap Operations:EXTRACT-MIN First, disassemble old min tree. disassemble Next, update “min” Process one tree at a time, starting with new “min”. Consolidate wherever possible: Link roots of = degree until at most one root remains of each degree orphan, consolidate = degree roots source: 91.503 textbook Cormen et al.

36. Fibonacci Heap Operations:EXTRACT-MIN Consolidation Degree 0 merge: Combine 7 with 23 Degree 1 merge: (keep going) Combine 7/23 with 17/30 Degree 2 merge: (keep going) Combine 7… with 24… source: 91.503 textbook Cormen et al.

37. Fibonacci Heap Operations:EXTRACT-MIN Consolidation source: 91.503 textbook Cormen et al.

38. Fibonacci Heap Operations:EXTRACT-MIN Pseudocode source: 91.503 textbook Cormen et al.

39. Fibonacci Heap Operations:EXTRACT-MIN Pseudocode may do multiple degree merges source: 91.503 textbook Cormen et al.

40. t (H) = #trees in root list m(H) = #marked nodes Heap Potential Function Fibonacci Heap Operations:EXTRACT-MIN Analysis D(n) = upper bound on maximum degree of any node in an n-node Fibonacci heap (shown in Section 21.3 to be in O(lgn)) Actual Work: >= O(D(n)) since at most D(n) children of minimum node When CONSOLIDATE is called, size of root list <= D(n) + t(H) - 1 Work in CONSOLIDATE’s for loop in O(D(n) + t(H)) due to tree linking in each iteration Total Actual Cost is in O(D(n) + t(H)) O(D(n)) Amortized cost is in O(lgn) source: 91.503 textbook Cormen et al.

41. Fibonacci Heap Operations: DECREASE-KEY PseudoCode cascading cut if heap property violation source: 91.503 textbook Cormen et al.

42. t (H) = #trees in root list m(H) = #marked nodes Heap Potential Function Change in potential is at most:: Amortized cost is in Q(1) Fibonacci Heap Operations: DECREASE-KEY Analysis source: 91.503 textbook Cormen et al. As soon as 2nd child of x is lost, cut x from parent, making it a new root. Actual Work: Dominated by cost of CASCADING-CUT Assume CASCADING-CUT called recursively c times Total Actual Cost is in O(c) t(H) to start with, c-1 added from cuts, + tree rooted at x c-1 unmarked by cascading cuts + at most 1 from last call to CASCADING-CUT

43. Fibonacci Heap Operations: DELETE DECREASE-KEY, EXTRACT-MIN Amortized cost is in O(lgn) O(D(n)) source: 91.503 textbook Cormen et al.

44. source: 91.503 textbook Cormen et al. Bounding the Maximum Degree EXTRACT-MIN (& DELETE) O(lgn) bounds require D(n) in O(lgn) Show:Cutting node when it loses 2nd child Link only occurs in EXTRACT-MIN, DELETE

45. Bounding the Maximum Degree source: 91.503 textbook Cormen et al.

46. Bounding the Maximum Degree Exercise 2.2-8 source: 91.503 textbook Cormen et al.

47. Bounding the Maximum Degree Thus, EXTRACT-MIN (& DELETE) have O(lgn) time bounds. source: 91.503 textbook Cormen et al.