Amortized Algorithm Analysis

1. Amortized Algorithm Analysis COP3503 July 25, 2007 Andy Schwartz

2. Contents • Indirect Solutions • Amortized Analysis Introduction • Extendable Array Example • Binomial Queue • Binomial Queue Example

3. Indirect Solution 10 yards / minute 100 yards / minute (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

4. Indirect Solution 10 yards / minute 100 yards / minute Geometric Series? The easy solution is indirect. It takes a kitten 5 minutes togo 50 yards, how far can the mother go in 5 minutes?.... 500 yards! (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

5. Amortized Analysis Introduction • The worst-case running time is not always the same as the worst possible average running time. • Example: • Worst case-time is O(n) • Amortized worst-case is O(1) • This could be from a series of table inserts and clears (Source: Arup Guha. CS2 Notes – Summer 2007.

6. Amortization Techniques • Two Techniques: • Potential Function • Accounting Method • Start with $X dollars for n operations. • Each simple operation costs $1, but larger operations are more expensive. • If $X dollars is enough for n operations, then the amortized worst-case is $x / n. • Example: School Semester costs (worst-case most expensive in Fall, but amortized worst-case including spring and summer is cheaper). • Compute the amortized worst-case running time of n consecutive variable-popping stack operations. (details on board) (Source: Arup Guha. CS2 Notes – Summer 2007.

7. Extendable Array Example • ArrayList (or Vector) in Java • Can we maintain an extendable array and keep the amortized worst-case running time down to O(1)? • Yes! (or this would be a bad example) • How?.... • Simple case: add an element and space is left over. • Worst case: ? (Source: Arup Guha. CS2 Notes – Summer 2007.

8. Extendable Array Example • Simple Case: Add an element and space is left over. • Worst Case: Add an element, and there is no more space after  we need to make space  allocate new array and copy elements • The time complexity of this worst-case depends on n, the size of the original array. How big should we make the new array? • There were n-1 simple operations in order to fill the array to this point, so let’s double it. That way a series of n-1 simple insertions, followed by one worst case will result in about n-1 + n = ~2n total operations. • 2n operations / n add operations = 2 = O(2) = O(1) *There were really more than n-1 operations if this was not the first extension, but we know a series of those operations were O(1) and O(1) + O(2) = O(1). O(1) O(k) (Source: Arup Guha. CS2 Notes – Summer 2007.

9. Binomial Queue • Binomial Trees: B0 B1 B2 B3 B4 Each tree “doubles” the previous.

10. Binomial Queue • Binomial Queue • A queue of binomial trees. A “Forest”. • Each tree is essentially a heap constrained to the format of a binary tree. • Example: B0, B2, B3 • Insertion: Create a B0 and merge • Deletion: remove minimum( the root) from tree Bk. This leaves a queue of trees: B0, B1, …, Bk-1. Merge this queue with the rest of the original queue. (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

11. Binomial Queue Example • The Most important step is Merge • Merge Rules: (for two binomial queues) • 0 or 1 Bk trees  just leave merged • 2 Bk trees  meld into 1 Bk+1 tree. • 3 Bk trees  meld two into 1 Bk+1, and leave the third. • Insertion Runtime: • M+1 steps, where M is smallest tree not present. Worst-case is k+2, when Bk+1 is the smallest tree not present. How does k relate to the total number of nodes in the tree? k = lg n, thus (nonamortized) worst-case time is O(lg n). (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

12. Binomial Queue Example • MakeBinQ Problem: Build a binomial queue of N elements. (Like makeHeap). How long should this take? • Insertion worst-case runtime: • Worst-case O(lg n) for 1 insert O(n lg n) n inserts, but we want O(n) – like makeHeap • Try amortized analysis directly: • Considering each linking step of the merge. The 1st, 3rd, 5th, ettc… odd steps require no linking step because there will be no B0. So ½ of all insertions require no linking, similarly ¼ require 1 linking steps. • We could continue down this path, but the it’ll be come especially difficult for deletion (we should learn an indirect analysis). (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

13. Binomial Queue Example • Indirect Analysis (time = M + 1) • If no B0 cost is 1 (M is 0) • Results in 1 B0 tree added to the forest • If B0 but no B1 cost is 2 (M is 1) • Results in same # of trees (new B1 but B0 is gone) • When cost is 3 (M is 2) • Results in 1 less tree (new B2 but remove B0 and B1) ….etc… • When cost is c (M is c – 1) • Results in increase of 2 – c trees (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

14. Binomial Queue Example • increase of 2 – c trees • How can we use this? Ti = # of trees after ithiteration T0 = 0 = # of trees initially Ci = Cost of ithiteration Then, Ti = Ti-1 + (2 – Ci)  Ci + (Ti – Ti-1) = 2 • This is only the ith iteration (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

15. Binomial Queue Example Ci + (Ti – Ti-1) = 2 • To get all iterations: C1 + (T1 – T0) = 2 C2 + (T2 – T1) = 2 … Cn-1 + (Tn-1 – Tn-2) = 2 Cn + (Tn – Tn-1) = 2 n ΣCi + (Tn – T0) = 2n (T1 .. Tn-1 cancel out) i=1

16. Binomial Queue Example n ΣCi + (Tn – T0) = 2n i=1 • T0 = 0 and Tn is definitely not negative, so Tn – T0 is not negative. n ΣCi <= 2n i=1 Thus, the total cost < 2n  makeBinQ = O(n) Since, makeBinQ consists of O(n) inserts, then the amortized worst-case of each insert is O(1).