CS 3343: Analysis of Algorithms

CS 3343: Analysis of Algorithms Review for Exam 2

Exam 2 • Closed book exam • One cheat sheet allowed (limit to a single page of letter-size paper, double-sided) • Thursday, Nov 17, class time + 5 minutes • Basic calculator (no graphing) is allowed but not necessary

Materials covered • Quick Sort • Heap sort, priority queue • Linear time sorting algorithms • Order statistics • Dynamic programming • Greedy algorithm • Questions will be similar to homework / quizzes • Familiar with the algorithm procedure • Some analysis of time/space complexity • One or two problems on algorithm design

QuickSort • Quicksort algorithm • Randimized Quicksort algorithm • Partition algorithm • Quicksort/Randomized Quicksort time complexity analysis Back

£x x ≥x Quick sort • Quicksort an n-element array: • Divide:Partition the array into two subarrays around a pivotx such that elements in lower subarray £x£ elements in upper subarray. • Conquer: Recursively sort the two subarrays. • Combine: Trivial. Key:Linear-time partitioning subroutine.

Pseudocode for quicksort • QUICKSORT(A, p, r) • ifp < r • thenq PARTITION(A, p, r)QUICKSORT(A, p, q–1) • QUICKSORT(A, q+1, r) Initial call:QUICKSORT(A, 1, n)

Partition Code Partition(A, p, r) x = A[p]; // pivot is the first element i = p; j = r + 1; while (TRUE) { repeat i++; until A[i] > x | i >= j; repeat j--; until A[j] < x | j <= i; if (i < j) Swap (A[i], A[j]); else break; } swap (A[p], A[j]); return j;

p r 6 10 5 8 13 3 2 11 x = 6 i j 6 10 5 8 13 3 2 11 scan i j 6 2 5 8 13 3 10 11 swap i j Partition example 6 2 5 8 13 3 10 11 scan i j 6 2 5 3 13 8 10 11 swap i j 6 2 5 3 13 8 10 11 scan j i p q r 3 2 5 6 13 8 10 11 final swap

3 2 5 6 11 8 10 13 2 3 5 6 10 8 11 13 2 3 5 6 8 10 11 13 2 3 5 6 8 10 11 13 6 10 5 8 11 3 2 13 Quick sort example

Quicksort Runtimes • Best case runtime Tbest(n)  O(n log n) • Worst case runtime Tworst(n)  O(n2) • Average case runtime Tavg(n)  O(n log n) • Expected runtime of randomized quicksort is O(n log n)

Randomized Partition • Randomly choose an element as pivot • Every time need to do a partition, throw a die to decide which element to use as the pivot • Each element has 1/n probability to be selected Rand-Partition(A, p, r){ d = random(); // draw a random number between 0 and 1 index = p + floor((r-p+1) * d); // p<=index<=q swap(A[p], A[index]); Partition(A, p, r); // now use A[p] as pivot }

Running time of randomized quicksort • The expected running time is an average of all cases T(0) + T(n–1) + dn if 0:n–1 split, T(1) + T(n–2) + dn if 1:n–2 split, M T(n–1) + T(0) + dn if n–1:0 split, T(n) = Expectation

Fact: • Need to Prove:T(n) ≤ c n log (n) • Assumption:T(k) ≤ ck log (k) for 0 ≤ k ≤ n-1 • Prove by induction If c ≥ 4

Heaps • Heap definition • Heapify • Buildheap • Procedure and running time (!) • Heapsort • Comparison with other sorting algorithms in terms of running time, stability, and in-place. • Priority Queue operations Back

16 14 10 8 7 9 3 2 4 1 Heaps • A heap can be seen as a complete binary tree: Perfect binary tree 16 14 10 8 7 9 3 2 4 1

Referencing Heap Elements • So… Parent(i) {return i/2;} Left(i) {return 2*i;} right(i) {return 2*i + 1;}

Heap Operations: Heapify() Heapify(A, i) { //precondition: subtrees rooted at l and r are heaps l = Left(i); r = Right(i); if (l <= heap_size(A) && A[l] > A[i]) largest = l; else largest = i; if (r <= heap_size(A) && A[r] > A[largest]) largest = r; if (largest != i) { Swap(A, i, largest); Heapify(A, largest); } } //postcondition: subtree rooted at i is a heap Among A[l], A[i], A[r], which one is largest? If violation, fix it.

Heapify() Example 16 4 10 14 7 9 3 2 8 1 A = 16 4 10 14 7 9 3 2 8 1

BuildHeap() // given an unsorted array A, make A a heap BuildHeap(A) { heap_size(A) = length(A); for (i = length[A]/2 downto 1) Heapify(A, i); }

BuildHeap() Example • Work through exampleA = {4, 1, 3, 2, 16, 9, 10, 14, 8, 7} 4 1 3 2 16 9 10 14 8 7 A = 4 1 3 2 16 9 10 14 8 7

4 1 3 2 16 9 10 14 8 7 A = 4 1 3 2 16 9 10 14 8 7

4 1 3 14 16 9 10 2 8 7 A = 4 1 3 14 16 9 10 2 8 7

16 14 10 8 7 9 3 2 4 1 A = 16 14 10 8 7 9 3 2 4 1

Analyzing BuildHeap(): Tight • To Heapify() a subtree takes O(h) time where h is the height of the subtree • h = O(lg m), m = # nodes in subtree • The height of most subtrees is small • Fact: an n-element heap has at most n/2h+1 nodes of height h • CLR 6.3 uses this fact to prove that BuildHeap() takes O(n) time

Heapsort Heapsort(A) { BuildHeap(A); for (i = length(A) downto 2) { Swap(A[1], A[i]); heap_size(A) -= 1; Heapify(A, 1); } }

Heapsort Example • Work through exampleA = {4, 1, 3, 2, 16, 9, 10, 14, 8, 7} 4 1 3 2 16 9 10 14 8 7 A = 4 1 3 2 16 9 10 14 8 7

Heapsort Example • First: build a heap 16 14 10 8 7 9 3 2 4 1 A = 16 14 10 8 7 9 3 2 4 1

Heapsort Example • Swap last and first 1 14 10 8 7 9 3 2 4 16 A = 1 14 10 8 7 9 3 2 4 16

Heapsort Example • Last element sorted 1 14 10 8 7 9 3 2 4 16 A = 1 14 10 8 7 9 3 2 4 16

Heapsort Example • Restore heap on remaining unsorted elements 14 8 10 4 7 9 3 2 1 16 Heapify A = 14 8 10 4 7 9 3 2 1 16

Heapsort Example • Repeat: swap new last and first 1 8 10 4 7 9 3 2 14 16 A = 1 8 10 4 7 9 3 2 14 16

Heapsort Example • Restore heap 10 8 9 4 7 1 3 2 14 16 A = 10 8 9 4 7 1 3 2 14 16

Heapsort Example • Repeat 9 8 3 4 7 1 2 10 14 16 A = 9 8 3 4 7 1 2 10 14 16

Implementing Priority Queues HeapMaximum(A) { return A[1]; }

Implementing Priority Queues HeapExtractMax(A) { if (heap_size[A] < 1) { error; } max = A[1]; A[1] = A[heap_size[A]] heap_size[A] --; Heapify(A, 1); return max; }

HeapExtractMax Example 16 14 10 8 7 9 3 2 4 1 A = 16 14 10 8 7 9 3 2 4 1

HeapExtractMax Example Swap first and last, then remove last 1 14 10 8 7 9 3 2 4 16 A = 1 14 10 8 7 9 3 2 4 16

HeapExtractMax Example Heapify 14 8 10 4 7 9 3 2 1 16 A = 14 8 10 4 7 9 3 2 1 16

Implementing Priority Queues HeapChangeKey(A, i, key){ if (key ≤ A[i]){ // decrease key A[i] = key; heapify(A, i); } else { // increase key A[i] = key; while (i>1 & A[parent(i)]<A[i]) swap(A[i], A[parent(i)]; } } Sift down Bubble up

HeapChangeKey Example Increase key 16 14 10 8 7 9 3 2 4 1 A = 16 14 10 8 7 9 3 2 4 1

HeapChangeKey Example Increase key 16 14 10 15 7 9 3 2 4 1 A = 16 14 10 15 7 9 3 2 4 1

CS 3343: Analysis of Algorithms