1 / 32

Analysis of Algorithms CS 477/677

Analysis of Algorithms CS 477/677. Instructor: Monica Nicolescu. A. 0. 1. 2. 3. 4. 5. C. 2. 2. 4. 7. 7. 8. 1. 1. 2. 2. 3. 3. 4. 4. 5. 5. 6. 6. 7. 7. 8. 8. 2. 0. 5. 0. 2. 3. 2. 0. 2. 3. 3. 3. 3. 0. 5. 3. B. Counting Sort. Assumption:

jlakey
Download Presentation

Analysis of Algorithms CS 477/677

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis of AlgorithmsCS 477/677 Instructor: Monica Nicolescu

  2. A 0 1 2 3 4 5 C 2 2 4 7 7 8 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 2 0 5 0 2 3 2 0 2 3 3 3 3 0 5 3 B Counting Sort • Assumption: • The elements to be sorted are integers in the range 0 to k • Idea: • Determine for each input element x, the number of elements smaller than x • Place element x into its correct position in the output array CS 477/677

  3. Radix Sort • Considers keys as numbers in a base-R number • A d-digit number will occupy a field of d columns • Sorting looks at one column at a time • For a d digit number, sort the least significant digit first • Continue sorting on the next least significant digit, until all digits have been sorted • Requires only d passes through the list CS 477/677

  4. Bucket Sort • Assumption: • the input is generated by a random process that distributes elements uniformly over [0, 1) • Idea: • Divide [0, 1) into n equal-sized buckets • Distribute the n input values into the buckets • Sort each bucket • Go through the buckets in order, listing elements in each one • Input: A[1 . . n], where 0 ≤ A[i] < 1 for all i • Output: elements ai sorted • Auxiliary array: B[0 . . n - 1] of linked lists, each list initially empty CS 477/677

  5. BUCKET-SORT Alg.: BUCKET-SORT(A, n) for i ← 1to n do insert A[i] into list B[nA[i]] for i ← 0to n - 1 do sort list B[i] with insertion sort concatenate lists B[0], B[1], . . . , B[n -1] together in order return the concatenated lists CS 477/677

  6. / / .12 / .39 / .26 .68 / .17 .78 .23 / .21 .72 / .94 / / / Example - Bucket Sort 1 0 2 1 3 2 4 3 5 4 6 5 7 6 8 7 9 8 10 9 CS 477/677

  7. / / .78 .23 .68 .78 / .17 / .72 .26 / .39 .94 / .72 .39 / .21 .12 .17 .12 .23 .94 / .26 .21 .68 / / / Example - Bucket Sort 0 1 2 3 4 5 6 7 Concatenate the lists from 0 to n – 1 together, in order 8 9 CS 477/677

  8. Correctness of Bucket Sort • Consider two elements A[i], A[ j] • Assume without loss of generality that A[i] ≤ A[j] • Then nA[i] ≤ nA[j] • A[i] belongs to the same group as A[j] or to a group with a lower index than that of A[j] • If A[i], A[j] belong to the same bucket: • insertion sort puts them in the proper order • If A[i], A[j] are put in different buckets: • concatenation of the lists puts them in the proper order CS 477/677

  9. Analysis of Bucket Sort Alg.: BUCKET-SORT(A, n) for i ← 1to n do insert A[i] into list B[nA[i]] for i ← 0to n - 1 do sort list B[i] with insertion sort concatenate lists B[0], B[1], . . . , B[n -1] together in order return the concatenated lists O(n) (n) O(n) (n) CS 477/677

  10. Conclusion • Any comparison sort will take at least nlgn to sort an array of n numbers • We can achieve a better running time for sorting if we can make certain assumptions on the input data: • Counting sort: each of the n input elements is an integer in the range 0 to k • Radix sort: the elements in the input are integers represented with d digits • Bucket sort: the numbers in the input are uniformly distributed over the interval [0, 1) CS 477/677

  11. A Job Scheduling Application • Job scheduling • The key is the priority of the jobs in the queue • The job with the highest priority needs to be executed next • Operations • Insert, remove maximum • Data structures • Priority queues • Ordered array/list, unordered array/list CS 477/677

  12. Example CS 477/677

  13. PQ Implementations & Cost Worst-case asymptotic costs for a PQ with N items Insert Remove max ordered array N 1 ordered list N 1 unordered array 1 N unordered list 1 N Can we implement both operations efficiently? CS 477/677

  14. Background on Trees • Def:Binary tree = structure composed of a finite set of nodes that either: • Contains no nodes, or • Is composed of three disjoint sets of nodes: a root node, a left subtree and a right subtree root 4 Right subtree Left subtree 1 3 2 16 9 10 14 8 CS 477/677

  15. 4 4 1 3 1 3 2 16 9 10 2 16 9 10 14 8 7 12 Complete binary tree Full binary tree Special Types of Trees • Def:Full binary tree = a binary tree in which each node is either a leaf or has degree exactly 2. • Def:Complete binary tree = a binary tree in which all leaves have the same depth and all internal nodes have degree 2. CS 477/677

  16. The Heap Data Structure • Def:A heap is a nearly complete binary tree with the following two properties: • Structural property: all levels are full, except possibly the last one, which is filled from left to right • Order (heap) property: for any node x Parent(x) ≥ x 8 It doesn’t matter that 4 in level 1 is smaller than 5 in level 2 7 4 5 2 Heap CS 477/677

  17. Definitions • Height of a node = the number of edges on a longest simple path from the node down to a leaf • Depth of a node = the length of a path from the root to the node • Height of tree = height of root node = lgn, for a heap of n elements Height of root = 3 4 1 3 Height of (2)= 1 Depth of (10)= 2 2 16 9 10 14 8 CS 477/677

  18. Array Representation of Heaps • A heap can be stored as an array A. • Root of tree is A[1] • Left child of A[i] = A[2i] • Right child of A[i] = A[2i + 1] • Parent of A[i] = A[ i/2 ] • Heapsize[A] ≤ length[A] • The elements in the subarray A[(n/2+1) .. n] are leaves • The root is the maximum element of the heap A heap is a binary tree that is filled in order CS 477/677

  19. Heap Types • Max-heaps (largest element at root), have the max-heap property: • for all nodes i, excluding the root: A[PARENT(i)] ≥ A[i] • Min-heaps (smallest element at root), have the min-heap property: • for all nodes i, excluding the root: A[PARENT(i)] ≤ A[i] CS 477/677

  20. Operations on Heaps • Maintain the max-heap property • MAX-HEAPIFY • Create a max-heap from an unordered array • BUILD-MAX-HEAP • Sort an array in place • HEAPSORT • Priority queue operations CS 477/677

  21. Operations on Priority Queues • Max-priority queues support the following operations: • INSERT(S, x): inserts element x into set S • EXTRACT-MAX(S): removes and returns element of S with largest key • MAXIMUM(S): returns element of S with largest key • INCREASE-KEY(S, x, k): increases value of element x’s key to k (Assume k ≥ x’s current key value) CS 477/677

  22. Maintaining the Heap Property • Suppose a node is smaller than a child • Left and Right subtrees of i are max-heaps • Invariant: • the heap condition is violated only at that node • To eliminate the violation: • Exchange with larger child • Move down the tree • Continue until node is not smaller than children CS 477/677

  23. Maintaining the Heap Property Alg:MAX-HEAPIFY(A, i, n) • l ← LEFT(i) • r ← RIGHT(i) • ifl ≤ n and A[l] > A[i] • thenlargest ←l • elselargest ←i • ifr ≤ n and A[r] > A[largest] • thenlargest ←r • iflargest  i • then exchange A[i] ↔ A[largest] • MAX-HEAPIFY(A, largest, n) • Assumptions: • Left and Right subtrees of i are max-heaps • A[i] may be smaller than its children CS 477/677

  24. A[2] violates the heap property A[4] violates the heap property Heap property restored Example MAX-HEAPIFY(A, 2, 10) A[2]  A[4] A[4]  A[9] CS 477/677

  25. MAX-HEAPIFY Running Time • Intuitively: • A heap is an almost complete binary tree  must process O(lgn) levels, with constant work at each level • Running time of MAX-HEAPIFY is O(lgn) • Can be written in terms of the height of the heap, as being O(h) • Since the height of the heap is lgn CS 477/677

  26. Alg:BUILD-MAX-HEAP(A) n = length[A] fori ← n/2downto1 do MAX-HEAPIFY(A, i, n) Convert an array A[1 … n] into a max-heap (n = length[A]) The elements in the subarray A[(n/2+1) .. n] are leaves Apply MAX-HEAPIFY on elements between 1 and n/2 1 4 2 3 1 3 4 5 6 7 2 16 9 10 8 9 10 14 8 7 Building a Heap A: CS 477/677

  27. 1 1 1 1 4 16 4 4 2 2 3 3 2 3 2 3 14 1 10 3 1 3 1 3 4 4 5 5 6 6 7 7 4 5 6 7 4 5 6 7 8 2 16 7 9 9 10 3 2 16 9 10 14 16 9 10 8 8 9 9 10 10 8 9 10 8 9 10 14 2 8 4 1 7 14 8 7 2 8 7 1 1 4 4 2 3 2 3 1 10 16 10 4 5 6 7 4 5 6 7 14 16 9 3 14 7 9 3 8 9 10 8 9 10 2 8 7 2 8 1 Example: A i = 5 i = 4 i = 3 i = 2 i = 1 CS 477/677

  28. 1 4 2 3 1 3 4 5 6 7 2 16 9 10 8 9 10 14 8 7 Correctness of BUILD-MAX-HEAP • Loop invariant: • At the start of each iteration of the for loop, each node i + 1, i + 2,…, n is the root of a max-heap • Initialization: • i = n/2: Nodes n/2+1, n/2+2, …, n are leaves  they are the root of trivial max-heaps CS 477/677

  29. 1 4 2 3 1 3 4 5 6 7 2 16 9 10 8 9 10 14 8 7 Correctness of BUILD-MAX-HEAP • Maintenance: • MAX-HEAPIFY makes node i a max-heap root and preserves the property that nodes i + 1, i + 2, …, n are roots of max-heaps • Decrementing i in the for loop reestablishes the loop invariant • Termination: • i = 0  each node 1, 2, …, n is the root of a max-heap (by the loop invariant) CS 477/677

  30. Running Time of BUILD MAX HEAP  Running time: O(nlgn) • This is not an asymptotically tight upper bound Alg:BUILD-MAX-HEAP(A) • n = length[A] • fori ← n/2downto1 • do MAX-HEAPIFY(A, i, n) O(n) O(lgn) CS 477/677

  31. Running Time of BUILD MAX HEAP • HEAPIFY takes O(h) the cost of HEAPIFY on a node i is proportional to the height of the node i in the tree Height Level No. of nodes h0 = 3 (lgn) i = 0 20 i = 1 h1 = 2 21 i = 2 h2 = 1 22 h3 = 0 i = 3 (lgn) 23 hi = h – i height of the heap rooted at level i ni = 2i number of nodes at level i CS 477/677

  32. Readings • Chapter 6 • Chapter 8 CS 477/677

More Related