Lower Bound on Comparison-based Search

1. Lower Bound on Comparison-based Search • We have now covered lots of searching methods • Contiguous Data (Arrays) • Sequential search • Binary Search • Dynamic Data (Linked Lists and Trees) • Sequential search on linked lists • Search Trees: BST, AVL tree, Splay Tree, B Trees, … • Question: • How fast can a searching algorithm be made using only comparison of keys? • That is, what’s the lower bound on comparison-based searching?

2. Comparison (Decision) Trees • A comparison tree (also called decision tree or search tree) of an algorithm is obtained by tracing through the actions of the algorithm • Represent each key comparison by a vertex of the tree, which we denote by a circle • Put the key against which we are comparing the targetinside the vertex • Branches (lines) drawn down a vertex represent possible outcomes of the comparison and are labeled accordingly

3. Comparison (Decision) Trees • When the algorithm terminates, we put either • F (for failure) OR • The location where the target is found at the end of the appropriate branch, which we call a leaf and denote by a square

4. Sequential Search – Flashback // Return the index of the array containing the key or –1 if key not found // intLinearSearch(int A[], intnoOfKeys, int key){ for (int k=0; k < noOfKeys; k++){ if (A[k] == key) return k; // Key found. Return the index of the array } //end-for return –1; // Key not found }

5. A[0] A[1] A[2] A[3] A[4] A[N] A[N] A[2] A[3] A[0] A[4] A[1] F Comparison Tree for Sequential Search • The # of steps for the algorithm to terminate in a particular search is the # of internal (circular) vertices traversed going from the top of the tree down the appropriate path to a leaf = = = = = Height = N Means that in the worst case the algorithm will take O(N) steps to terminate. So its running time is O(N) =

6. Binary Search - Flashback // Return the index of the array containing the key or –1 if key not found int BinarySearch(int A[], int noOfKeys, int key){ left = 0; right = noOfKeys-1; while (left <= right){ int middle = (left+right)/2; // Index of the key to test against if (A[middle] == key) return middle; // Key found. Return the index else if (key < A[middle]) right = middle – 1; // Eliminate the right side else left = middle+1; // Eliminate the left side } //end-while return –1; // Key not found } //end-BinarySearch

7. A[6] A[3] A[1] A[5] A[0] A[2] A[4] A[6] A[0] A[3] A[1] A[5] A[4] A[2] 50 41 22 55 78 20 10 right left F F F F F F F F Comparison Tree for Binary Search 5 4 6 1 3 0 2 A: > < = Height = log2(7) = 3 < < > > = = < > < < > > < = > = = =

8. A[middle] A[middle] Comparison Tree for Binary Search Height = log2(N) < > = Target greater than key in the middle: Search (left=middle+1, right) Target less than key in the middle: Search (left, right = middle-1) Since we divide the data set in half with each comparison, the maximum height of the comparison treewill be log2N. So the running time of binary search is O(log2N).

9. Lower Bound on Comparison-based Algorithms • Theorem: • Suppose that an algorithm uses ONLY comparison of keys to solve a problem. • If there are “n” possible outcomes, then the algorithm must take AT LEASTlog2Ncomparison of keys in the worst case to solve the problem.

10. C C C C C C C C C O4 O1 O6 O5 O2 On-1 O3 O6 O5 On Lower Bound on Comparison-based Algorithms – Informal Proof • Notice that the comparison tree that would result in the smallest height is a complete binary tree. • The height of a complete binary tree of Nleaves is log2N • This completes the informal proof. Height = log2(N) …………………. ……………………………………………. …………………

11. Lower Bound on Comparison Based Search • Let’s take comparison-based searching problem: • Search for a key in a data set consisting of N keys using only comparison of keys • How many possible outcomes? • N + 1: N successes, 1 failure • Then, the running time of ANY comparison-based search algorithm is (log2N)