Efficiency of Algorithms: Logarithmic Orders Binary Search Algorithm

1. Efficiency of Algorithms:Logarithmic OrdersBinary Search Algorithm

2. Logarithmic function • Definition: The logarithmic function with base b (b>0, b1) is the following function from R+ to R: logb(x) = the exponent to which b must raised to obtain x . Symbolically, logbx = y  by = x . • Property: If the base b>1, then the logarithmic function is increasing: if x1<x2 , then logb(x1) < logb(x2) . Note: Logarithmic function grows very slowly, e.g., log2(1,024)=10, log2(1,048,576)=20 .

3. A property of logarithmic function • Proposition 1:Ifk is an integer and x is a real number with 2k x < 2k+1, thenlog2x = k . • Proof: 2k x < 2k+1 log2(2k)  log2(x) < log2(2k+1) (since log2x increasing) k log2(x) < k+1 (by definition of log2x) log2x = k (by definition of floor function) ■

4. An application of logarithms • Question: Given a positive integer n, how many binary digits are needed to represent n? • Solution: Binary representation of n: 1ck-1ck-2…c2c1c0 which corresponds to n = 2k + ck-1∙2k-1 + … + c2∙22 + c1∙2 + c0 Since ci 1, n = 2k + ck-1∙2k-1 + … + c2∙22 + c1∙2 + c0  2k + 2k-1 + … + 22 + 2 + 1 = (2k+1-1) / (2-1) (as a sum of geometric sequence) = 2k+1-1 <2k+1(1) On the other hand, n = 2k + ck-1∙2k-1 + … + c2∙22 + c1∙2 + c0 ≥ 2k(2) Combining (1) and (2):2k n < 2k+1(3) Based on (3) and Proposition 1, k = log2n and the number of binary digits is log2n + 1.

5. Exponential and Logarithmic Orders • For all real numbers b and r with b>1 and r>0 andfor all sufficiently largevalues of x,  logbx  xr; ( which implies that logbx is O(xr))  xr bx ( which implies that xr is O(bx) ) • For all real numbers b with b>1 andfor all sufficiently largevalues of x, x  x logbx  x2 (which implies that xis O(x logbx) and x logbxis O(x2) )

6. Binary Search Algorithm • The algorithm searches for an element x in an ascending array of elements a[1],…,a[n] . • Algorithm body: index:=0, bot:=1, top:=n while (top ≥ botandindex=0) mid :=  (bot+top) / 2 if a[mid] = xthen index := mid if a[mid] > x thentop := mid-1 elsebot := mid+1 end while Output: index (Ifindex has the value 0 thenx is not in the array; otherwise, index gives the index of the array where x is located.)

7. Binary Search Algorithm: Example • Suppose a[1]=Amy, a[2]=Bob, a[3]=Dave, a[4]=Erin, a[5]=Jeff, a[6]=John, a[7]=Larry, a[8]=Mary, a[9]=Mike, a[10]=Sam, a[11]=Steve, a[12]=Tom. (sorted in alphabetical order) • Search for x=Erin. • The table tracing the binary search algorithm:

8. The Efficiency of the Binary Search Algorithm • At each iteration, the length of the new subarray to be searched is approximately half of the previous one. • If n = 2k+m, where 0  m < 2k, then n can be split approximately in half k times. • Since 2k  n < 2k+1, then k = log2n(by proposition 1) • Thus, the number of iterations of the while loop in a worst-case execution of the algorithm is log2n+1. • The number of operations in each loop is constant (doesn’t increase with n). • Thus, the binary search algorithm is O(log2n) .