CS200: Algorithm Analysis

CS200: Algorithm Analysis

RANDOMIZED ALGORITHMS General Quicksort: Quicksort(A,p,r) if p < r then q = Partition(A,p,r) Quicksort(A,p,q-1) Quicksort(A,q+1,r) • Partition subdivides array so that the lower sub-array is <= the pivot element while the upper sub-array is > pivot. • Runtime : Worst-case = ? Occurs when array is?

Best-case = ?Occurs when ? • Average-case = ? Occurs when all permutations of input are equally likely. • How can we avoid the worst-case runtime for Quicksort?

RANDOM METHODOLOGY • impose a random distribution on inputs so that all permutations are equally likely. • the run-time thereby becomes independent of the input order. • no specific input elicits worst-case behavior. • worst-case is determined only by the output of the random number generator.

Randomized Quicksort: RandomizedPartition(A,p,r) i = Random(p,r) swap(A[i],A[r]) return(Partition(A,p,r) RandomizedQuicksort(A,p,r) if p < r then q = RandomizedPartition(A,p,r) RandomizedQuicksort(A,p,q-1) RandomizedQuicksort(A,q+1,r)

Average case runtime recurrence for randomized quicksort T(n) = 1/n( Sn-1[T(k) +T(n-1-k)]) + Q(n) k=0 Notice that for k = 0, 1, 2, 3,…, n-1, each term T(q) of the sum appears once as T(k) and once as T(n-1-k). These terms can be collapsed to get = 2/n( Sn-1[T(k)]) + Q(n) = O(nlogn) k=1 Note: 2/n is dropped for k = 0. Please refer to text for solution to recurrence (on pages 156-158). Solution requires an iterated sum. Not responsible for this solution but it does rely on next topic, which you should understand.

PROBABILITY THEORY- read Appendix C2 and C3 • In order to do analysis of random algorithms some basic concepts of probability theory must be understood. • PROBABILITY: is defined over a sample space, S, that is a set of elementary events (elements). • i.e. S = {the set of 36 ways 2 dice can fall}, where each roll outcome is an elementary event. • What is the sample space of flipping 2 coins? • What is the sample space for choosing 1 letter at random from the word DIVIDE? • What is the sample space for choosing 1 jelly bean at random from a jar containing 5 red, 7 blue and 2 green jelly beans? • each elementary event is viewed as a possible outcome of an experiment where event A, is a subset of S.

i.e. A = {(1,6) (6,1) (3,4) (4,3) (2,5) (5,2)}, then A is the event of rolling a sum of 7 with 2 dice. A probability distribution, Pr{} on S is a mapping from events of S to the real numbers ST the following probability axioms are satisfied. 1. Pr{A} >= 0 for any event A, where Pr{A} is the probability of event A occurring. i.e. A = {(1,1)}, where A is the event of rolling a sum of 2 with 2 dice, then Pr{A}= 1/36 or |A|/36. 2. Pr{S} = 1, the probability of a CERTAIN event is taken as 1 for convenience. 3. Pr{A  B}= Pr{A} + Pr{B}; for any two mutually exclusive events, that is, A and B do not intersect.

i.e. A = rolling a sum of 2, B = rolling a sum of 7, then Pr{A  B}= 1/36 + 6/36 = 7/36. Theorem : For any 2 events A and B that are not mutually exclusive: Pr{A B} = Pr{A} + Pr{B} - Pr{A  B} <= Pr{A} + Pr{B} i.e. Pr{roll sum of 4 or doubles} where Pr{roll 4} = {(2,2) (1,3) (3,1)} and Pr{roll doubles} = {(1,1) (2,2) (3,3) (4,4) (5,5) (6,6)} thus Pr{roll sum of 4 or doubles} = 3/36 + 6/36 - {(2,2)} = 8/36

15 minute activity – Probability Worksheet

Probability Distributions: A probability distribution is discrete if S is a finite or countable infinite sample space. Let S be a sample space then for any event A Pr{A} = S (Pr{s}) s e A i.e. if A = {roll doubles} then s= {(2,2)} is an element of A = { (1,1) (2,2) (3,3) (4,4) (5,5) (6,6)} {(2,2)} = s , Pr{s} = 1 / S => uniform probability distribution on S. and Pr{A} =1/36 + 1/36 + 1/36 + 1/36 +1/36 + 1/36

DISCRETE RANDOM VARIABLES A discrete random variable C is a function from a finite or countable infinite sample space to the real numbers. • X –> S x R, X is a mapping from a sample space S to the real numbers. X associates a real number x with each event in S. • for a random variable X and a real number x, the event X = x is defined as {s e S : X(s) = x} • i.e. Roll 2 dice then |S| = 36 possible outcomes. • Each element s of S has a uniform distribution, that is, each element has a probability of occurrence =1/|S| = 1/36. • Let X be the sum of dice, then Pr{X=5} = 4/36; {(1,4) (4,1) (2,3) (3,2)}

Expected Value of a Random Variable (average of the values taken on by a discrete random variable) • expectation, mean, average are synonymous. The expected value of a discrete random variable X is  E[X] = S (x * Pr{X=x}) x = 0

i.e. X = sum of dice. Thus E[X] = 252/36 = 7 the average value of X is 7. Note : E[X] = S (x * Pr{X=x}) x = 0 Sum of dice(x) Pr{X=x} x* Pr{X=x} 1 0/36 0/36 2 1/36 2/36 3 2/36 6/36 4 3/36 12/36 5 4/36 20/36 6 5/36 30/36 7 6/36 42/36 8 5/36 40/36 9 4/36 36/36 10 3/36 30/36 11 2/36 22/36 12 1/36 12/36

Apply Expectation to average case analysis of Linear Search Assume searchVal is in A. Sample space is number of comparisons needed to find searchVal. What is X = x? What is the Pr{X=x}? Construct table? Expectation of comparisons? Number of comparisons (x) Pr{X=x} x* Pr{X=x} 1 1/n 1/n 2 1/n 2/n 3 1/n 3/n 4 1/n 4/n 5 1/n 5/n . . . . . . n 1/n n/n What is E[ S (x * Pr{X=x})] ? x = 1 1/n(Snx ) = (n(n+1)/2n) = (n+1)/2 x=1

Expectations of discrete random variables are linear: for any 2 random variables X and Y and any constant a, E[aX + Y] = aE[X] + E[Y] • Let Z = number on die 1 and Y = number on die 2 and let X = sum of dice then X = Z + Y. • E[Z] = E[Y] = (x) Prob x*Prob 1 1/6 1/6 2 1/6 2/6 3 1/6 3/6 4 1/6 4/6 5 1/6 5/6 6 1/6 6/6 21/6 = 3.5 • E[X] = E[Z] + E[Y] = 3.5 + 3.5 = 7

Independence • If two random variables X and Y are independent then for all x,y: Pr{X=x and Y=y} = Pr{X=x}*Pr{Y=y} • if X and Y are independent then E[XY] = E[X]*E[Y] • i.e. X = 5, Y = 6, dice values • Pr{X=5 and Y=6}= 1/6 * 1/6 = 1/36 (one of 36 possibilities).

Summary • Know how to avoid worst case run time of QuickSort • Understand how average runtime recurrence for QuickSort is constructed. • Know basic probability theorems and their application • Be able to apply expectation, E[X], to compute average case runtime for simple algorithms

CS200: Algorithm Analysis