Randomized Algorithms CS648

1. Randomized AlgorithmsCS648 Lecture 4 Linearity of Expectation with applications (Most important tool for analyzing randomized algorithms)

2. RECAP from the last lecture

3. Random variable Definition:A random variable defined over a probability space (Ω,P) is a mapping Ω  R. Examples: • The number of HEADS when a coin is tossed 5 times. • The sum of numbers seen when a dice is thrown 3 times. • The number of comparisons during Randomized Quick Sort on an array of size n. Notations for random variables : • X,Y, U, …(capital letters) • X() denotes the value of X on elementary event .

4. Expected Value of a random variable(average value) Definition: Expected value of a random variable X defined over a probability space (Ω,P) is E[X] = X= c X= b X= a Ω E[X] =

5. Examples Random experiment 1: A fair coin is tossed n times Random Variable X: The number of HEADS E[X] = = = Random Experiment 2: balls into bins Random Variable X: The number of empty bins E[X] =

6. Can we solve these problems ? Random Experiment 1balls into bins Random Variable X: The number of empty bins E[X]= ?? Random Experiment 2Randomized Quick sort onelements Random Variable Y: The number of comparisons E[Y]= ??

7. Balls into Bins(number of empty bins) 1 2 3 4 5 … m-1 m Question : X is random variable denoting the number of empty bins. E[X]= ?? Attempt 1: (based on definition of expectation) E[X] = = = A subset of bins 1 2 3 … … n This is a right but useless answer !

8. Randomized Quick Sort(number of comparisons) Question : Yis random variable denoting the number of comparisons. E[Y]= ?? Attempt 1: (based on definition of expectation) E[Y] = A recursion tree associated with Randomized Quick Sort We can not proceed from this point …

9. 1 2 3 4 5 … m-1 m Balls into Bins(number of empty bins) Randomized Quick Sort(number of comparisons) 1 2 3 … … n

10. Balls into Bins(number of empty bins) 1 2 3 4 5 … m-1 m Question: Let be a random variable defined as follows. = What is E[] ? Answer : E[] = + = = 1 2 3 … … n

11. Balls into Bins(any relation between and ’s ?) Consider any elementary event. 1 2 3 4 5 6 An elementary event 0 1 0 1 0 1 2 3 4 5

12. Sum of Random Variables Definition:Let be random variables defined over a probability space (Ω,P) such that for each ϵΩ Then is said to be the sum of random variables and . A compact notation : Definition:Let and be random variables defined over a probability space (Ω,P) such that for each ϵΩ Then is said to be the sum of random variables . A compact notation :

13. Randomized Quick Sort(number of comparisons) Elements of A arranged in Increasing order of values Question : Let , for any , be a random variable defined as follows. = What is E[] ? Answer : E[] = + = =

14. Randomized Quick Sort(any relation between and ’s ?) Consider any elementary event. Question: What is relation between and and ? Answer: Any elementary event … … 1 0 … 0 1 1 … 0 Hence

15. What have we learnt till now? Balls into Bin experiment X: random variable denoting the number of empty bins Aim:E[X]= ?? E[] = Randomized Quick Sort Y: random variable for the number of comparisons Aim:E[Y]= ?? E[] = Hence E E

16. The main question ? Let be random variables defined over a probability space (Ω,P) such that ,

17. Balls into Bins(number of empty bins) 1 2 3 4 5 … m-1 m : random variable denoting the number of empty bins. Using Linearity of Expectation [] for 1 2 3 … … n

18. Randomized Quick Sort(number of comparisons) : r. v. for the no. of comparisons during Randomized Quick Sort on elements. Using Linearity of expectation:

19. Linearity of Expectation Theorem: • (For sum of 2 random variables) Ifare random variables defined over a probability space (Ω,P) such that , then • (For sum of more than 2 random variables) If , then

20. Where to use Linearity of expectation ? Whenever we need to find E[U] but none of the following work • E[] = • E[] = In such a situation, Try to express as, such that it is “easy” to calculate . Then calculate using

21. Think over the following questions? • Letbe random variables defined over a probability space (Ω,P) such that , for some real no. , then Answer: yes (prove it as homework) • Why does linearity of expectation holds always ? (even when and are not independent) Answer: (If you have internalized the proof of linearity of expectation, this question should appear meaningless.)

22. Think over the following questions? Definition:(Product of random variables) Let be random variables defined over a probability space (Ω,P) such that for each ϵΩ Then is said to be the product of random variables and . A compact notation is • If , then Answer: No (give a counterexample to establish it.) • If and both and are independent then Answer: Yes (prove it rigorously and find out the step which requires independence)

23. Independent random variables In the previous slides, we used the notion of independence of random variable. This notion is identical to the notion of independence of events: Two random variables are said to be independent if knowing the value of one random variable does not influence the probability distribution of the other. In other words, for all ϵand ϵ .

24. Some Practice problemsas homework • Balls into bin problem: • What is the expected number of bins having exactly 2 balls ? • We toss a coin n times, what is the expected number of times pattern HHT appear ? • A stick has n joints. The stick is dropped on floor and in this process each joint may break with probability pindependent of others. As a result the stick will be break into many substicks. • What is the expected number of substicksof length 3 ? • What is the expected number of all the substicks ?

25. Problems of the next lecture

26. Fingerprinting Techniques Problem 1: Given three ⨯matrices ,, and , determine if . Best deterministic algorithm: •  ; • Verify if ? Time complexity: Randomized Monte Carlo algorithm: Time complexity: Error probability: for any .

27. Fingerprinting Techniques Problem 2: Given two large files A and B of bits located at two computers which are connected by a network. We want to determine if A is identical to B. The aim is to transmit least no. of bits to achieve it. Randomized Monte Carlo algorithm: Bits transmitted : Error probability: for any .