1 / 25

Randomized Algorithms

Jan 12, WSAC 2010. Randomized Algorithms. Kyomin Jung KAIST Applied Algorithm Lab. Randomness in Algorithms. A Probabilistic Turing Machine is a TM given with a (binary) random “coin”. In the computation process of the PTM, PTM can toss a coin to decide its decision.

emma-blackwell
Download Presentation

Randomized Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Jan 12, WSAC 2010 Randomized Algorithms Kyomin Jung KAIST Applied Algorithm Lab

  2. Randomness in Algorithms • A Probabilistic Turing Machine is a TM given with a (binary) random “coin”. • In the computation process of the PTM, PTM can toss a coin to decide its decision. • Ex) Computer games, random screen savor… • Note: in computer programming, the random number generating function (ex, rand() in C) takes an initial random seed, and generates a “deterministic” sequence of numbers. Hence they are not “truly” random.

  3. Why randomness can be helpful? • A Simple example • Suppose we want to check whether an integer set has an even number or not. • Even when A has n/2 many even numbers, if we • run aDeterministic Algorithm, it may check n/2+1 many elements in the worst case. • A Randomized Algorithm: At each time, choose an elements (to check) at random. • Smooths the “worst case input distribution” into “randomness of the algorithm”

  4. Property of “independent” random trials • Law of Large Number • If we repeat independent random samplings many times according to a fixed distribution D, the “the average becomes close to the expectation” (ex: dice rolling) • Ex) Erdos-Renyi Random Graph G(n,p). Voting poll …

  5. Chernoff Bound • Suppose we have a coin with probability of heads is p. • Let X be a random variable where X=1 if the coin flip gives heads and X=0 otherwise. • E[X] = 1*p + 0*(1-p) = p • After flipping a coin w times we can estimate the heads prob by average of xi. • The Chernoff Bound tells us that this estimate converges exponentially fast to the true mean p.

  6. Las Vegas vs Monte Carlo • A Las Vegas algorithm is a randomized algorithm that always gives correct results • Ex: randomized quick sort • A Monte Carlo algorithm is one whose running time is deterministic, but whose output may be correct only with a certain probability. • Class BPP(Bounded-error, Probabilistic, Polynomial time) • problem which has a Monte Carlo algorithm to solve it

  7. Randomized complexity classes • Model: probabilistic Turing Machine • BPP • L BPP if there is a p.p.t. TM M: x  L  Pry[M(x,y) accepts] ≥ 2/3 x  L  Pry[M(x,y) rejects] ≥ 2/3 • “p.p.t” = probabilistic polynomial time • y : (a sequence of) random coin tosses

  8. Randomized complexity classes • RP(Random Polynomial-time) • L RP if there is a p.p.t. TM M: x  L  Pry[M(x,y) accepts] ≥ ½ x  L  Pry[M(x,y) rejects] = 1 • coRP(complement of Random Polynomial-time) • L coRP if there is a p.p.t. TM M: x  L  Pry[M(x,y) accepts] = 1 x  L  Pry[M(x,y) rejects] ≥ ½

  9. Error reduction for BPP • given L, and p.p.t. TM M: x  L  Pry[M(x,y) accepts] ≥ ½ + ε x  L  Pry[M(x,y) rejects] ≥ ½ + ε • new p.p.t. TM M’: • simulate M k/ε2 times, each time with independent coin flips • accept if majorityof simulations accept • otherwise reject

  10. Error reduction for BPP • Xi random variable indicating “correct” outcome in i-th simulation (out of m = k/ε2 ) • Pr[Xi = 1] ≥ ½ + ε • Pr[Xi = 0] ≤ ½ - ε • E[Xi] ≥ ½+ε • X = ΣiXi • μ = E[X] ≥ (½ + ε)m • By Chernoff Bound: Pr[X ≤ m/2] ≤ 2-(ε2m)

  11. Derandomization • Question: is BPP=P? • Goal: try to simulate BPP in polynomial time • use Pseudo-Random Generator (PRG): • Good if t=O(log m) • Blum-Micali-Yao PRG seed lengtht = mδ G seed output string t bits m bits

  12. Example of randomized algorithm : Minimum Cut Problem C A B D Note: Size of the min cut must is no larger than the smallest node degree in graph

  13. Application: Internet Minimum Cut

  14. Randomized Algorithm (by David Karger) • While |V| > 2: • Pick a random edge (x, y) from E • Contract the edge: • Keep multi-edges, remove self-loops • Combine nodes • The two remaining nodes represent reasonable choices for the minimum cut sets

  15. Analysis • Suppose Cis a minimum cut (set of edges that disconnects G) • When we contract edge e: • Unlikely that e  C • So, C is likely to be preserved What is the probability a randomly chosen edge is in C?

  16. Random Edge in C? • |C| must be  degree of every node in G • How many edges in G: |E| = sum of all node degrees / 2  n |C| / 2 Probability a random edge is in C  2/n

  17. Iteration • How many iterations? • Probability for first iteration: Prob(e1 C)  1 – 2/n • Probability for second iteration: Prob(e2 C | e1 C)  1 – 2/(n-1) • ... • Probability for last iteration: Prob(en-2 C |…)  1 – 2/(n-(n-2-1))  1 – 2/3 n - 2

  18. Probability of finding C? Probability of not finding C = 1 – 2/(n(n-1))

  19. Probability of finding C? Probability of not finding C on one trial:  1 – 2/(n(n-1))  1 – 2/n2 Probability of not finding C on k trials:  [1 – 2/n2]k If k = cn2, Prob failure  (1/e)c Recall: lim (1 – 1/x)x = 1/e x 

  20. Example of randomized algorithm :Random Sampling • What is a random sampling? • Given a probability distribution , pick a point according to . • e.g. Monte Carlo method for integration • Choose numbers uniformly at random from the integration domain, and sum up the value of f at those points

  21. How to use Random Sampling? • Volume computation in Euclidean space.

  22. Hit and Run • Hit and Run algorithm is used to sample from a convex set in an n-dimensional Eucliden space. • Assume that we can evaluate its ratios at given points • It converges in time.

  23. Example of randomized algorithm :Markov Chain Monte Carlo • Let be a probability density function on S={1,2,..n}. • f(‧) is any function on S and we want to estimate • Construct P={Pij}, the transition matrix of an irreducible Markov chain with states S, where and π is its unique stationary distribution.

  24. Markov Chain Monte Carlo • Run this Markov chain for times t=1,…,N and calculate the Monte Carlo sum then

  25. Applied Algorithm Lab (http://aa.kaist.ac.kr/) Research Projects • Markov Random Field and Image segmentation • Network consensus algorithm design • Community detection in complex networks • Routing/scheduling algorithm analysis in wireless networks • Motion surveillance in video images • Character recognition in images • SAT solver design based on formula structures • Decentralized control algorithms for multi-agent robot system • Financial data mining Members Professor Graduate Students Kyomin Jung Nam-ju Kwak Yongsub Lim Boyoung Kim

More Related