Extracting Randomness From Few Independent Sources

1. Extracting Randomness From Few Independent Sources Boaz Barak, IASRussell Impagliazzo, UCSDAvi Wigderson, IAS

2. Plan: 1. Discuss problem and model 2. State our result 3. Introduce main tool – Thm by [BKT,K] 4.* Prove our main theorem.

3. Randomness Extraction Randomness is central to CS (c.f., randomized algorithms, cryptography, distributed computing) How do you execute randomized algorithms and protocols? Solution: sample some “random” physical data (coin tossing, thermal noise, hard disk movement,…) Problem: data from physical sources is not a sequence of ideal coin tosses.

4. Randomness Extractors “Definition”: E:{0,1}n{0,1}0.1k is an extractorif 8 r.v. X with entropy ¸k , E(X) is close to U0.1k Idea: X E randomized algorithm / protocol extractor uniform output high entropy data

5. Randomness Extractors “Definition”: E:{0,1}n{0,1}0.1k is an extractorif 8 r.v. X with entropy ¸k , E(X) is close to U0.1k Problem:No extractor exists. Thm: 8E:{0,1}n{0,1}0.1k there’s a r.v. X w/ entropy ¸n-1 s.t. first bit of E(X) is constant. Proof Sketch: Assume wlog |{ x | E1(x)=0 }| ¸ 2n/2 let X be the uniform dist over this set.

6. X has min-entropy¸k (denoted H(X)¸k) if 8x Pr[ X=x ] · 2-k. Every such dist is convex comb of “flat” dist – uniform dist on set of size ¸2k. In this talk: entropy = min-entropy Solution 1: Seeded Extractors Def: E:{0,1}n£{0,1}d{0,1}0.1k is a (seeded) extractorif 8 r.v. X w/ min-entropy ¸k| E(X,Ud) – U0.1k|1 < 1/100 . “Definition”: E:{0,1}n{0,1}0.1k is an extractorif 8 r.v. X with entropy ¸k , E(X) is close to U0.1k Many exciting results, applications and connections [Z,NZ,Ta,Tr,RSW,STV,TSZ,SU,…]. Thm [LRVW]: For every n,k there’s a seeded extractor with d=O(log n) Corollary: Any probabilistic algorithm can be simulated w/ weak random source + polynomial overhead.

7. Solution 1: Seeded Extractors Thm [LRVW]: For every n,k there’s a seeded extractor with d=O(log n) Corollary: Any probabilistic algorithm can be simulated w/ weak random source + polynomial overhead. Question: What about other uses of randomness? For example, can we use this for cryptography? Answer: No! For example, if we concatenate encryptions according to all possible seeds this won’t be secure! Need to use seedless extractors!

8. 2-(k) Seedless Extractors Idea: Bypass impossibility result by making additional assumption on the high entropy input. Long history and many results [vN,P,B,SV,CW,TV,KZ,..] In this work: We assume that input comes from few independent distributions ([CG]). Def: E:{0,1}nc{0,1}0.1k is a c-sampleextractorif 8 ind. r.v. X1,…,Xc w/ min-entropy ¸k| E(X1,…,Xc) – U0.1k|1 < 1/100 Motivation:mathematically clean and plausible model.

9. Def: E:{0,1}nc{0,1}0.1k is a c-sampleextractorif 8 ind. r.v. X1,…,Xc w/ min-entropy ¸k| E(X1,…,Xc) – U0.1k|1 < 2-(k) Optimal (non-explicit) construction: c=2 , every k¸(log n) Previous best explicit construction [SV,V,CG,ER,DEOR]: c=2 , every k¸(1+)n/2 Obtained by variants of following 1-bit output extractor:E(x,y) = <x,y> Problematic, since natural entropy sources often have entropy less than n/2.

10. Main Thm: 8>09c=poly(1/) and poly-timeE:{0,1}nc{0,1}n s.t. if 8 ind. r.v. X1,…,Xc w/ min-entropy ¸n| E(X1,…,Xc) – Un|1 < 2-(n) Def: E:{0,1}nc{0,1}0.1k is a c-sampleextractorif 8 ind. r.v. X1,…,Xc w/ min-entropy ¸k| E(X1,…,Xc) – U0.1k|1 < 2-(k) Optimal (non-explicit) construction: c=2 , every k¸(log n) Previous best explicit construction [SV,V,CG,ER,DEOR]: c=2 , every k¸(1+)n/2 Our Result:For every>0c=poly(1/) , k=n

11. Main Thm: 8>09c=poly(1/) and poly-timeE:{0,1}nc{0,1}n s.t. if 8 ind. r.v. X1,…,Xc w/ min-entropy ¸n| E(X1,…,Xc) – Un|1 < 2-(n) Plan: 1. Discuss problem and model 2. State our result 3. Introduce main tool – Thm by [BKT,K] Show BKT (almost) immediately implies dispersers. 4. Prove our main theorem.

12. Main Thm: 8>09c=poly(1/) and poly-timeE:{0,1}nc{0,1}n s.t. if 8 ind. r.v. X1,…,Xc w/ min-entropy ¸n| E(X1,…,Xc) – Un|1 < 2-(n) Our main tool is the following result: Thm 1 [BKT,K]: 9 absolute constant >0 s.t. for prime field F, and set AµF, max{ |A+A| , |A ¢ A| } ¸ min{ |A|1+ , |F| } 1. Finite field analog of a theorem by [ES].2. Note Thm 1 would be false if F had non-trivial subfields.3. Note if A is arithmetic (resp. geometric) sequence, then |A+A| (resp. |A¢ A|) is small. A+A = { a+b | a,b 2 A }A ¢ A = { a¢b | a,b 2 A }

13. Thm 1 [BKT,K]: 9 absolute constant >0 s.t. for prime field F, and set AµF, max{ |A+A| , |A ¢ A| } ¸ |A|1+ How is this related to extractors? Disperser Lemma [BKT]:Let >0 and F a prime field, then9c=poly(1/) and poly-time E:FcF s.t. if X1,…,XcµF satisfy |Xi|¸ |F|, then E(X1,…,Xc) = F Corollary: Identify {0,1}n w/ prime field F of size 2n. Then, we get poly-time E s.t. if r.v.’s X1,…,Xc have entropy ¸n, then Supp{E(X1,…,Xc)}={0,1}nThis is called a disperser.

14. Thm 1 [BKT,K]: 9 absolute constant >0 s.t. for prime field F, and set AµF, max{ |A+A| , |A ¢ A| } ¸ |A|1+ Thm 1 [BKT,K]: 9 absolute constant >0 s.t. for prime field F, and sets A,B,CµF, (with |A|=|B|=|C|) |A¢B+C| ¸ |A|1+ How is this related to extractors? Disperser Lemma [BKT]:Let >0 and F a prime field, then9c=poly(1/) and poly-time E:FcF s.t. if X1,…,XcµF satisfy |Xi|¸ |F|, then E(X1,…,Xc) = F Proof: Use lemma of Rusza to get “asymmetric” version of Thm 1. Lemma [R,N]: If A,B µG w/ |A|=|B|=M, and |A­B| · M1+, then |A­A| · M1+O() We let E be recursive application of a,b,ca¢b+c with depth O(log(1/)). |A ¢ A| large ) |A ¢ B| large) |A¢B+C| large|A+A| large) |A+C| large) |A¢B+C| large

15. ¢ ¢ + + ¢ ¢ ¢ ¢ ¢ ¢ + + + + + + Thm 1 [BKT,K]: 9 absolute constant >0 s.t. for prime field F, and sets A,B,CµF, (with |A|=|B|=|C|) |A¢B+C| ¸ |A|1+ ¢ . + . . . . . . . . a1 , a2, … apoly(1/delta)

16. Plan: 1. Discuss problem and model 2. State our result 3. Introduce main tool – Thm by [BKT,K] Show BKT (almost) immediately implies dispersers. 4. Prove our main theorem.

17. Distributional Version of [BKT] Thm 1 [BKT,K]: 9 absolute constant >0 s.t. for prime field F, and sets A,B,CµF, (with |A|=|B|=|C|) |A¢B+C| ¸ |A|1+ Our Main Lemma: 9 absolute constant >0 s.t. for prime field F, and distributions A,B,CµF, (with H(A)=H(B)=H(C)), the distributionA¢B+C is 2-H(A)close to having entropy¸ (1+)H(A) ( The distribution A¢B+C assigns to x the prob that a¢b+c=x with a2RA , b2RB , c2RC ) Main Lemma ) Main Theorem.

18. ¢ . + . . . . . . . . ¢ ¢ + + ¢ ¢ ¢ ¢ ¢ ¢ + + + + + + a1 , a2, … apoly(1/delta) Our Main Lemma: 9 absolute constant >0 s.t. for prime field F, and distributions A,B,CµF, (with H(A)=H(B)=H(C)), the distributionA¢B+C is 2-H(A)close to having entropy¸ (1+)H(A) Main Lemma ) Main Theorem.

19. Our Main Lemma: 9 absolute constant >0 s.t. for prime field F, and distributions A,B,CµF, (with H(A)=H(B)=H(C)), the distributionA¢B+C is 2-H(A)close to having entropy¸ (1+)H(A) Plan: Prove Main Lemma by reducing to [BKT].We use “magic lemmas” of Gowers & Ruszain the reduction.

20. Our Main Lemma: 9 absolute constant >0 s.t. for prime field F, and distributions A,B,CµF, (with H(A)=H(B)=H(C)), the distributionA¢B+C is 2-H(A)close to having entropy¸ (1+)H(A) Detailed Plan: 1. Introduce collision probability – a different entropy measure. 2. Rephrase Main Lemma in terms of C.P. 3. Show naïve approach to proving, and show counterexample 4. Use Gowers’ & Rusza’s lemmas to show counterexample essentially captures all cases

21. Collision Probability cp(X) = Prx,x’X[ x= x’ ] = x px2 Fact 1: If H(X)¸k then cp(X)·2-k Fact 2: If cp(X)·2-k(1+)then is 2-k/2 close to having min-entropy at least k(1+/2). Notation:If D is r.v., then the 2-entropy of D is H2(D) = log(1/cp(D)) Fact 1 + Fact 2)H2(D) ~ H(D) Fact 3: If X is convex combination of X1,…,Xmthen cp(X) · max { cp(X1), … , cp(Xm) }

22. Main Lemma: 9>0 s.t. for prime field F, dists A,B,CµF, (with H(A)=H(B)=H(C), the distributionA¢B+C is 2-H(A)close to entropy¸ (1+)H(A) Main Lemma (CP version): 9>0 s.t. for prime field F, and sets A,B,CµF (with |A|=|B|=|C| ), the distributionA¢B+C is |A|-close to having 2-entropy ¸ (1+)log |A| Thus, it is sufficient to prove CP version.

23. Main Lemma (CP version): 9>0 s.t. for prime field F, and sets A,B,CµF (with |A|=|B|=|C| ), the distributionA¢B+C is |A|-close to having 2-entropy ¸ (1+)log |A| Detailed Plan: 1. Introduce collision probability – a different entropy measure. 2. Rephrase Main Lemma in terms of C.P. 3. Show naïve approach to proving, and show counterexample 4. Use Gower’s and Rusza’s lemmas to show counterexample essentially captures all cases

24. Naïve Approach Prove direct analog to BKT “Conjecture”: 9>0 s.t. for prime F, and set AµF max { H2(A+A) , H2(A¢ A) } ¸ (1+)log|A| Counter Example: A=AG [AAAG - geometric seq. AA- (disjoint) arithmetic seq. cp(A+A),cp(A¢A)¸1/10|A| hence H2(A+A), H2(A¢A)·log|A|+O(1) However, in this case H2(A¢ A+A) ¸ (1+)log |A|

25. Naïve Approach Counter Example: A=AG [ AAAG- geometric seq.AA- (disjoint) arithmetic seq. Claim:H2(A¢A + A) ¸ (1+)log |A| Sketch:A¢A+A is convex comb of AA ¢A+A and AG¢A+A. cp(AA¢A+A)· cp(AA¢A) which is low since A¢ is an arithmetic seq AG¢A+A is convex comb of AGa+A butcp(AGa+A) is low since AGa is a geometric seq

26. Main Lemma: 9 absolute constant >0s.t. for prime field F, and sets A,B,CµF (with |A|=|B|=|C| ), the distributionA¢B+C is |A|-close to having c.p.· |A|-(1+) Detailed Plan: 1. Introduce collision probability – a different entropy measure. 2. Rephrase Main Lemma in terms of C.P. 3. Show naïve approach to proving, and show counterexample 4. Use Gowers’ and Rusza’s lemmas to show counterexample essentially captures all cases

27. Proof of Main Lemma Main Lemma (CP version): 9 absolute constant >0s.t. for prime field F, and sets A,B,CµF (with |A|=|B|=|C| ), the distributionA¢B+C is |A|-close to having 2-entropy ¸ (1+)log |A| (Loose) Notations: Let M=|A|=|B|=|C| and fix some >0(e.g., BKT’s  divided by 100) A number ¸ M1+ is called “large”A number ¸ M1-() is called “not-too-small”A distribution D has “high 2-entropy” if H2(D) ¸ (1+)log M Our Goal:Prove that A¢B+C is close to having high 2-entropy.(i.e., it is close to having c.p. · 1/M1+)

28. Tools: Thm 1 [BKT,K]: If AµF is not too small then either |A¢A| or |A+A| is large. Lemma [R,N]: If |A­A| is large then |A­B| is large. Magic Lemma [G,BS]: Either H2(A­B) is large or9 not-too-small subsets A’µA, B’µB s.t. |A’­B’| is not large.

29. A First Distributional Analog: Cor [BKT+R]: If 9 not-too-small B s.t. |A¢B| is not large then |A+C| is large 8 not-too-small C. Proof: |A¢B| is not large )|A¢A| is not large [R] )|A+A|islarge [BKT] )|A+C| is large [R]. Natural Analog: If 9 not-too-small B s.t. H2(A¢B) is not large then H2(A+C) is large 8 not-too-small C. This is false:e.g.,A=B=C=AG[AA However, the following is true: PF Lemma: If 9 not-too-small B s.t. |A¢B| is not large then H2(A+C) is large 8 not-too-small C.

30. PF Lemma: If 9 not-too-small B s.t. |A¢B| is not large then H2(A+C) is large 8 not-too-small C. Proof: IfH2(A+C)is not large then by Gowers’s Lemma9 not-too-small A’µA, C’µC s.t. |A’+C’| is not large. By Rusza’s lemma|A’+A’| is not large )by BKT|A’¢A’|is large. Since A’µA , |A¢A| is also large )by Rusza’s lemma|A¢B| is large – contradiction! Def: A not-too-small setAµF is “plus friendly” if H2(A+C)is large 8not-too-small set C. 1. A plus-friendly, b2F)Ab plus-friendly.2. A’ , A’’ plus-friendly, disjoint )A’[A’’ plus-friendly.

31. Our Goal:Prove A¢B+C close to having “low c.p.”. Assume H2(A¢B+C) not large.We’ll show A=A+[A¢ s.t. A+,A¢are disjoint and 1) A+ is “plus friendly” (or A+ is empty) 2) H2(A¢¢B) is large (or |A¢|· M1-) 1+2) contradiction since A¢B+C is M- close to convex comb of A+¢B+C and A¢¢B+C, but a)H2(A+¢B+C) is large since convex comb of A+b+C and A+b is plus-friendly. b)H2(A¢¢B+C) is large since convex comb of A¢B+c which are permutations of A¢B.

32. Our Goal:Prove A¢B+C close to having “low c.p.”. Assume H2(A¢B+C) not large.We’ll show A=A+[A¢ s.t. A+,A¢ disjoint and 1) A+ is “plus friendly” (or A+ is empty) 2) H2(A¢¢B) is large (or |A¢|· M1-) We build partition iteratively. Initially A+=; , A¢=A. Assume A¢ is not-too-small (o/w we’re done). Assume H2(A¢¢B) is not large (o/w we’re done). By Gowers’ lemma, 9 not-too-small subsets A’µA¢, B’µBs.t. |A’¢B’| not large. By PF LemmaA’ is plus-friendly, remove A’ from A¢ and add it to A+.

33. This finishes the proof of the Main Lemma and hence the Main Theorem. Main Lemma: 9 absolute constant >0 s.t. for prime field F, and distributions A,B,CµF, (with H(A)=H(B)=H(C)<0.8log|F|), the distributionA¢B+C is 2-H(A)close to having entropy¸ (1+)H(A) Main Thm: 8>09c=poly(1/) and poly-timeE:{0,1}nc{0,1}n s.t. if 8 ind. r.v. X1,…,Xc w/ min-entropy ¸n| E(X1,…,Xc) – Un|1 < 2-(n) 2-10n

34. Another Result: A disperser for the case that all samplescome from same distribution, which only requires (log n) entropy (using [EH]).

35. Open Problems • Extractors/Dispersers with lower entropy requirement (k=n(1) or even k=(log n) ) • Improvement for the case of twosamples (related to constructing Ramsey graphs). • More applications of results/techniques.