1 / 40

Randomness Extractors: Motivation, Applications and Constructions

Randomness Extractors: Motivation, Applications and Constructions. Ronen Shaltiel University of Haifa. Outline of talk. Extractors as graphs with expansion properties Extractors as functions which extract randomness Applications Explicit Constructions.

umika
Download Presentation

Randomness Extractors: Motivation, Applications and Constructions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Randomness Extractors: Motivation, Applications and Constructions Ronen Shaltiel University of Haifa

  2. Outline of talk • Extractors as graphs with expansion properties • Extractors as functions which extract randomness • Applications • Explicit Constructions

  3. An extractor is an (unbalanced) bipartite graph M<<N. (e.g. M=Nδ, M=exp( (log N)δ). Every vertex x on the left has D neighbors. the extractor is better when D is small. (e.g. D=polylog N) Convention: N=2n, M=2m, D=2d … {1,…,N}≈ {0,1}n E(x,1) .. E(x,D) Extractor graphs:Definition [NZ] N≈{0,1}n M≈{0,1}m D edges x

  4. (K,ε)-Extractor: ∀set Xof size K the dist. E(X,U) ε-close to uniform. =>“expansion” property: ∀set Xofsize K, |Γ)x)| ≥ (1-ε)M. Distribution versus Set size Identify X with the uniform distribution on X Extractor graphs: expansion properties N≈{0,1}n M≈{0,1}m x X Γ(X) K (1-ε)M *A distribution P is ε-close to uniform if ||P-U||1≤ 2ε=> P supports 1-εelements.

  5. X Γ(X) Extractors and Expander graphs N≈{0,1}n N≈{0,1}n N≈{0,1}n M≈{0,1}m D=2d edges X Γ(X) K K (1+δ)K (1-ε)M (1+δ)-Expander Extractor

  6. X Γ(X) Extractors and Expander graphs N≈{0,1}n N≈{0,1}n N≈{0,1}n Allows constant degree Expands sets smaller than threshold K Absolute expansion: K -> (1+δ)K Balanced graph M≈{0,1}m X Γ(X) K K (1+δ)K (1-ε)M Unbalanced graph Relative expansion: K -> (1-ε)M K/N -> (1-ε) Requires degree log N Expands sets larger than threshold K (1+δ)-Expander Extractor

  7. Outline of talk • Extractors as graphs with expansion properties • Extractors as functions which extract randomness • Applications • Explicit Constructions

  8. We have access to distributions in nature: Electric noise Key strokes of user Timing of past events These distributions are “somewhat random” but not “truly random”. Paradigm: [SV,V,VV,CG,V,CW,Z]. Randomness Extractors Assumption for this talk: Somewhat random = uniform over subset of size K. Randomness Extractor The initial motivation: running probabilistic algorithms with “real-life” sources Successful Paradigm in CS: Probabilistic Algorithms. Probabilistic Algorithms/Protocols: Use an additional input stream of independent coin tosses. Helpful in solving computational problems. Where can we get random bits? Somewhat random random coins Probabilistic algorithm input output

  9. seed Y Extractor random output Extractors as functions that use few bits to extract randomness • We allow an extractor to also receive an additional input of (very few) random bits. • Extractors use few random bits to extract many random bits from arbitrary distributions which “contain” sufficient randomness. Parameters: (function view) • Source length: n (= log N) • Seed length: d ~ O(log n) • Entropy threshold: k ~ n/100 • Output length: m ~ k • Required error: ε~ 1/100 source distribution X Randomness Definition: A (K,ε)-extractor is a function E(x,y) s.t. For every set. X of size K, E(X,U) is ε-close* to uniform. Lower bounds [NZ,RT]: seed length (in bits) ≥ log n Probabilistic method [S,RT]: Exists optimal extractor which matches lower bound and extractsall the k=log K random bits in the source distribution. Explicit constructions: E(x,y) can be computed in poly-time.

  10. Randomness Extractor seed Simulating probabilistic algorithms using weak random sources Goal: Run prob algorithm using a somewhat random distribution. Where can we get a seed? Idea: Go over all seeds. • Given a source element x. • ∀y compute zy= E(x,y) • Compute Alg(input,zy) • Answer majority vote. Seed=O(logn) => poly-time Explicit constructions. Somewhat random random coins Probabilistic algorithm input output

  11. Outline of talk • Extractors as graphs with expansion properties • Extractors as functions which extract randomness • Applications • Explicit Constructions

  12. Applications • Simulating probabilistic algorithms using weak sources of randomness [vN,SV,V,VV,CG,V,CW,Z]. • Constructing Graphs (Expanders, Super-concentrators) [WZ]. • Oblivious sampling [S,Z]. • Constructions of various pseudorandom generators [NZ,RR,STV,GW,MV]. • Distributed algorithms [WZ,Z,RZ]. • Cryptography [CDHK,L,V,DS,MST]. • Hardness of approximations [Z,U,MU]. • Error correcting codes [TZ].

  13. Expanders that beat the eigenvalue bound [WZ] Goal: Construct low deg expanders with huge expansion. Line up two low degree extractors. ∀set X ofsize K, |Γ)x)| ≥ (1-ε)M > M/2. ∀sets X,X’ ofsize K X and X’ havecommon neighbour. • Contract middle layer. • Low degree (ND2/K) bipartite graph in which every set of size K sees N-K vertices. • Better constructions for large K [CRVW]. N≈{0,1}n N≈{0,1}n X’ X

  14. Random walk variables v1..vD behave like i.i.d: ∀A of size ½M Hitting property: Pr[∀i: vi∊A]≤δ= 2-Ω(D). Chernoff style property: Pr[#i : vi∊A far from exp.] ≤ 2-Ω(D). # of random bits used for walk: m+O(D)=m+O(log(1/δ)) # of random bits for i.i.d. m∙D=m ∙O(log(1/δ)) Randomness efficient (oblivious) sampling using expanders Random walk on constant degree expander M≈{0,1}m v2 v3 v1 vD

  15. Given parameters m,δ: Use E with K=M=2m, N=M/δand small D. Choose random x: m+log(1/δ) random bits. Set vi=E(x,i) Ext property⇒Hitting property ∀A of size ½M Call x badif E(x) inside A. # of bad x’s < K Pr[x is bad] < K/N= δ E(x,1) .. E(x,D) Randomness efficient (oblivious) sampling using extractors [S] N≈{0,1}n M≈{0,1}m D edges x (1-ε)M A bad x’s

  16. An (oblivious) sampling scheme uses a random n bit string x to generated D random variables with Chrnoff style property. Thm: [Z] The derived graph is an extractor. Extractors oblvs Sampling v1 .. vD Every (oblivious) sampling scheme yields an extractor N≈{0,1}n M≈{0,1}m D=2d edges x

  17. Outline of talk • Extractors as graphs with expansion properties • Extractors as functions which extract randomness • Applications • Explicit Constructions

  18. Constructions

  19. Extractors from error correcting codes • Can construct extractors from error-correcting code [ILL,SZ,T]. • Short seed. • Extract one additional bit • Extractors that extract one additional bit  List-decodable error-correcting codes • Extractors that extract many bits  codes with strong list-recovering properties [TZ].

  20. x x EC(x) EC(x) x EC(x)’ EC(x)’ noisy channel extremely noisy channel decoding encoding encoding x3 x1 x2 List-decodable error-correcting codes [S] 20% errors List decoding 49% errors • EC(x) is 20%-decodable if for every w there is a unique x s.t. EC(x) differs from w in 20% of positions. • EC(x) is (49%,t)-list-decodable if for every w there are at most t x’s s.t. EC(x) differs from w in 49% of positions. • There are explicit constructions of such codes.

  21. Extractors from list-decodable error-correcting codes [ILL,T] • Thm: If EC(x)is (½-ε,εK)-list-decodable then E(x,y)=(y,EC(x)y) is a (K,2ε)-extractor. • Note: E outputs its seed y. Such an extractor is called “strong”. • E outputs only one additional output bit EC(x)y • There are constructions of list-decodable error correcting codes with |y|=O(log n). • Strong extractors with one additional bit  List-decodable error correcting codes. • Strong extractors with many additional bits translate into very strong error correcting codes [TZ].

  22. Extractors from list-decodable error-correcting codes: proof Thm: If EC(x)is (½-ε,εK)-list-decodable then E(x,y)=(y,EC(x)y) is a (K,2ε)-extractor. Proof: by contradiction. Let X be a distribution/set of size K s.t. E(X,Y)=(Y,EC(X)Y) is far from uniform. Observation: Y and EC(X)Y are both uniform. • They are correlated. • Exists P s.t. P(Y)=EC(X)Y with prob > ½+2ε.

  23. Extractors from list-decodable error-correcting codes: proof II Thm: If EC(x)is (½-ε,εK)-list-decodable then E(x,y)=(y,EC(x)y) is a (K,2ε)-extractor. Exists P s.t. PrX,Y[P(Y)=EC(X)Y] > ½+2ε. By a Markov argument: For εKx’s in X PrY[P(Y)=EC(x)Y] > ½+ε. Think of P as a string Py=P(y). We have that P and EC(x) differ in ½-εcoordinates. Story so far: If E is bad then there is a string P s.t. for εKx’s P and EC(x) differ in few coordinates.

  24. x EC(x) P=EC(x)’ noisy channel encoding x1 x2 x3 Extractors from list-decodable error-correcting codes: proof III Thm: If EC(x)is (½-ε,εK)-list-decodable then E(x,y)=(y,EC(x)y) is a (K,2ε)-extractor. Story so far: If E is bad then there is a string P s.t. for εKx’s P and EC(x) differ in ½-ε coordinates. List decoding 49% errors By list-decoding properties of the code: • # of such x’s < εK. Contradiction!

  25. Roadmap • Can construct extractors from error-correcting code. • Short seed. • Output = Seed + 1. • Next: How to extract more bits. • General paradigm: Once you construct one extractor you can try to boost its quality.

  26. Extracting more bits [WZ] Starting point: An extractor E that extracts only few bits. Idea: (X|E(X,Y)) contains randomness. We can apply E to extract randomness from (X|E(X,Y)). Need a “fresh” seed. E’(X;(Y,Y’))=E(X,Y),E(X,Y’) Extract more randomness. Use larger seed. X Z Z Z Extractor Y Y’ X New Extractor Y’ Y

  27. Trevisan’s extractor: reducing the seed length Idea: Use few random bits to generate (correlated) seeds Y1,Y2,Y3… • Walk on expander? • Extractor? Works but gives small savings. Trevisan: use Nisan-Wigderson pseudorandom generator (based on combinatorial designs). [TZS,SU]: Use Y,Y+1,Y+2,... (based on the [STV] algorithm for list-decoding Reed-Muller code). X Extractor Y2 Y1 Y

  28. The extractor designer tool kit • Many ways to “compose” extractors with themselves and related objects. • Arguments use “entropy manipulations” depend on “function view” of extractors. • Impact on other graph construction problems: • Expander graphs (zig-zag product) [RVW,CRVW]. • Ramsey graphs that beat the Frankl-Wilson construction [BKSSW,BRSW].

  29. Entropy manipulations: composing two extractors [Z,NZ] Two independent sources X2 Z X1 Small Extractor Y’ Large Extractor Observation: Can compose a small ext. and a large ext. and obtain ext. which inherits small seed and large output. Paradigm: If given only one source try to convert it into two sources that are “sufficiently independent”.

  30. seed Y Extractor random output Summary: Extractors are Graphs Functions M≈{0,1}m source distribution X X Γ(X) Randomness K=2k (1-ε)M

  31. Conclusion • Unifying role of extractors: • Expanders, Oblivious samplers, Error correcting codes, Pseudorandom generators, hash functions… • Open problems: • More applications/connections. • The quest for explicitly constructing the optimal extractor. (Current record [LRVW]). • Direct and simple constructions. • Things I didn’t talk about: • Seedless extractors for special families of sources.

  32. That’s it…

  33. E(x)1 .. E(x)D Extractor graphs N≈{0,1}n M≈{0,1}m D=2d edges D=2d edges x x

  34. Extractor graphs: expansion N≈{0,1}n M≈{0,1}m X Γ(X) K=2k (1-ε)M

  35. Extractor random output Issues in a formal definition: 2. One extractor for all sources • Goal: Design one extractor function E(x) that works on all sufficiently high entropy distributions. • Problem: Impossible to extract even 1 bit from distributions with n-1 bits of entropy. • Have to settle for less! source distribution X Randomness {0,1}n x:E(x)=1 x:E(x)=0 Distribution X with entropy n-1 on which E(X) is fixed

  36. seed Y Extractor random output Definition of extractors [NZ] • We allow an extractor to also receive an additional seed of (very few) random bits. • Extractors use few random bits to extract many random bits from arbitrary distributions with sufficiently high entropy. Parameters: • Source length: n • Seed length: d ~ O(log n) • Entropy threshold: k ~ n/100 • Output length: m ~ k • Required error: ε~ 1/100 source distribution X Randomness Definition: A (k,ε)-extractor is a function E(x,y) s.t. For every distribution X with min-entropy k, E(X,Y) is ε-close* to uniform. Lower bounds [NZ,RT]: seed length ≥ log n + 2log(1/ε) Probabilistic method [S,RT]: Exists optimal extractor which matches lower bound and extracts k+d-2log(1/ε) bits. *A distribution P is ε-close to uniform if ||P-U||1≤ 2ε=> P supports 1-εelements.

  37. An extractor is an (unbalanced) bipartite graph M<<N. (e.g. M=Nδ, M=exp( (log N)δ). Every vertex x on the left has D neighbors. E(x)=(E(x)1,..,E(x)D) the extractor is better when D is small. (e.g. D=polylog N) Convention: E(x,y) = E(x)y E(x)1 .. E(x)D Extractor graphs:Definition [NZ] N≈{0,1}n M≈{0,1}m D edges x

  38. Extractor random output Issues in a formal definition: 1. Notion of entropy • The source distribution X must “contain randomness” • Necessary condition for extracting k bits: ∀x Pr[X=x]≤2-k • Dfn: X has min-entropyk if ∀x Pr[X=x]≤2-k • Example: flat distributions: X is uniformly distributed on a subset of size 2k. • Every X with min-entropy k is a convex combination of flat distributions. source distribution X Randomness {0,1}n 2k=|S|

  39. x x’ noisy channel Noisy channels and error corrections Goal: Transmit messages using a noisy channel errors Guarantee: x’ differs from x in at most (say) 20% positions. Coding Theory: Encode x prior to transmission.

More Related