Algebraic Constructions of Randomness Extractors

Algebraic Constructions of Randomness Extractors Chris Umans Caltech Based on joint work with Venkat Guruswami and Salil Vadhan and joint work with Amnon Ta-Shma

Randomness extractors • Computers are inherently deterministic machines, yet we want to use randomness • one solution: use pseudo-random generators • Question: can we use “real” randomness? • physical source • imperfect – biased, correlated

Randomness extractors • “Hardware” side • what physical source? • ask the physicists… • “Software” side • what is the minimum we need from the physical source?

Randomness extractors • imperfect sources: • “stuck bits”: • “correlation”: • “stranger correlation”: 111111 ““““““ perfect squares • there are specific ways to get independent unbiased random bits from specific imperfect physical sources

Randomness extractors • want to assume we don’t know details of physical source • general model capturing all of these? • yes: “min-entropy” • universal procedure for all imperfect sources? • yes: “extractors”

Min-entropy • General model of physical source w/ k < n bits of hidden randomness 2kstrings string sampled uniformly from this set {0,1}n • Definition: random variable X on {0,1}n has min-entropyminx –log(Pr[X = x]) • min-entropy k implies no string has weight more than 2-k

Randomness extractors • Dozens of constructions over 15+ years (e.g.NZ96, GW97, SZ99, Z97, TS96, NTS99, T99, RRV99, ISW00, RSW00, RVW00, TUZ01, TZS01, SU01, LRVW03, R05, Z06, GUV09, DKSS09, TU12) Goals: optimal: “milestone”: short seed log n + O(1) O(log n) long output m = k + d - O(1) m = (1 - )k source string -close to uniform 2kstrings E seed m bits {0,1}n d bits 15+ year quest for optimal…

Applications of extractors • randomness extractors are extremely versatile objects • different settings of parameters turn into • families of hash functions • error-correcting codes • expander graphs with the “unique neighbor” property • … • many uses beyond original motivation

Applications of extractors • Derandomization [Sip88,NZ93,INW94, GZ97,RR99, MV99,STV99,GW02] • Distributed & Network Algorithms [WZ95,Zuc97,RZ98,Ind02] • Hardness of Approximation [Zuc93,Uma99,MU01,Zuc06] • Data Structures [Ta02] • Cryptography [CDHKS00,Lu02,DRS04,NV04] • Metric Embeddings [Ind06]

Constructions over the years • 1990 – 1999: largely combinatorial • hashing • composition, iteration • 1999: new ingredient – error-correcting codes • 2003 - present: “milestone” parameters achieved, and slightly surpassed • composition + polynomial method [LRVW03 + DKSS09] • “purely algebraic”[GUV09 + TU12]

Condensers • Intermediate object for obtaining extractor: Goals: minimize d and m • “lossless” if k’ = k + d source string 2kstrings 2k’strings C seed {0,1}n d bits m bits • “k !² k’ condenser”

Graph viewpoint C:{0,1}n x {0,1}d! {0,1}m degree D =2d [N]={0,1}n [M]={0,1}m C(x,y) x subset T BAD(T) “too many neighbors in T” argue that BAD(T) is small

Graph viewpoint C:{0,1}n x {0,1}d! {0,1}m [N]={0,1}n [M]={0,1}m D =2d C(x,y) When BAD(T) = {x:Pry[C(x, y) 2 T] = 1} C is a lossless condenser if and only if |BAD(T)|· (1+²)|T|/D x T BAD(T) “too many neighbors in T”

Graph viewpoint C:{0,1}n x {0,1}d! {0,1}m [N]={0,1}n [M]={0,1}m D =2d C(x,y) When BAD(T) = {x:Pry[C(x, y) 2 T] ¸²} C a log(K/²) !² log(K’/²) condenser if for |T| = K’we have |BAD(T)| · K x T BAD(T) “too many neighbors in T”

Graph viewpoint C:{0,1}n x {0,1}d! {0,1}m [N]={0,1}n [M]={0,1}m D =2d C(x,y) x Goal #1: prove |BAD(T)| < (1+²)|T|/D T BAD(T) Goal #2: handle sets T as large as M/poly(n) “too many neighbors in T” #1 + #2 would give optimal extractors!

Outline for rest of talk • first construction and proof [Guruswami, Umans, Vadhan 2009] • second construction: using the idea twice [Ta-Shma, Umans 2012] • an open question

First constructionand its analysis

First construction • Fqfinite field • parameter h ≤ q • deg. n polynomial E(Y) irreducible over Fq • source: degree n-1 univariate polynomial f • define fi(Y) = fhi(Y) mod E(Y) C(f, y 2 Fq) = (y, f0(y), f1(y), f2(y), , fm-1(y)) source 2kstrings C seed {0,1}n d bits

First construction define: fi(Y) = fhi(Y) mod irreducible E(Y) C(f, y 2 Fq) = (y, f0(y), f1(y), f2(y), , fm-1(y)) • Fix T µ Fqm+1of size at most (1 - ²)q¢hm • note goal #2 was q¢qm/poly(n) • Define BAD(T) = {f : Pry[C(f, y) 2 T] = 1} • We will prove: |BAD(T)| < hm • this meets goal #1

First construction define: fi(Y) = fhi(Y) mod irreducible E(Y) C(f, y 2 Fq) = (y, f0(y), f1(y), f2(y), , fm-1(y)) • Q(W, W0, …, Wm-1) vanishes on T • deg(W) · (1-²)q and deg(Wi) · h-1 • Rf(Y) = Q(Y, f0(Y), …, fm-1(Y)) • f 2 BAD(T) ) Rf(y) = 0 8y 2 Fq • deg(Rf) ·(1 - ²)q + hmn < q T µ Fqm+1BAD(T) = {f : Pry[C(f, y) 2 T] = 1}

First construction define: fi(Y) = fhi(Y) mod irreducible E(Y) C(f, y 2 Fq) = (y, f0(y), f1(y), f2(y), , fm-1(y)) • Q(W, W0, …, Wm-1) vanishes on T • deg(W) · (1-²)q and deg(Wi) · h-1 • Rf(Y) = Q(Y, f0(Y), …, fm-1(Y)) • f 2 BAD(T) ) Rf(y) = 0 8y 2 Fq • deg(Rf) ·(1 - ²)q + hmn < q [require q > hnm/²] T µ Fqm+1BAD(T) = {f : Pry[C(f, y) 2 T] = 1}

First construction define: fi(Y) = fhi(Y) mod irreducible E(Y) C(f, y 2 Fq) = (y, f0(y), f1(y), f2(y), , fm-1(y)) • Q(W, W0, …, Wm-1) vanishes on T f 2 BAD(T) )Rf(Y) = Q(Y, f0(Y), …, fm-1(Y)) ´ 0 ) (Y,f0(Y), …,fm-1(Y)) root of Q ) f root of Q*(Z) = Q(Y, Z, Zh, …, Zhm-1) mod E(Y) T µ Fqm+1BAD(T) = {f : Pry[C(f, y) 2 T] = 1} Conclude: |BAD(T)| · deg(Q*) = hm-1

First construction – recap define: fi(Y) = fhi(Y) mod irreducible E(Y) C(f, y 2 Fq) = (y, f0(y), f1(y), f2(y), , fm-1(y)) • Fix T µ Fqm+1of size at most (1 - ²)q¢hm • We proved: |BAD(T)| < hm • Two requirements force h < q1 - ®(® constant) • q > nmh/² • q · poly(n) • So |T| < qhm· q(qm)1-®¼M1-® [want close to M] best possible

Graph viewpoint C:{0,1}n x {0,1}d! {0,1}m [N]={0,1}n [M]={0,1}m D =2d C(x,y) x Goal #1: prove |BAD(T)| < (1+²)|T|/D T BAD(T) Goal #2: handle sets T as large as M/poly(n) “too many neighbors in T” #1 + #2 would give optimal extractors!

Many 0s below ) root above info about:polynomial: type of poly: univariate over Fqn Q* BAD(T) µ Fqn f is a root ) degree argument many 0s on curve defined by f multivariate over Fq Q interpolates T T µ Fqm+1 Next: use this idea twice…

Secondconstructionand its analysis

First modification • Fqfinite field • deg. n polynomial E(Y) irreducible over Fq • source: degree n-1 univariate polynomial f • fi(Y) = fhi(Y) mod E(Y) Gi(f) for Gi:Fqn! Fqn source C 2kstrings seed d bits {0,1}n • C(f, y 2Fq) = (G0(f)(y), G1(f)(y), , Gm-1(f)(y)) (deg(Gi) will be hm-1 – same as before)

Second modification • Fq = Fh[Z]/D(Z) • deg. n polynomial E(Y) irreducible over Fq • source: degree n-1 univariate polynomial f • fi(Y) = Gi(f) for Gi:Fqn! Fqn C(f; y2Fq, z2Fh) = (G0(f)(y)(z), G1(f)(y)(z), , Gm-1(f)(y)(z)) source degree 2 extension C 2kstrings seed d bits {0,1}n now C maps into Fhm

Graph viewpoint – reminder C:{0,1}n x {0,1}d! {0,1}m [N]={0,1}n [M]={0,1}m D =2d C(x,y) x Goal #1: prove |BAD(T)| < (1+²)|T|/D T BAD(T) Goal #2: handle sets T as large as M/poly(n) “too many neighbors in T” #1 + #2 would give optimal extractors!

Analysis of 2nd construction Fqn C(f; y2Fq, z2Fh) = (G0(f)(y)(z), G1(f)(y)(z), , Gm-1(f)(y)(z)) • Fix T µ Fhmof size at most (1 - ²)hm • this meets goal #2 • Define BAD(T) = {f : Pry,z[C(f; y,z) 2 T] = 1} • will (try to) prove: |BAD(T)| < hm¢(small) • note goal #1 was |BAD(T)| · hm/(qh) Fq Fh

Analysis of 2nd construction Fqn C(f; y2Fq, z2Fh) = (G0(f)(y)(z), G1(f)(y)(z), , Gm-1(f)(y)(z)) • Q(W0, …, Wm-1) vanishes on T with mult. t • deg(Q) · ht-1 Fq T µ FhmBAD(T) = {f : Pry,z[C(f; y,z) 2 T] = 1} Fh

Calculation… • T µ Fhm of size (1 - ²)hm • Q(W0, …, Wm-1) • vanishes on T with multiplicity t • total degree ht-1 if t > (m2/²) # of monomials # constraints for each point in T

Analysis of 2nd construction Fqn C(f; y2Fq, z2Fh) = (G0(f)(y)(z), G1(f)(y)(z), , Gm-1(f)(y)(z)) • Q(W0, …, Wm-1) vanishes on T with mult. t • deg(Q) · ht-1 • Rf, y(Z) = Q(G0(f)(y)(Z), , Gm-1(f)(y)(Z)) • f 2 BAD(T), y 2 Fq) Rf, y(z) = 0 8z 2 Fh (mult. t) • deg(Rf, y) ·ht-1 < ht Fq T µ FhmBAD(T) = {f : Pry,z[C(f; y,z) 2 T] = 1} Fh

Analysis of 2nd construction Fqn C(f; y2Fq, z2Fh) = (G0(f)(y)(z), G1(f)(y)(z), , Gm-1(f)(y)(z)) • Q(W0, …, Wm-1) vanishes on Twith mult. t • Rf, y(Z) = Q(G0(f)(y)(Z), , Gm-1(f)(y)(Z)) • f 2 BAD(T) ) Rf, y = 0 for all y 2 Fh • Sf(Y) = Q(G0(f)(Y), , Gm-1(f)(Y)) ) Sf(y) = 0 8y 2 Fq; deg(Sf) · htn < q Fq T µ FhmBAD(T) = {f : Pry,z[C(f; y,z) 2 T] = 1} Fh • [require h > tn]

Analysis of 2nd construction Fqn C(f; y2Fq, z2Fh) = (G0(f)(y)(z), G1(f)(y)(z), , Gm-1(f)(y)(z)) • Q(W0, …, Wm-1) vanishes on Twith mult. t f 2 BAD(T) )Sf(Y) = Q(G0(f)(Y), …,Gm-1(f)(Y)) ´ 0 ) (G0(f)(Y), …, Gm-1(f)(Y)) root of Q ) f root of Q*(Z) = Q(G0(Z), …, Gm-1(Z)) Fq T µ FhmBAD(T) = {f : Pry,z[C(f; y,z) 2 T] = 1} Fh Conclude: |BAD(T)| · deg(Q*) < ht¢deg(Gi) = ht¢hm-1

Second construction – recap Fqn C(f; y2Fq, z2Fh) = (G0(f)(y)(z), G1(f)(y)(z), , Gm-1(f)(y)(z)) • Fix T µ Fhmof size at most (1 - ²)hm • this meets goal #2 • we proved*: |BAD(T)| < hm¢t • note goal #1 was |BAD(T)| · hm/(qh) Fq Fh * but Q*(Z) = Q(G0(Z), …, Gm-1(Z)) may be zero!

Choice of Gi + problem solved • Can choose Giof degree hm-1 s.t. • each Gi is a linearized polynomial (sparse) • (Fh)m is contained in image of map G = (G0, …, Gm-1) : Fqn! (Fqn)m G Fhm (Fqn)m

Choice of Gi + problem solved • Can choose Giof degree hm-1 s.t. • (Fh)m is contained in image of map G = (G0, …, Gm-1) : Fqn! (Fqn)m • Q(W0, …, Wm-1) vanishes on T with mult. 2t • price: T of size only ¼ (h/2)minstead of ¼ hm • payoff: some · t-order derivative Q(v) satisfies • Q(v) not zero on all of Fhm • hence Q(v)(G0(Z), …, Gm-1(Z))  0 • still vanishes on T with multiplicity at least t

Condensers • Intermediate object for obtaining extractor: • “lossless” if k’ = k + d source string 2kstrings 2k’strings C seed {0,1}n d bits m bits • 2nd construction achieves d = O(log n) and • k’¼ (1 – 1/log n)k “sublinear entropy loss” • k’¼ (1 – 1/log n)m “sublinear entropy deficiency”

Getting an extractor source string C seed 2kstrings d1bits E only needs to work for “dense” sources {0,1}n source string -close to uniform 2k’strings E seed m bits {0,1}n’ d2bits

Getting an extractor Various works: from source with minentropy rate (1 - ®) can extract (1-3)k bits with seed d = O(optimal) source string -close to uniform 2kstrings E seed m bits {0,1}n d bits

Randomness extractors Goals: optimal: “milestone”: this work: short seed log n + O(1) O(log n) O(log n) long output m = k+d-O(1) m = (1 - )k m = (1 - ®)k source string -close to uniform 2kstrings E seed m bits {0,1}n d bits ® any constant ® = 1/log n currently the “world record”…

an open question

A question • find an explicit curve • G = (G0, …, Gm-1) : Fqn! (Fqn)m • with deg(Gi) ·hm¢poly(h,m), so that • for every T µ Fhmof size hm/poly(h,m) there is an interpolating polynomial QT(W0, …, Wm-1) of deg ht-1 vanishing on T with multiplicity t, but QT(G0(Z), …, Gm-1(Z))  0 h = poly(m) t = poly(m)

A question • exists by simple probabilistic argument • for each T, find QT 0 as before • probability QT is 0 on random point < ½ • probability QT is 0 on hm random points < 2-hm • union bound over < 2hm different sets T • Related question: are sparse or linearized polynomials sufficient?

Conclusions • algebraic constructions of randomness extractors with “world record” parameters • main objects: • proof idea: “bad” strings are roots of poly Q* define: fi(Y) = fhi(Y) mod irreducible E(Y) C(f, y 2 Fq) = (y, f0(y), f1(y), f2(y), , fm-1(y)) • curve G = (G0, …, Gm-1) : Fqn! (Fqn)m • C(f; y 2 Fq, z 2 Fh) = (G0(f)(y)(z), , Gm-1(f)(y)(z))

Open problems • Obtain an optimal extractor construction! • construct optimal extractors for extremely dense sources (minentropy k = (1 – o(1))n) • answering open question + overcoming a few technical hurdles would give condensers meeting goal #1 and #2

Thank you!

Algebraic Constructions of Randomness Extractors