1 / 31

Discrete Aggregation and Ranking and Gaussian Isoperimetric Inequalities

Discrete Aggregation and Ranking and Gaussian Isoperimetric Inequalities. Elchanan Mossel U.C. Berkeley http://stat.berkeley.edu/~mossel. Discrete Aggregation and Ranking. Motivation Aggregation of signals in order to rank is an important practical problem (example: Google).

palila
Download Presentation

Discrete Aggregation and Ranking and Gaussian Isoperimetric Inequalities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discrete Aggregation and Ranking and Gaussian Isoperimetric Inequalities Elchanan Mossel U.C. Berkeley http://stat.berkeley.edu/~mossel

  2. Discrete Aggregation and Ranking • Motivation • Aggregation of signals in order to rank is an important practical problem (example: Google). • Many important interesting variants • Today: Condorcet aggregations • Good probabilistic theory • Nice Gaussian Open problems • Same Theory applies to questions in computer science.

  3. Gaussian Noise • Let -1  1 and f, g : Rn Rm. • Define <f, g> := E[<f(N) , g(M) >], where N,M ~ Normal(0,I) with E[Ni Mj] = (i,j). • For sets A,B let: <A,B> := <1A,1B> • Let n := standard Gaussian volume • Let n := Lebesgue measure. • Let n-1, n-1 := corresponding (n-1)-dims areas.

  4. Some isoperimetric results • I. Ancient: Among all sets withn(A) = 1the minimizer ofn-1( A)is A = Ball. • II. Recent (Borell, Sudakov-Tsierlson 70’s) Among all sets withn(A) = athe minimizer ofn-1( A)is A=Half-Space. • III.More recent (Borell 85): For all > 0,among all sets with (A) = a the maximizer of <A,A> is given by A =Half-Space.

  5. Influences and Noise in product Spaces • Let X be a probability space. • Let f  L2(Xn,R). Thei’th influence of f is given by: • Ii(f) := E[ Var[f | x1,…,xi-1,xi+1,…,xn] ] • (Ben-Or,Kalai,Linial; Efron-Stein 80s) • Given a reversible Markov operator T on X and • f, g: Xn R define the T- noise form by • <f, g>T := E[f T n g] • The 2nd eigen-value(T) of T is defined by • (T) := max {|| :  spec(T),  < 1}

  6. Influences and Noise in product Spaces – Example 1 • Let X = {-1, 1} with the uniform measure. • For the dictator function f(x) = xj: Ii(xj) = (i,j). • For the majoritym(x) = sgn(1  i  n xi) function: Ii(m)  (2  n)-1/2. • Let Tbe the “Beckner Operator” on X: • Ti,j = (i,j) + (1-)/2. • T xi =  xiand <xi, xi>T = . • <m, m>T ~ 2 arcsin() /  • (T) = .

  7. Condorcet Paradox • n voters are to choose between 3 options / candidates. • Voter i ranks the three candidates A, B & C via a permutation i S3 • Let XABi = +1 if i(A) > i(B) • XABi = -1 if i(B) > i(A) • Aggregate rankings via: f : {-1,1}n! {-1,1}. • Assume thatf(-x1,…,-xn) = -f(x1,…,xn) • Aggregation: A is better than B if f(xAB) = 1. • A Condorcet Paradox occurs if: • f(xAB) = f(xBC) = f(xCA). • Defined by Marquis de Condorcet in 18’th century. B A C

  8. Arrow’s Impossibility Thm • Thm (Condorcet): If n > 2 is odd and f is the majority function then there exist rankings 1,…,n resulting in a Paradox • Thm (Arrow’s Impossibility): For all n > 2, unless f is the dictator function, there exist rankings 1,…,n resulting in a paradox.

  9. Kalai’s Random Ranking: • Each voter tosses a dice. • Vote according to the corresponding order on A,B and C.

  10. Probability of a Paradox • Rankings are chosen uniformly in S3n • What is the probability of a paradox: • PDX(f) = P[f(xAB) = f(xBC) = f(xCA)]? • Arrow’s:: If fdictator then PDX(f) > 0. • Note: If f = dictator then PDX(f) = 0. • Thm(Kalai 02):PDX(f) = ¼ - ¾ <f, f>T where Ti,j = 1/3(i,j) + 1/3 is the Beckner1/3 operator.

  11. Low influence aggregation • A question: What is the most rational aggregation function f not allowing dictatorships or Juntas? • For example, consider only monotonef that are invariant under a transitive action on the coordinates. • More generally: What is the minimum of PDX(f) among f satisfying for all voters i: Ii(f) = P[f(x1,…,xi,…,xn)  f(x1,…,-xi,…,xn)] <  where n  and  0. • Conjecture (Kalai 02): Among all low influence f’s, the one that minimizes PDX(f) is given by the majority function. • (PDX(f)  0.0876 for f = majority) X

  12. Influences and Noise -Example 2 • Let X = {0,1,2} with the uniform measure. • Let Ti,j = ½ (i  j) • Then (T) = ½ and • Claim (Colouring Graph): ConsiderXn as a graph where (x,y)  Edges(Xn) iff xi yi for all i. • Let A,B  Xn. Then <A, B>T = 0 iff there are no edges between A and B. In particular, A is an independent set iff <A, A>T = 0. • Q: How do “large” independent sets look like? • Important for the analysis of complexity of graph coloring.

  13. Graph Colouring – An Algorithmic Problem • Let (G) := min # of colours needed • to colour the vertices of a graph G so that no edge is monochramatic. • ApxCol(q,Q): • Given a graph G, is (G)  q or (G)  Q ? • This is an algorithmic problem. How hard is it? • Khot’s “games conjecture” + following claim  NP hard. • Claim: Consider {0,1,2}n as a graph G where (x,y)  Edges(G) iff xi yi for all i. • Let Q > 3. Suppose that  > 0 such that for all n if there are no edges between A and B  {0,1,2}n (<A,B>T = 0) and |A|,|B| > 3n/Q then there exists ani such that Ii(A) >  and Ii(B) > .

  14. Graph Colouring – An Algorithmic Problem u

  15. [u] Graph Colouring – An Algorithmic Problem u

  16. Gaussian Noise Bounds • Def: For a, b,  [0,1] , let • (a, b, r) := sup {< F,G > | F,G  R, [F] = a, [G] = b} • (a, b, r) := inf {< F,G > | F,G  R, [F] = a, [G] = b} • Thm: Let X be a finite space. Let T be a reversible Markov operator on Xwith  = (T) < 1. • Then  > 0  > 0 such that for all n and all f,g : Xn [0,1] satisfying maxi min(Ii(f), Ii(g)) <  • It holds that <f, g>T(E[f], E[g], ) +  and • <f, g>T(E[f], E[g], ) -  • M-O’Donnell-Oleskiewicz-05 + Dinur-M-Regev-06

  17. Application: Example 1 • Taking T on {-1,1} defined by Ti,j = (i,j)/3 + 1/3 • Thm : Claim:  f : {-1,1}n {-1,1} with Ii(f) <  for all i and E[f] = 0 it holds that: • <f, f>T <F, F>1/3 +  where F(x) = sgn(x) • <F, F>1/3 = 2 arcsin(1/3)/ (F is known by Borell-85) • Kalai’s conjecture: The probability of Paradox in Condorcet voting with low influences f is minimized by f = majority. • Weaker results obtained by Bourgain 2001.

  18. Application: Example 2 • Taking T on {0,1,2} defined by Ti,j = ½ (i  j) • Thm Claim: > 0  > 0 s.t. if A,B  {0,1,2}n have no edges between them and P[A], P[B]  then • There exists an i s.t. Ii(A), Ii(B) . • Proof follows from Borell-85 showing (,,1/2) > 0. • Claim  Hardness of approximation result for graph-colouring: • “For any constant K, it is NP hard to • colour 3-colorable graphs using K colours”. • Dinur-M-Regev-06

  19. More results • Other results that use Noise-Stability bounds include: • Social choice: Majority is most stable function to errors in voting machines (M-O’Donnell-Oleskiewicz-05), • Most predictable from sample (Mossel 07). • Hardness of approximation: • Khot-Vishnoy etc.

  20. Gaussian Noise Bounds • Def: For a, b,  [0,1] , let • (a, b, r) := sup {< F,G > | F,G  R, [F] = a, [G] = b} • (a, b, r) := inf {< F,G > | F,G  R, [F] = a, [G] = b} • Thm: Let X be a finite space. Let T be a reversible Markov operator on Xwith  = (T) < 1. • Then  > 0  > 0 such that for all n and all f,g : Xn [0,1] satisfying maxi min(Ii(f), Ii(g)) <  • It holds that <f, g>T(E[f], E[g], ) +  and • <f, g>T(E[f], E[g], ) -  • M-O’Donnell-Oleskiewicz-05 + Dinur-M-Regev-06

  21. Gaussian Noise Bounds • Proof Idea: • Low influence functions are close to functions in L2() = L2(N1,N2,…). • More formally: Let H[a,b] be: • n{ f : Xn [a, b] |  i: Ii(f) < , E[f] = 0, E[f2] = 1} • Then: H ““ {f  L2() : E[f] =0, E[f2] = 1, a  f  b} • noise forms in H [a,b] ~ noise forms of [a, b] bounded functions in L2() • Note: Our results generalize and strengthen limit results for U and V statistics.

  22. An Invariance Principle • For example, we prove: • Invariance Principle [M+O’Donnell+Oleszkiewicz(05)]: • Let p(x) = 0 < |S| · k aSi 2 S xi be a degree k multi-linear polynomial with |p|2 = 1 and Ii(p)  for all i. • Let X = (X1,…,Xn) be i.i.d. P[Xi =  1] = 1/2 . • N = (N1,…,Nn) be i.i.d. Normal(0,1). • Then for all t: • |P[p(X) · t] - P[p(N)· t]| · O(k 1/(4k)) • Note: Noise form “kills” high order monomials. • Proof works for any hyper-contractive random vars.

  23. Invariance Principle – Proof Sketch • Suffices to show that 8 smooth F (sup |F(4)| · C ),E[F(p(X1,…,Xn)] is close to E[F(p(N1,…,Nn))]. • Very similar to Lindberg proof of CLT (also Rotar, Chatterjee in non-linear settings).

  24. Invariance Principle – Proof Sketch • Write: p(X1,…,Xi-1, Ni, Ni+1,…,Nn) = R + Ni S • p(X1,…,Xi-1, Xi, Ni+1,…,Nn) = R + Xi S • F(R+Ni S) = F(R) + F’(R) Ni S + F’’(R) Ni2 S2/2 + F(3)(R) Ni3 S3/6 + F(4)(*) Ni4 S4/24 • E[F(R+ Ni S)] = E[F(R)] + E[F’’(R)] E[Ni2] /2 + E[F(4)(*)Ni4S4]/24 • E[F(R + Xi S)] = E[F(R)] + E[F’’(R)] E[Xi2] /2 + E[F(4)(*)Xi4 S4]/24 • |E[F(R + Ni S) – E[F(R + Xi S)|  C E[S4] • But, E[S2] = Ii(p). • And by Hyper-Contractivity, E[S4]  9k-1 E[S2] • So: |E[F(R + Ni S) – E[F(R + Xi S)|  C 9k Ii2

  25. Thm1 (“Double-Bubble”): • Among all pairs of disjoint sets A,Bwith n(A) = a, n(B) = b, the minimizer of n-1( A  B) is a “Double Bubble” • Thm2(“Peace Sign”): • Among all partitions A,B,C ofRnwith (A) = (B) = (C) = 1/3 , the minimum of ( A  B  C) is obtained for the “Peace Sign” • 1.Hutchings, Morgan, Ritore, Ros. + Reichardt, Heilmann, Lai, Spielman 2. Corneli, Corwin, Hurder, Sesum, Xu, Adams, Dvais, Lee, Vissochi Double bubbles

  26. The Peace-Sign Conjecture • Conj: • For all 0  1, • all n  2 • The maximum of • <A, A> + <B, B> + <C, C> • among all partitions (A,B,C) of Rnwith n(A) = n(B) = n(C) = 1/3 is obtained for • (A,B,C) = “Peace Sign”

  27. The Peace-Sign Conjecture • Conj: • For all 0  1, • all n  2 • The maximum of • <A, A> + <B, B> + <C, C> • among all partitions (A,B,C) of Rnwith n(A) = n(B) = n(C) = 1/3 is obtained for • (A,B,C) = “Peace Sign”

  28. Summary • Prove the “Peace Sign Conjecture” (Isoperimetry) • “Plurality is Stablest” (Low Inf Bounds) • MAX-3-CUT hardness (CS) and voting. • (more modest: almost optimal results using isoperimetric theory). • Other possible application of invariance principle: • To Convex Geometry? • To Additive Number Theory?

  29. Papers and Collaborators • Noise Stability of Functions with low influences: • Invariance and Optimality • M-O’Donnell-Oleszkiewicz. • Conditional Hardness for Approximate Coloring • Dinur-M-Regev. • Optimal Inapproximability results for MAX-CUT and 2CSPproblems? • Khot-Kindler-M-O’Donnell

More Related