1 / 48

Pareto Optimal Solutions for Smoothed Analysts

Pareto Optimal Solutions for Smoothed Analysts. Ryan O’Donnell (CMU, IAS) joint work with Ankur Moitra (MIT). Binary optimization, linear objective. F ⊆ {0,1} n , “fea sible solutions”. max. Binary optimization, linear objective s. F ⊆ {0,1} n , “fea sible solutions”.

marlened
Download Presentation

Pareto Optimal Solutions for Smoothed Analysts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Pareto Optimal Solutionsfor Smoothed Analysts Ryan O’Donnell (CMU, IAS) joint work with Ankur Moitra (MIT)

  2. Binary optimization, linear objective F ⊆ {0,1}n, “feasible solutions” max

  3. Binary optimization, linear objectives F ⊆ {0,1}n, “feasible solutions” “max”

  4. Pareto optima def: x ∈ F is Pareto optimal if not dominated on every objective by some y ∈ F. Obj2 y x Obj1

  5. Pareto optima def: x ∈ F is Pareto optimal if not dominated on every objective by some y ∈ F. Obj2 y x Obj1

  6. Obj2 Obj1

  7. thm: If d+1 linear objectives are “semi-random” (in the Smoothed Analysis sense) then for any F ⊆ {0,1}n, E[# Pareto optima in F] ≤ O(n2d).

  8. Application: 0-1 Knapsack [Nemhauser−Ullmann69]: For i=1…n, compute Pareto optimal 〈weight,value〉 pairs achievable by items 1…i. [Beier−Vöcking03]: With 2 semi-rand objs, E[# PO’s] ≤ O(n4). ⇒ O(n5)-time alg for 0-1 Knapsack in Smoothed Analysis model!

  9. Model [BV03] F ⊆ {0,1}n arbitrary Obj(x) = d+1 linear objs x ϕ≥ 1 a parameter Each is a r.v. on [0,1] with pdf bdd by ϕ.All independent.

  10. Model [BV03] ϕ 1/ϕ Each is a r.v. on [0,1] with pdf bdd by ϕ.All independent.

  11. Prior work

  12. (F = {0,1}n) [BV03]:d = 1, E[# PO’s] ≤ O(ϕn4) [B04]:Same for arbit. F. Conj: ≤ O(nf(d)) [BRV07]:d = 1, E[# PO’s] ≤ O(ϕn2) [RT09]:E[# PO’s] ≤ roughly for n ≥ exp(exp(O(d2 log d)) [MO11]:E[# PO’s] ≤ 2(4ϕd)d(d+1)/2n2d Remark:All bounds here hold even assuming Obj0 adversarial, nonlinear, arbitrary. [BV03]:Ω(n2) when Obj0 adversarial, ϕ = 1 [BR11]:Ω(ϕn2), d = 1; Ω(ϕn)d-logd roughly, d > 1

  13. Warmup: Alternate proof (sketch) for[BRV07]’s d = 1 bound

  14. F ⊆ {0,1}n Obj0(x) arbitrary E[# PO’s] ≤ O(ϕn2) Obj1(x) ϕ-semi-random ? 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj1 Obj0

  15. F ⊆ {0,1}n Obj0(x) arbitrary E[# PO’s] ≤ O(ϕn2) Obj1(x) ϕ-semi-random ? .4 .8 .2 .9 0 0 1 1 1 0 1 1 1 1 0 1 0 1 1 1 0 1 0 0 0 1 0 0 0 0 1 1 0 1 1 1 0 1 1 1 1 1 1 0 Obj1 Obj0

  16. Goal: For each ϵ-strip S,E[# PO’s in S] ≈ Pr[∃ a PO in S] ≤ O(n)ϕϵ ϵ2 Union-bound over (n/ϵ) strips ⇒ E[# PO’s] ≤ O(ϕn2) 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj1 ϵ S Obj0

  17. Given x, Pr[ Vx ∈ S ] ? 0 1 0 1 1 1 0 1 0 0 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj1 ϵ S Obj0 0 1 1 0 1

  18. Given x, Pr[ Vx ∈ S ] ? Take any j s.t. xj = 1. 0 1 0 1 1 1 0 1 0 0 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj1 ϵ S Obj0 0 1 1 0 1

  19. Given x, Pr[ Vx ∈ S ] ? Take any j s.t. xj = 1. = Pr[ Vj ∈ certain width-ϵ interval] ≤ϕϵ. .4 .8 .2 .9 1 1 1 1 0 0 0 1 1 1 1 1 0 1 1 0 1 0 0 0 0 1 1 0 1 1 0 1 0 0 0 1 0 1 1 0 1 1 1 1 Obj1 ϵ S Obj0

  20. Given x, Pr[ Vx ∈ S ] ? Take any j s.t. xj = 1. = Pr[ Vj ∈ certain width-ϵ interval] ≤ϕϵ. Boundedness Lemma: Pr[ Vx ∈ S] ≤ O(ϵ), even just using the randomness of Vj. 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 ϵ S Obj0

  21. event Tx,S = “x is PO and Vx ∈ S” .4 .8 .2 .9 1 1 0 1 1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 1 1 0 0 0 1 1 1 Obj1 ϵ S Obj0

  22. event Tx,S = “x is PO and Vx ∈ S” Fantasy: Now a unique x s.t. Tx,S may yet occur .4 .8 .2 .9 1 1 0 1 1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 0 0 0 1 1 1 1 0 0 0 1 1 1 0 1 1 1 1 Obj1 ϵ S Obj0

  23. more complicated event: Tx,j,S=“x is PO and xj = 1 and Vx ∈ S and first PO to x’s left, call it y, has yj = 0” Uniqueness Lemma: Draw all Vi’s except Vj. Then ∃ at most 1 x s.t. Tx,j,S may still occur.

  24. Tx,j,S=“x is PO and xj = 1 and Vx ∈ S and first PO to x’s left, call it y, has yj = 0” .4 .8 .2 .9 0 0 1 1 1 0 1 1 0 1 0 1 0 1 1 1 1 1 1 0 1 0 1 0 0 1 1 0 1 1 0 1 0 0 0 0 1 1 1 1 Obj1 ϵ S Obj0

  25. Remainder of the d=1 proof sketch

  26. Tx,j,S=“x is PO and xj = 1 and Vx ∈ S and …” Uniqueness Lemma: Draw all Vi’s except Vj. Then ∃ at most 1 x s.t. Tx,j,S may still occur. Bddness Lemma: For that x, Pr[Vx ∈ S] ≤ϕϵ. Union-bound over all j, S: E[ # PO x s.t. first PO to x’s left, y, has yj ≠ 1 = xj for some j ] ≤ n(n/ϵ)ϕϵ = n2ϕ. For each PO x, ∃ j s.t. yj ≠ c = xj. Maybe c = 0… Union-bound over c ∈ {0,1}… + another trick.

  27. Our result: Larger d

  28. d=2: Obj0 arbitrary; Obj1, Obj2ϕ-semi-random Obj1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj2 Obj0

  29. V = Obj1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj2 Obj0

  30. V = Obj1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 ϵ ϵ S Obj2 Obj0

  31. more complicated event: Tx,j,S=“x is PO and xj = 1 and Vx ∈ S and first PO to x’s left, call it y, has yj = 0”

  32. more complicated event: Tx,j,c,S=“x is PO and xj = c and Vx − ∈ S and first PO to x’s left, call it y, has yj ≠ c”

  33. still more complicated event: Tx,J,C,S=“x is PO and…” J: list of d coordinates C: d×d matrix of bits w/ certain pattern S: d-dimensional strip, plus “nearby” d′-dim. stripsfor all d′ < d

  34. still more complicated event: Tx,B=“x is PO and conditions involving V, x, B.” J: list of d coordinates C: d×d matrix of bits w/ certain pattern S: d-dimensional strip, plus “nearby” d’-dim. stripsfor all d’ < d + = B: “blueprint” [AMR09]

  35. Tx,B=“x is PO and conditions involving V, x, B.” Given B, entries of V are partitioned into two sets, V[fewB] and V[mostB]. V =

  36. Tx,B=“x is PO and conditions involving V, x, B.” Given B, entries of V are partitioned into two sets, V[fewB] and V[mostB]. Uniqueness Lemma: Draw V[mostB]. Then ∃ at most 1 x s.t. Tx,B may still occur. Boundedness Lemma: For any V[mostB] outcome,Pr [condits involving V, x, B] ≤ϕd(d+1)/2 ϵd(d+1)/2. V[fewB] Counting Lemma: O(d/ϵ)d(d+1)/2 n2dpossible B. Union bound ⇒ E[# PO’s] ≤O(dϕ)d(d+1)/2n2d.

  37. Tx,j,c,S=“x is PO and xj = c and Vx − ∈ S and first PO to x’s left, call it y, has yj ≠ c” Tx,B=“x is PO and conditions involving V, x, B.” J: list of d coordinates C: d×d matrix of bits w/ certain pattern S: d-dimensional strip, plus “nearby” d′-dim. stripsfor all d′ < d + = B: “blueprint”

  38. Tx,B=“x is PO and SKETCH(x,V) = B” where SKETCH() is a certain deterministic algorithm.

  39. SKETCH(x,V):

  40. Tx,B=“x is PO and SKETCH(x,V) = B” Uniqueness Lemma: Draw V[mostB]. Then ∃ at most 1 x s.t. Tx,B may still occur. Bddness Lemma: For any V[mostB] outcome,Pr [SKETCH(x,V) = B] ≤ϕd(d+1)/2 ϵd(d+1)/2. V[fewB]

  41. Tx,B=“x is PO and SKETCH(x,V) = B” Uniqueness Lemma: Draw V[mostB]. Then ∃ at most 1 x s.t. Tx,B may still occur. Equivalent Lemma: Suppose x, V are such that x is PO. AssumeSKETCH(x,V) = B. Then RECONSTRUCT(B,V[mostB]) = x.

  42. RECONSTRUCT(B,V[most]):

  43. Uniqueness Lemma: Suppose x, V are such that x is PO. AssumeSKETCH(x,V) = B. Then RECONSTRUCT(B,V[mostB]) = x. Bddness Lemma: For any V[mostB] outcome,Pr [SKETCH(x,V) = B] ≤ϕd(d+1)/2 ϵd(d+1)/2. V[fewB] Counting Lemma: O(d/ϵ)d(d+1)/2 n2dpossible B.

  44. A sketch of SKETCH(x,V)for d = 2

  45. SKETCH(x,V): SKETCH(x,V): Let F0= F Let D1 = { z ∈ F0 : V1z > V1x and V2z > V2x } Let y1= argmax{ Obj0(y) : y ∈ D1} Let j1 = least index s.t. Let F1= { z ∈ F0 : Obj0(z) > Obj0(y1) and } Let D2 = { z ∈ F1 : V2z > V2x } Let y2= argmax{ Obj1(y) : y ∈ D2 } Let j2 = least index s.t. Let F2= { z ∈ F1 : Obj1(z) > Obj1(y1) and } // if Vx is PO then it must be argmax{ Obj2(z) : z ∈ F2 } 10. Output: J = (j1, j2), C = S= ( strip of Obj1,2(x) − diag(Vj1,j2 C), strip of Obj2(x) − )

  46. Close gap of nd vs. n2d. • Lower bounds when all objs semirandom? • F ⊆ {0,1,2,…,k}n ? • Bounds on Var[ # POs] ? • More Smoothed Analysis via “blueprints”? Future directions?

  47. The End. Any questions?

More Related