Pareto Optimal Solutions for Smoothed Analysts

1. Pareto Optimal Solutionsfor Smoothed Analysts Ryan O’Donnell (CMU, IAS) joint work with Ankur Moitra (MIT)

2. Binary optimization, linear objective F ⊆ {0,1}n, “feasible solutions” max

3. Binary optimization, linear objectives F ⊆ {0,1}n, “feasible solutions” “max”

4. Pareto optima def: x ∈ F is Pareto optimal if not dominated on every objective by some y ∈ F. Obj2 y x Obj1

5. Pareto optima def: x ∈ F is Pareto optimal if not dominated on every objective by some y ∈ F. Obj2 y x Obj1

6. Obj2 Obj1

7. thm: If d+1 linear objectives are “semi-random” (in the Smoothed Analysis sense) then for any F ⊆ {0,1}n, E[# Pareto optima in F] ≤ O(n2d).

9. Application: 0-1 Knapsack [Nemhauser−Ullmann69]: For i=1…n, compute Pareto optimal 〈weight,value〉 pairs achievable by items 1…i. [Beier−Vöcking03]: With 2 semi-rand objs, E[# PO’s] ≤ O(n4). ⇒ O(n5)-time alg for 0-1 Knapsack in Smoothed Analysis model!

10. Model [BV03] F ⊆ {0,1}n arbitrary Obj(x) = d+1 linear objs x ϕ≥ 1 a parameter Each is a r.v. on [0,1] with pdf bdd by ϕ.All independent.

11. Model [BV03] ϕ 1/ϕ Each is a r.v. on [0,1] with pdf bdd by ϕ.All independent.

12. Prior work

13. (F = {0,1}n) [BV03]:d = 1, E[# PO’s] ≤ O(ϕn4) [B04]:Same for arbit. F. Conj: ≤ O(nf(d)) [BRV07]:d = 1, E[# PO’s] ≤ O(ϕn2) [RT09]:E[# PO’s] ≤ roughly for n ≥ exp(exp(O(d2 log d)) [MO11]:E[# PO’s] ≤ 2(4ϕd)d(d+1)/2n2d Remark:All bounds here hold even assuming Obj0 adversarial, nonlinear, arbitrary. [BV03]:Ω(n2) when Obj0 adversarial, ϕ = 1 [BR11]:Ω(ϕn2), d = 1; Ω(ϕn)d-logd roughly, d > 1

14. Warmup: Alternate proof (sketch) for[BRV07]’s d = 1 bound

15. F ⊆ {0,1}n Obj0(x) arbitrary E[# PO’s] ≤ O(ϕn2) Obj1(x) ϕ-semi-random ? 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj1 Obj0

16. F ⊆ {0,1}n Obj0(x) arbitrary E[# PO’s] ≤ O(ϕn2) Obj1(x) ϕ-semi-random ? .4 .8 .2 .9 0 0 1 1 1 0 1 1 1 1 0 1 0 1 1 1 0 1 0 0 0 1 0 0 0 0 1 1 0 1 1 1 0 1 1 1 1 1 1 0 Obj1 Obj0

17. Goal: For each ϵ-strip S,E[# PO’s in S] ≈ Pr[∃ a PO in S] ≤ O(n)ϕϵ ϵ2 Union-bound over (n/ϵ) strips ⇒ E[# PO’s] ≤ O(ϕn2) 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj1 ϵ S Obj0

18. Given x, Pr[ Vx ∈ S ] ? 0 1 0 1 1 1 0 1 0 0 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj1 ϵ S Obj0 0 1 1 0 1

19. Given x, Pr[ Vx ∈ S ] ? Take any j s.t. xj = 1. 0 1 0 1 1 1 0 1 0 0 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj1 ϵ S Obj0 0 1 1 0 1

20. Given x, Pr[ Vx ∈ S ] ? Take any j s.t. xj = 1. = Pr[ Vj ∈ certain width-ϵ interval] ≤ϕϵ. .4 .8 .2 .9 1 1 1 1 0 0 0 1 1 1 1 1 0 1 1 0 1 0 0 0 0 1 1 0 1 1 0 1 0 0 0 1 0 1 1 0 1 1 1 1 Obj1 ϵ S Obj0

21. Given x, Pr[ Vx ∈ S ] ? Take any j s.t. xj = 1. = Pr[ Vj ∈ certain width-ϵ interval] ≤ϕϵ. Boundedness Lemma: Pr[ Vx ∈ S] ≤ O(ϵ), even just using the randomness of Vj. 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 ϵ S Obj0

22. event Tx,S = “x is PO and Vx ∈ S” .4 .8 .2 .9 1 1 0 1 1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 1 1 0 0 0 1 1 1 Obj1 ϵ S Obj0

23. event Tx,S = “x is PO and Vx ∈ S” Fantasy: Now a unique x s.t. Tx,S may yet occur .4 .8 .2 .9 1 1 0 1 1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 0 0 0 1 1 1 1 0 0 0 1 1 1 0 1 1 1 1 Obj1 ϵ S Obj0

24. more complicated event: Tx,j,S=“x is PO and xj = 1 and Vx ∈ S and first PO to x’s left, call it y, has yj = 0” Uniqueness Lemma: Draw all Vi’s except Vj. Then ∃ at most 1 x s.t. Tx,j,S may still occur.

25. Tx,j,S=“x is PO and xj = 1 and Vx ∈ S and first PO to x’s left, call it y, has yj = 0” .4 .8 .2 .9 0 0 1 1 1 0 1 1 0 1 0 1 0 1 1 1 1 1 1 0 1 0 1 0 0 1 1 0 1 1 0 1 0 0 0 0 1 1 1 1 Obj1 ϵ S Obj0

26. Remainder of the d=1 proof sketch

27. Tx,j,S=“x is PO and xj = 1 and Vx ∈ S and …” Uniqueness Lemma: Draw all Vi’s except Vj. Then ∃ at most 1 x s.t. Tx,j,S may still occur. Bddness Lemma: For that x, Pr[Vx ∈ S] ≤ϕϵ. Union-bound over all j, S: E[ # PO x s.t. first PO to x’s left, y, has yj ≠ 1 = xj for some j ] ≤ n(n/ϵ)ϕϵ = n2ϕ. For each PO x, ∃ j s.t. yj ≠ c = xj. Maybe c = 0… Union-bound over c ∈ {0,1}… + another trick.

28. Our result: Larger d

29. d=2: Obj0 arbitrary; Obj1, Obj2ϕ-semi-random Obj1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj2 Obj0

30. V = Obj1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 Obj2 Obj0

31. V = Obj1 0 1 0 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 1 1 1 1 0 ϵ ϵ S Obj2 Obj0

32. more complicated event: Tx,j,S=“x is PO and xj = 1 and Vx ∈ S and first PO to x’s left, call it y, has yj = 0”

33. more complicated event: Tx,j,c,S=“x is PO and xj = c and Vx − ∈ S and first PO to x’s left, call it y, has yj ≠ c”

34. still more complicated event: Tx,J,C,S=“x is PO and…” J: list of d coordinates C: d×d matrix of bits w/ certain pattern S: d-dimensional strip, plus “nearby” d′-dim. stripsfor all d′ < d

35. still more complicated event: Tx,B=“x is PO and conditions involving V, x, B.” J: list of d coordinates C: d×d matrix of bits w/ certain pattern S: d-dimensional strip, plus “nearby” d’-dim. stripsfor all d’ < d + = B: “blueprint” [AMR09]

36. Tx,B=“x is PO and conditions involving V, x, B.” Given B, entries of V are partitioned into two sets, V[fewB] and V[mostB]. V =

37. Tx,B=“x is PO and conditions involving V, x, B.” Given B, entries of V are partitioned into two sets, V[fewB] and V[mostB]. Uniqueness Lemma: Draw V[mostB]. Then ∃ at most 1 x s.t. Tx,B may still occur. Boundedness Lemma: For any V[mostB] outcome,Pr [condits involving V, x, B] ≤ϕd(d+1)/2 ϵd(d+1)/2. V[fewB] Counting Lemma: O(d/ϵ)d(d+1)/2 n2dpossible B. Union bound ⇒ E[# PO’s] ≤O(dϕ)d(d+1)/2n2d.

38. Tx,j,c,S=“x is PO and xj = c and Vx − ∈ S and first PO to x’s left, call it y, has yj ≠ c” Tx,B=“x is PO and conditions involving V, x, B.” J: list of d coordinates C: d×d matrix of bits w/ certain pattern S: d-dimensional strip, plus “nearby” d′-dim. stripsfor all d′ < d + = B: “blueprint”

39. Tx,B=“x is PO and SKETCH(x,V) = B” where SKETCH() is a certain deterministic algorithm.

40. SKETCH(x,V):

41. Tx,B=“x is PO and SKETCH(x,V) = B” Uniqueness Lemma: Draw V[mostB]. Then ∃ at most 1 x s.t. Tx,B may still occur. Bddness Lemma: For any V[mostB] outcome,Pr [SKETCH(x,V) = B] ≤ϕd(d+1)/2 ϵd(d+1)/2. V[fewB]

42. Tx,B=“x is PO and SKETCH(x,V) = B” Uniqueness Lemma: Draw V[mostB]. Then ∃ at most 1 x s.t. Tx,B may still occur. Equivalent Lemma: Suppose x, V are such that x is PO. AssumeSKETCH(x,V) = B. Then RECONSTRUCT(B,V[mostB]) = x.

43. RECONSTRUCT(B,V[most]):

44. Uniqueness Lemma: Suppose x, V are such that x is PO. AssumeSKETCH(x,V) = B. Then RECONSTRUCT(B,V[mostB]) = x. Bddness Lemma: For any V[mostB] outcome,Pr [SKETCH(x,V) = B] ≤ϕd(d+1)/2 ϵd(d+1)/2. V[fewB] Counting Lemma: O(d/ϵ)d(d+1)/2 n2dpossible B.

45. A sketch of SKETCH(x,V)for d = 2

46. SKETCH(x,V): SKETCH(x,V): Let F0= F Let D1 = { z ∈ F0 : V1z > V1x and V2z > V2x } Let y1= argmax{ Obj0(y) : y ∈ D1} Let j1 = least index s.t. Let F1= { z ∈ F0 : Obj0(z) > Obj0(y1) and } Let D2 = { z ∈ F1 : V2z > V2x } Let y2= argmax{ Obj1(y) : y ∈ D2 } Let j2 = least index s.t. Let F2= { z ∈ F1 : Obj1(z) > Obj1(y1) and } // if Vx is PO then it must be argmax{ Obj2(z) : z ∈ F2 } 10. Output: J = (j1, j2), C = S= ( strip of Obj1,2(x) − diag(Vj1,j2 C), strip of Obj2(x) − )

47. Close gap of nd vs. n2d. • Lower bounds when all objs semirandom? • F ⊆ {0,1,2,…,k}n ? • Bounds on Var[ # POs] ? • More Smoothed Analysis via “blueprints”? Future directions?

48. The End. Any questions?