MCS 312: NP Completeness and Approximation algorthms

1. MCS 312: NP Completeness and Approximation algorthms Instructor Neelima Gupta ngupta@cs.du.ac.in

2. LP- Based Approximation

3. Table of Contents • Lp –rounding • Dual Fitting • LP-Duality

4. Linear Programming Problem • A linear programming (LP) problem is an optimization problem in which we minimize or maximize a linear objective function subject to a given set of linear constraints. Example: Minimize 3x1 − 5x2 + + 3x3 + 2x4 subject to: 3x1 + 4x2 = 6 −x3 + 2x1 − x2 ≥ 22 x5 ≤ −3.5 x3 + .5x4 = .8 xi ≥ 0 for all i

5. Solutions • Feasible Solution A feasible solution to a linear program is a solution that satisfies all constraints. • Optimal Solution An optimal solution to a linear program is a feasible solution with the largest(smallest) objective function value for a maximization(minimization) problem. Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

6. Many optimization problems involve selecting a subset of a given set of elements. • Examples: A vertex cover is a subset of vertices. A spanning tree is really a subset of edges. A knapsack solution is a subset of items. • Can be formulated as LPswith integrality constraints.

7. Integer Program • An Integer Program (IP) is an LP with Integrality Constraints • Integrality Constraints: Some or all the variables are constrained to be integers.

8. Solving Linear/Integer Programming Problems • LPs can be solved efficiently (polynomially but slowly). • IPs generally cannot be solved efficiently (it is NP hard). Some specific IPs can be solved efficiently. Actually, their LP optimal is guaranteed to be integral. Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

9. Using Indicator Variables Many selection problems can be formulated as IPs using indicator variables (or 0-1 variables). • An indicator variable is defined for each element . A value of 1 indicating the selection of the element and a value of 0 indicating otherwise.

10. Few Examples are : • vertex cover • Set Cover • Knapsack Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

11. Example: Unweighted Vertex Cover • Variables: {xv | v ∈ V }. • The IP: Minimize ∑ xv s.t. xu + xv ≥ 1 ∀ (u, v) ∈ E, xv ∈ {0, 1} ∀ v ∈ V.

12. Example: Knapsack • Let the item names be {1, . . . , n}. • Variables: {xi | 1 ≤ i ≤ n}. • The IP: • Minimize ∑i cixi s.t. ∑isixi ≤ K, xi ∈ {0, 1} ∀ 1 ≤ i ≤ n.

13. Solving Linear/Integer Programming Problems • LPs can be solved efficiently (polynomially but slowly). • IPs generally cannot be solved efficiently (it is NP hard). Some specific IPs can be solved efficiently. Actually, their LP optimal is guaranteed to be integral.

14. LP Relaxation (Drop the integrality constraint) • Example: Unweighted Vertex Cover • The IP: Minimize ∑vxv s.t. xu + xv ≥ 1 ∀ (u, v) ∈ E, xv ∈ {0, 1} ∀ v ∈ V. • The LP relaxation: Minimize ∑vxv s.t. xu + xv ≥ 1 ∀ (u, v) ∈ E, xv >= 0 ∀ v ∈ V.

15. Example: Weighted Vertex Cover • Variables: {xv | v ∈ V }. • The IP: Min ∑Cv xv where Cv : cost associated with vertex xv : indicator variable s.t: xu + xv ≥ 1 ∀ (u, v) ∈ E xv ∈ {0, 1} ∀ v ∈ V Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

16. LP Relaxation (Drop the integrality constraint) • Example: Weighted Vertex Cover • The IP: Min ∑Cv xv s.t: xu + xv ≥ 1 ∀ (u, v) ∈ E xv ∈ {0, 1} ∀ v ∈ V • The LP relaxation: Min ∑Cv xv s.t: xu + xv ≥ 1 ∀ (u, v) ∈ E xv ≥ 0 ∀ v ∈ V Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

17. LP- Rounding

18. LP rounding • If xv ≥½, round it up to 1 Else round it down to 0. • Here xv is the solution obtained from LP • E.g: LP: ¼ c1 + ½ c2 + ¾ c3 + 4∕5 c4 IP : c2 + c3 + c4 Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

19. Claim 1: Solution Obtained is feasible Let (u,v) ∈ E Since the solution of LP is feasible, values of xv , v ∈ V, satisfy xu + xv ≥ 1 (1) ⇒ atleast one of xu and xv ≥ ½ Assume x’u and x’v be the solutions obtained after rounding, then at least one of them must be 1, i.e. x’u + x’v ≥ 1 So the solution, obtained after rounding, is feasible. Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

20. Claim 2: C(S) ≤ 2LOPT • According to the strategy some of the variables have been increased to a maximum of double & some have been reduced to 0, i.e Cv’<= 2Cv. Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

21. So, C(S): cost of solution obtained by IP C(S) ≤ ∑v’ Cv’xv’ ≤ 2 ∑v Cv xv ( x’v ≤ 2* Xv ) =2 LPOPT Hence claim 2 follows Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

22. Set Cover Problem A finite set (universe) U of n elements, U= {e1, e2,…, en}, a collection of subsets of U i.e. S1, S2,…., Sk with some cost, select a minimum cost collection of these sets that covers all elements of U. Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

23. IP: • Indicator variable xs, xs∈{0,1} xs =0 if set S is not picked xs =1 if set S is picked Min ∑s Cs xs s.t. ∑s:e belongs to S xs ≥ 1 ∀ e ∈ U xs = {0,1} LP Relaxation: Min ∑s Cs xs s.t. ∑s:e belongs to S xs ≥ 1 ∀ e ∈ U xs> 0 Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

24. LP rounding for SC Let f denote the maximum frequency of any element in U Si • Find an optimal solution to LP-Relaxation xs>1/f round it to 1 xs<1/f discard the set, i.e. round it down to 0. Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

25. Claims • Claim 1: solution is feasible • Claim 2: It gives factor f approximation Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

26. Claim 1: Solution is feasible Let, ei ∈ U , 1≤i≤n S be the collection of subsets of U em : 1<m <n belongs to l subsets of S where 1<l<k Since the solution of LP is feasible i.e. values of xs s ∈ S obtained satisfies xs1 + xs2 + xs3 + ….+ xsl>1 (1) ⇒ atleast one of xs1, xs2, xs3,…., xl >1/f ⇒ x’s1 + x’s2 + x’s3 +….+ x’l> 1 Where x’si is the solution obtained after rounding. Thus it is feasible. Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

27. Claim 2: Factor f approximation • For each set s ∈ Collection of picked sets(S), xs has been increased by a factor of atmost f. Let C(s): Cost of our solution Therefore, C(S) ≤ ∑s Csx’s ∀ s ∈ S ≤ f ∑s Cs xs ( x’s ≤ f* xs) =f LPOPT Hence it is a factor ’f’ approximation. Note: f factor could be large. Later we’ll see a technique of rounding that gives O(log n) factor. Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

28. LP- Duality

29. Linear Programming - Example Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to: 3x1 + 4x2 ≥ 6 3x2 + x3 + x4 ≥ 5 xi ≥ 0 for all i x = (2, 1,0, 3) is a feasible solution. 8*2 + 5*1 + 2*3 = 27 is an upper bound.

30. What is the Lower Bound? Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to: 3x1 + 4x2 ≥ 6 3x2 + x3 + x4 ≥ 5 xi ≥ 0 for all i LB: 8x1 + 5x2 + 5x3 + 2x4 ≥ 3x1 + 4x2 ≥ 6 Better LB: 8x1 + 5x2 + 5x3 + 2x4 ≥ (3x1 + 4x2 ) + (3x2 + x3 + x4) ≥ 6+5 = 11

31. How to compute a good LB Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to: 3x1 + 4x2 ≥ 6 ……………….y1 3x2 + x3 + x4 ≥ 5……………y2 xi ≥ 0 for all i • Assign a non-negative coefficient yi to every inequality such that • 8x1 + 5x2 + 5x3 + 2x4 ≥ y1 (3x1 + 4x2 ) + y2(3x2 + x3 + x4 ) • Then, LHS ≥ 6y1 + 5y2. We are interested in finding yi’s such that RHS is maximum. This leads to our dual problem.

32. The corresponding dual for the given example will be: max 6y1 + 5y2 such that, 3y1 < 8 4y1 + 3y2< 5 y1< 5 y2< 2 and, yi> 0 for all i Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

33. Weak Duality Theorem Theorem: If x and y are feasible then, > Proof: > = > Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

34. Set Cover xs is 1 iff set S in included in the cover. The Primal : Objective : min ∑ Cs xs s.t > 1 U xs = {0,1} LP relaxation: xs > 0 Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

35. Introduce an indicator variable ye for each of the constraints in primal. The Dual : objective: max s.t <CSi for i = 1 to k Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

36. Example S = { x, y, z, w} S1 = { x, y} S2 = { y, z} S3 = { x, w, y} Let xs1 , xs2 , xs3 be an indicator variable for S1 , S2 , S3 respectively. Let Cs1 , Cs2 , Cs3 is the cost of S1 , S2 , S3 respectively. Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

37. Primal Min : Cs1 xs1 + Cs2 x2+ Cs3x3 Subject to xs1 + xs3> 1 (yx) xs1 + xs2 + xs3> 1 (yy) xs2> 1 (yz) xs3> 1 (yw) xs1, xs2, xs3> 0 Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

38. Dual Max: yx + yy + yz + yw Subject to yx + yy< Cs1 yy + yz< Cs2 yx + yy + yw< Cs3 yx, yy , yz, yw > 0 Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

39. Dual Fitting From set cover via lp

40. Primal-Dual Schema

41. Complementary Slackness Conditions

42. Relaxed Complementary Slackness Conditions

43. Example: Weighted Vertex Cover Primal: Min ∑Cv xv s.t: xu + xv ≥ 1 ∀ (u, v) ∈ E xv ∈ {0, 1} ∀ v ∈ V Dual: Max ∑ye s.t: ∑e:e is incident on v ye < Cv ∀ v ∈ V ye ∈ {0, 1} ∀ e ∈ E

44. Primal Dual Schema 1 • U = empty, y = 0 • For each edge e = (u, v) • ye = min {c(u) − ∑e′:u∈e′ye′ , c(v) − ∑e′:v∈e′ye′ } • U = U union argmin {c(u) − ∑e′:u∈e′ye′ , c(v) − ∑e′:v∈e′ye′ } • Output U

45. 3 5 2 7 4 3 1 3 2 Thanks to Neha& Neha Katyal

46. 3 5 2 7 4 (1) Ye =3 3 (0) 1 3 2 For every edge pick minimum of two vertices Min{4,3} = 3 Set ye=3 U has vertex having red color Thanks to Neha& Neha Katyal

47. 3 5(4) 2 Ye =1 7 4 (1) (0) Ye =3 3(0) 1 3 2 Min{1,5} = 1 Set ye=1 Thanks to Neha& Neha Katyal

48. 3 5(4) 2 Ye =1 7 4 (1) (0) Ye =3 3(0) Ye =0 Ye =0 Ye =0 1 3 2 Min{1,0} = 0 Set ye=0 Min{2,0} = 0 Set ye=0 Min{3,0} = 0 Set ye=0 Thanks to Neha& Neha Katyal

49. 3 5(4)(0) 2 Ye =1 Ye =4 7(3) 4 (1) (0) Ye =3 3(0) Ye =0 Ye =0 Ye =0 Thanks to Neha& Neha Katyal

50. 3 Ye =0 5(4)(0) 2 Ye =1 Ye =4 7(3) 4 (1) (0) Ye =3 3(0) Ye =0 Ye =0 Ye =0 Thanks to Neha& Neha Katyal