600 likes | 729 Views
LP- Based Approximation. Lecture.6. Table of Contents. Lp –rounding Dual Fitting LP-Duality. Linear Programming Problem. A linear programming (LP) problem is an optimization problem in which we minimize or maximize a linear objective function subject to a given set of linear constraints.
E N D
LP- Based Approximation Lecture.6
Table of Contents • Lp –rounding • Dual Fitting • LP-Duality
Linear Programming Problem • A linear programming (LP) problem is an optimization problem in which we minimize or maximize a linear objective function subject to a given set of linear constraints. Example: Minimize 3x1 − 5x2 + 3x3 + 2x4 subject to: 3x1 + 4x2 = 6 −x3 + 2x1 − x2 ≥ 22 x5 ≤ 3.5 x3 + .5x4 = .8 xi ≥ 0 for all i
Solutions • Feasible Solution A feasible solution to a linear program is a solution that satisfies all constraints. • Optimal Solution An optimal solution to a linear program is a feasible solution with the largest(smallest) objective function value for a maximization(minimization) problem.
Many optimization problems involve selecting a subset of a given set of elements. • Examples: A vertex cover is a subset of vertices. A spanning tree is really a subset of edges. A knapsack solution is a subset of items. • Can be formulated as LPs with integrality constraints.
Integer Program • An Integer Program (IP) is an LP with Integrality Constraints • Integrality Constraints: Some or all the variables are constrained to be integers.
Solving Linear/Integer Programming Problems • LPs can be solved efficiently (polynomially but slowly). • IPs generally cannot be solved efficiently (it is NP hard). Some specific IPs can be solved efficiently. Actually, their LP optimal is guaranteed to be integral.
Using Indicator Variables Many selection problems can be formulated as IPs using indicator variables (or 0-1 variables). • An indicator variable is defined for each element . A value of 1 indicating the selection of the element and a value of 0 indicating otherwise.
Few Examples are : • vertex cover • Set Cover • Knapsack
Example: Unweighted Vertex Cover • Variables: {xv | v ∈ V }. • The IP: Minimize ∑ xv s.t. xu + xv ≥ 1 ∀ (u, v) ∈ E, xv ∈ {0, 1} ∀ v ∈ V.
Example: Knapsack • Let the item names be {1, . . . , n}. • Variables: {xi | 1 ≤ i ≤ n}. • The IP: • Max ∑i cixi s.t. ∑isixi ≤ K, xi ∈ {0, 1} ∀ 1 ≤ i ≤ n.
Solving Linear/Integer Programming Problems • LPs can be solved efficiently (polynomially but slowly). • IPs generally cannot be solved efficiently (it is NP hard). Some specific IPs can be solved efficiently. Actually, their LP optimal is guaranteed to be integral.
LP Relaxation (Drop the integrality constraint) • Example: Unweighted Vertex Cover • The IP: Minimize ∑vxv s.t. xu + xv ≥ 1 ∀ (u, v) ∈ E, xv ∈ {0, 1} ∀ v ∈ V. • The LP relaxation: Minimize ∑vxv s.t. xu + xv ≥ 1 ∀ (u, v) ∈ E, xv >= 0 ∀ v ∈ V.
Example: Weighted Vertex Cover • Variables: {xv | v ∈ V }. • The IP: Min ∑Cv xv where Cv : cost associated with vertex xv : indicator variable s.t: xu + xv ≥ 1 ∀ (u, v) ∈ E xv ∈ {0, 1} ∀ v ∈ V
LP Relaxation (Drop the integrality constraint) • Example: Weighted Vertex Cover • The IP: Min ∑Cv xv s.t: xu + xv ≥ 1 ∀ (u, v) ∈ E xv ∈ {0, 1} ∀ v ∈ V • The LP relaxation: Min ∑Cv xv s.t: xu + xv ≥ 1 ∀ (u, v) ∈ E xv ≥ 0 ∀ v ∈ V
LP rounding • If xv ≥½, round it up to 1 Else round it down to 0. • Here xv is the solution obtained from LP • E.g: LP: ¼ c1 + ½ c2 + ¾ c3 + 4∕5 c4 IP : c2 + c3 + c4
Claim 1: Solution Obtained is feasible Let (u,v) ∈ E Since the solution of LP is feasible, values of xv , v ∈ V, satisfy xu + xv ≥ 1 (1) ⇒ atleast one of xu and xv ≥ ½ Assume x’u and x’v be the solutions obtained after rounding, then at least one of them must be 1, i.e. x’u + x’v ≥ 1 So the solution, obtained after rounding, is feasible.
Claim 2: C(S) ≤ 2LOPT • According to the strategy some of the variables have been increased to a maximum of double & some have been reduced to 0, i.e Cv’<= 2Cv.
So, C(S): cost of solution obtained by IP C(S) ≤ ∑v’ Cv’xv’ ≤ 2 ∑v Cv xv ( x’v ≤ 2* Xv ) =2 LPOPT Hence claim 2 follows
Set Cover Problem A finite set (universe) U of n elements, U= {e1, e2,…, en}, a collection of subsets of U i.e. S1, S2,…., Sk with some cost, select a minimum cost collection of these sets that covers all elements of U.
IP: • Indicator variable xs, xs∈{0,1} xs =0 if set S is not picked xs =1 if set S is picked Min ∑s Cs xs s.t. ∑s:e belongs to S xs ≥ 1 ∀ e ∈ U xs = {0,1} LP Relaxation: Min ∑s Cs xs s.t. ∑s:e belongs to S xs ≥ 1 ∀ e ∈ U xs> 0
LP rounding for SC Let f denote the maximum frequency of any element in U Si • Find an optimal solution to LP-Relaxation xs >1/f round it to 1 xs <1/f discard the set, i.e. round it down to 0.
Claims • Claim 1: solution is feasible • Claim 2: It gives factor f approximation
Claim 1: Solution is feasible Let, ei ∈ U , 1≤i≤n S be the collection of subsets of U em : 1<m <n belongs to l subsets of S where 1<l<k Since the solution of LP is feasible i.e. values of xs s ∈ S obtained satisfies xs1 + xs2 + xs3 + ….+ xsl>1 (1) ⇒ atleast one of xs1, xs2, xs3,…., xsl>1/f ⇒ x’s1 + x’s2 + x’s3 +….+ xsl’> 1 Where x’si is the solution obtained after rounding. Thus it is feasible.
Claim 2: Factor f approximation • For each set s ∈ Collection of picked sets(S), xs has been increased by a factor of atmost f. Let C(s): Cost of our solution Therefore, C(S) ≤ ∑s Csx’s ∀ s ∈ S ≤ f ∑s Cs xs ( x’s ≤ f* xs) =f LPOPT Hence it is a factor ’f’ approximation. Note: f factor could be large. Later we’ll see a technique of rounding that gives O(log n) factor.
Linear Programming - Example Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to: 3x1 + 4x2 ≥ 6 3x2 + x3 + x4 ≥ 5 xi ≥ 0 for all i x = (2, 1,0, 3) is a feasible solution. 8*2 + 5*1 + 2*3 = 27 is an upper bound.
What is the Lower Bound? Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to: 3x1 + 4x2 ≥ 6 3x2 + x3 + x4 ≥ 5 xi ≥ 0 for all i LB: 8x1 + 5x2 + 5x3 + 2x4 ≥ 3x1 + 4x2 ≥ 6 Better LB: 8x1 + 5x2 + 5x3 + 2x4 ≥ (3x1 + 4x2 ) + (3x2 + x3 + x4) ≥ 6+5 = 11
How to compute a good LB Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to: 3x1 + 4x2 ≥ 6 ……………….y1 3x2 + x3 + x4 ≥ 5……………y2 xi ≥ 0 for all i • Assign a non-negative coefficient yi to every inequality such that • 8x1 + 5x2 + 5x3 + 2x4 ≥ y1 (3x1 + 4x2 ) + y2(3x2 + x3 + x4 ) • Then, LHS ≥ 6y1 + 5y2. We are interested in finding yi’s such that RHS is maximum. This leads to our dual problem.
The corresponding dual for the given example will be: max 6y1 + 5y2 such that, 3y1 < 8 4y1 + 3y2< 5 y1< 5 y2< 2 and, yi > 0 for all i
Weak Duality Theorem Theorem: If x and y are feasible then, > Proof: > = >
Set Cover xs is 1 iff set S in included in the cover. The Primal : Objective : min ∑ Cs xs s.t > 1 U xs = {0,1} LP relaxation: xs > 0
Introduce an indicator variable ye for each of the constraints in primal. The Dual : objective: max s.t <CSi for i = 1 to k
Example S = { x, y, z, w} S1 = { x, y} S2 = { y, z} S3 = { x, w, y} Let xs1 , xs2 , xs3 be an indicator variable for S1 , S2 , S3 respectively. Let Cs1 , Cs2 , Cs3 is the cost of S1 , S2 , S3 respectively.
Primal Min : Cs1 xs1 + Cs2 x2+ Cs3x3 Subject to xs1 + xs3> 1 (yx) xs1 + xs2 + xs3> 1 (yy) xs2> 1 (yz) xs3> 1 (yw) xs1, xs2, xs3> 0
Dual Max: yx + yy + yz + yw Subject to yx + yy< Cs1 yy + yz< Cs2 yx + yy + yw< Cs3 yx, yy , yz, yw > 0
Dual Fitting From set cover via lp
Example: Weighted Vertex Cover Primal: Min ∑Cv xv s.t: xu + xv ≥ 1 ∀ (u, v) ∈ E xv ∈ {0, 1} ∀ v ∈ V Dual: Max ∑ye s.t: ∑e:e is incident on v ye < Cv ∀ v ∈ V ye ∈ {0, 1} ∀ e ∈ E
Primal Dual Schema 1 • U = empty, y = 0 • For each edge e = (u, v) • ye = min {c(u) − ∑e′:u∈e′ye′ , c(v) − ∑e′:v∈e′ye′ } • U = U union argmin {c(u) − ∑e′:u∈e′ye′ , c(v) − ∑e′:v∈e′ye′ } • Output U
3 5 2 7 4 3 1 3 2
3 5 2 7 4 (1) Ye =3 3 (0) 1 3 2 For every edge pick minimum of two vertices Min{4,3} = 3 Set ye=3 U has vertex having red color
3 5(4) 2 Ye =1 7 4 (1) (0) Ye =3 3(0) 1 3 2 Min{1,5} = 1 Set ye=1
3 5(4) 2 Ye =1 7 4 (1) (0) Ye =3 3(0) Ye =0 Ye =0 Ye =0 1 3 2 Min{1,0} = 0 Set ye=0 Min{2,0} = 0 Set ye=0 Min{3,0} = 0 Set ye=0
3 5(4)(0) 2 Ye =1 Ye =4 7(3) 4 (1) (0) Ye =3 3(0) Ye =0 Ye =0 Ye =0
3 Ye =0 5(4)(0) 2 Ye =1 Ye =4 7(3) 4 (1) (0) Ye =3 3(0) Ye =0 Ye =0 Ye =0
3 (1) Ye =2 Ye =0 2 (0) 5(4)(0) Ye =1 Ye =4 7(3) 4 (1) (0) Ye =3 3(0) Ye =0 Ye =0 Ye =0 Red-colored nodes form a vertex-cover