Introduction to LP Relaxation and SDP Relaxation

1. Introduction to LP Relaxation and SDP Relaxation Speaker: Yao-Ting Huang Advisor: Kun-Mao Chao National Taiwan University Department of Computer Science & Information Engineering Algorithms and Computational Biology Lab.

2. References • Hochbaum, D.S. Approximation algorithms for NP-hard problems, PWS Publishing Company, 1997 • Klerk, E.D. Aspects of semidefinite programming: Interior Point Algorithms and Selected Applications. Kluwer Acdemic Publisheres, 2002. • Wolkowicz, H., Saigal R., and Vandenberghe L. Handbook of semidefinite programming: Theory, Algorithms, and Applications. Kluwer Academic Publishers, 2000. • Bretscher, O. Linear Algebra with Applications. 1997 • Chen, T. Haplotype Inference by maximum parsimony.

3. Outline • Approaches for NP-hard problems • Introduction to LP-relaxation • An iterative LP-relaxation algorithm • Introduction to SDP-relaxation • An SDP-relaxation algorithm for the MAXCUT problem

4. Approaches to NP-hard problems • Heuristic methods • e.g., greedy approach • Formulate the problem to an integer problem • Relax toaLinear Programming (LP)problem and solve it. • Formulate the problem to an integer quadratic problem • Relax toaSemi-Definite Programming (SDP)problem and solve it. • The analysis is based on linear algebra.

5. S4 S4 S4 S4 S1 S1 S1 S1 S2 S3 S2 S3 An Example for the Greedy Approach • C is the set of robust tag SNPs which allow m missing SNPs • iff each pair of patterns is distinguished by at least (m+1) SNPs • iffC covers each pair of patterns for at least (m+1) times. S1 S2 S3 S4 (1,2) (1,3) (1,4) (2,3) (2,4) (3,4)

6. LP Relaxation relaxation Integer Program Linear Program Approximation Solver (e.g., Simplex Method) Integral solution Fractional solution Randomized Rounding Lu’s illustration

7. An example for the LP-relaxation Approach (1/2) • Given NSNPs, find the minimum number of robust tag SNPs which allow m missing tag SNPs. • Define D(Pi , Pj) as the set of SNPs which are able to distinguish Piand Pj patterns. • Definexk= 1 if the k-th SNP is selected and 0 otherwise. Integer problem formulation:

8. An example for the LP-relaxation Approach (2/2) LP relaxation: Randomized rounding: • Lety1, y2, …, yNbe the optimal solution obtained from the above LP problem. • Randomized rounding method: • Assignxk= 1 with probability yk • Assignxk= 0 with probability 1-yk • This randomized rounding method may violate some integral constraints.

9. An iterative LP-relaxationalgorithm (1/2) • Step 1. Formulate the integer linear problem. • Step 2. Relax the integer constraint to linear constraint. • Step 3. Obtain the optimal solution of the LP problem and perform randomized rounding. • Step 4. The randomized rounding may invalidate some integral inequalities. We repeat Steps 1, 2, and 3 for those unsatisfied inequalities until all of them are satisfied. Consider a simpler case

10. An iterative LP-relaxationalgorithm (2/2) relaxation Integer Linear Program Linear Program Approximation yes Solver no (e.g., Simplex Method) All inequalities satisfied? Fractional solution Integral solution Randomized Rounding

11. Analysis of the iterative LP-relaxation algorithm (1/2) • The expected sum of xk equals the optimal solution of the LP problem, OPT(LP). • The probability that one inequality is not satisfied is

12. The probability that all K pairs of patterns can be distinguished aftert iterations is Analysis of the iterative LP-relaxation algorithm (2/2) • Let the set of solutions returned at each iteration be Z1, Z2, …, Zt .

13. SDP Relaxation relaxation reformulation Integer Quadratic Program Vector Program Semidefinite Program Solver Approximation (e.g., Interior point method) Vector Solution Semidefinite Solution Integral Solution Incomplete Cholesky Decomposition Randomized Rounding

14. Linear Algebra Background • A symmetric nnmatrix A is positive semidefiniteiffxTAx  0, for every xRn. • Denoted as A 0 • All the eigenvalues of A are non-negative. • A=BTB , for some mnmatrix B. • The inner product of symmetric matrices A and B is

15. Semidefinite Programming Linear Programming Semidefinite Programming max cx s.t. ai x  bi x  0 max C  X s.t. Ai X Ai X 0 Can be solved exactlyin polynomial time Can be solved almost exactlyin polynomial time

16. The First Application of SDP on Approximation Algorithms • 0.878 randomized approximation ratio for the MAXCUT problem by SDP relaxation technique. • Goemans, M. and Williamson, D. at ACM STOC 1994.

17. 1 1 1 1 The MAXCUT Problem • Given an undirected graph with n nodes G={x1,x2, …, xn}, find a cut to maximize the number of edges on the cut. • Let xi be 1 if the vertex is at one side of the cut, and -1 if the vertex is at the other side of the cut. -1 -1 -1

18. Integer Quadratic Programming • Define aij be 1 if the edge (xi,xj) exists and 0 otherwise. • Relax the integer constraint of xi to be the unit length vector in dimensionm(i.e., xi xi=1).

19. Semidefinite Programming Formulation Let X be (v1,v2, …, vn)T (v1, v2,…, vn). x2 x1 x3 x4 Note thatX is a positive semidefinite matrix.

20. vi vj Ө Randomized Rounding Method • Once X is found, perform Cholesky decomposition to obtain the vector solutions {v1, v2, …, vn}. • Pick a random unit vector r and • Set xi = 1 if vi  r ≥0 • Set xi = -1 if vi  r <0 • Note that cosӨ = vi  vj • The edge (vi , vj) is on the cut iff (vi  r ) and (vj r) has different sign. r 1 1 1 -1 -1

21. Analysis Denote C as the size of the cut found by the above algorithm. The expectation that each edge (xi , xj) is the solution is