1 / 21

A Newton Method for Linear Programming

A Newton Method for Linear Programming. Olvi L. Mangasarian. University of California at San Diego. Outline. Fast Newton method for class of linear programs Large number of constraints (millions) Moderate number of variables (hundreds) Method is based on an overlooked fact

zayit
Download Presentation

A Newton Method for Linear Programming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Newton Method for Linear Programming Olvi L. Mangasarian University of California at San Diego

  2. Outline • Fast Newton method for class of linear programs • Large number of constraints (millions) • Moderate number of variables (hundreds) • Method is based on an overlooked fact • Dual of asymptotic exterior penalty problem of primalLP • Provides an exact least 2-norm solution of the dual LP • Use a finite value of the penalty parameter • Exact least 2-norm dual solution generates: • Highly accurate primal solution when it is unique • Globally convergent Newton method established • Eleven lines of MATLAB code • Method solves LP with 2-million constraints & 100 variables • 14-place accuracy in 17.4 minutes on 400Mhzmachine • CPLEX 6.5 ran out of memory on same problem • Dual method for million-variable LP

  3. Primal LP: Dual LP: Asymptotic Primal Exterior Penalty: Exact Dual Least 2-Norm: Primal-Dual LPsPrimal Exterior Penalty & Dual Least 2-Norm Problem

  4. The Plus Function

  5. Karush-Kuhn-Tucker Optimality Conditionsfor Least 2-Norm Dual Lagrangian: KKT Conditions: Dual Least 2-Norm: Equivalently:

  6. We have: Thus : Hence: Thus: solves the exterior penalty problem for the primal LP Karush-Kuhn-Tucker Optimality Conditionsfor Least 2-Norm Dual (Continued)

  7. Equivalence of Exact Least 2-Norm Dual Solution to Finite-Parameter Primal Penalty Minimization

  8. Gradient: Generalized Hessian: where: Primal Exterior Penalty Primal Exterior Penalty(exact least 2-norm dual solution) :

  9. LP Newton Algorithm (LPN)

  10. LPN Convergence

  11. Remarks on LPN

  12. lpgen: Random Solvable LP Generator %lpgen: Generate random solvable lp: min c'x s.t. Ax =< b; A:m-by-n%Input: m,n,d(ensity); Output: A,b,c; (x,u): primal-dual solutionpl=inline('(abs(x)+x)/2');%pl(us) function tic;A=sprand(m,n,d);A=100*(A-0.5*spones(A));u=sparse(10*pl(rand(m,1)-(m-3*n)/m));x=10*spdiags((sign(pl(rand(n,1)-rand(n,1)))),0,n,n)*(rand(n,1)-rand(n,1));c=-A'*u;b=A*x+spdiags((ones(m,1)-sign(pl(u))),0,m,m)*10*ones(m,1);toc0=toc;format short e;[m n d toc0] • Elements of A uniformly distributed between –50 and +50 • Primal random solution x with elements in [-10,10], approximately half of which are zero • Dual random solution u in [0,10], approximately 3n of which are positive

  13. lpnewt1: MATLAB LPN Algorithm without Armijo %lpnewt1: Solve primal LP: min c'x s.t. Ax=<b %Via Newton for least 2-norm dual LP: max -b'v s.t. -A'v=c, v>=0%Input: c,A,b,epsi,delta,tol,itmax;Output:v(l2norm dual sol),z primal solepsi=1e-3;tol=1e-12;delta=1e-4;itmax=100;%default inputspl=inline('(abs(x)+x)/2');%pl(us) functiontic;i=0;z=0;y=((A(1:n,:))'*A(1:n,:)+epsi*eye(n))\(A(1:n,:))'*b(1:n);%y0while (i<itmax & norm(y-z,inf)>tol & toc<1800) df=A'*pl((A*y-b))+epsi*c; d2f=A'*spdiags(sign(pl(A*y-b)),0,m,m)*A+delta*speye(n); z=y;y=y-d2f\df; i=i+1;endtoc1=toc;v=pl(A*y-b)/epsi;t=find(v);z=A(t,:)\b(t);toc2=toc;format short e;[epsi delta tol i-1 toc1 toc2 norm(x-y,inf) norm(x-z,inf)]

  14. LPN & CPLEX 6.5 Comparison(LPN without Armijo vs. Dual Simplex) (400 Mhz Pentium II 2Gig) (oom=out of memory)

  15. What About the Case of: n>> m ? • LPN is not appropriate • LPN needs to invert a very large n-by-n matrix • Use instead DLPN (Dual LPN): • Find least 2-normprimal solution • By solving dualexterior penalty problem with finite penalty parameter

  16. Equivalence of Exact Least 2-Norm Primal Probelmand Asymptotic Dual Penalty Problem

  17. Replace: • By: Difficulty: Nonnegativity Constrained Dual Penalty • Use an exterior penalty to handle nonnegativity constraint

  18. DLPN & CPLEX 7.5 Comparison(DLPN without Armijo vs. Primal Simplex )(1.8 MHz Pentium 4 1Gig)(oom=out of memory;error 1001)(infs/unbd=presolve determines infeasibility/unboundedness;error 1101)

  19. Conclusion • LPN & DLPN: Fast new methods for solving linear programs • LPN capable of solving problems with millions ofconstraints and hundreds of variables • DLPN capable of solving problems with hundreds ofconstraints and millions of variables • Eleven lines of MATLAB code for each of LPN & DLPN • Competitive with state-of-the-art CPLEX • Very suitable for classification problems • Armijo stepsize not needed in many applications • Typical termination in 3 to 30 steps

  20. Future Work • Investigate & establish finite termination for LPN • Extend to convex quadratic programming • Establish convergence/termination under no assumptions for LPN • Generate self-contained codes for classification and data-mining problems

  21. Paper & MATLAB Code Available www.cs.wisc.edu/~olvi

More Related