CS5 3 21 Numerical Optimization

CS5321 Numerical Optimization 14 Linear Programming: The Interior Point Method

Interior point methods for LP • There are many variations of IPM for LP • Primal IPM, dual IPM, primal-dual IPM • Potential Reduction Methods • Path Following Methods • Short-step methods • Long-step methods • Predictor-corrector methods • We will cover the path following method and the long-step method

The dual problem of LP minxz=cTx subject to Ax = b, x  0 • The dual problem is (see chap 12) maxbT subject to AT +s = c, s  0 • Weak duality For any x and  that are feasible in the primal and the dual problems, bT  cTx. • Strong duality If x* and * are solutions to the primal and the dual problems, bT* = cTx*.

T ¸ A + s c = b A x = 0 ¸ x s ; 0 x s = i i Optimality conditions of LP minxz=cTx subject to Ax = b, x  0 • The Lagrangian function L(x,,s) = cTx − T(Ax−b) − sTx • The optimality conditions (KKT, see chap 12) • The last one is the complementarity condition

0 1 ¢ x 0 T 1 ¸ A + ¡ s c ( ) ( ) ¸ ¸ ¢ ¸ J F ¡ x s x s = @ A ; ; ; ; ( ) b ¸ f A F 0 0 ¸ ¡ x x s o r x s = = @ A ¢ ; ; ; ; s X S e Primal-dual method • The optimality conditions can be expressed as • X=diag(x), S=diag(s) and e=(1,1,…,1)T. • F is a nonlinear equation: R2n+m  R2n+m. • Solved by the Newton’s method: (J the Jacobian of F.) (x, , s)+=(x, , s),

0 0 T 1 1 0 0 T 1 1 ¸ r r r A A I 0 + ¡ r r r r s c ¸ x c c s c c ( ) b ¸ r r A r A J F 0 0 ¡ r r x r r x s = = = = b ¸ b b b @ @ A A @ @ A A x s ; ; r r X r S S X 0 r r r e r ¸ X S X S X S X S x s Jacobian of F • Let . • The Jacobian J (x, , s) of F(x, , s) is

µ 0 T T ¶ µ 1 0 ¶ 1 µ 0 1 ¶ 2 1 ¡ ¡ 1 1 ¡ ¡ D A A I ¢ ¢ X 0 ¢ X ¡ X S ¢ ¡ + ¡ ¡ ¡ x x r r r X S s r x = X S = c = c 1 2 1 2 ¡ D S X = = ¸ ¸ A A ¢ ¢ 0 0 0 ¡ ¡ r r = b b @ A @ A @ A S X ¢ 0 ¡ s r X S Solving the linear systems • The Newton’s direction is obtained by solving • S is an artificial variable. Let • The left hand side is symmetric, but indefinite.

n T 1 x s X ¹ x s = = i i n n i 1 = Without centering Centered direction Centering • In practice, rXS=XSe −e rather than rXS=XSe. • Duality measure: • Centering parameter   [0,1]: • Decreasing as approaching solutions • This is related to the barrier method (chap 15)

The path following algorithm • Given (x0,0,s0) with (x0,s0)>0 • For k = 0, 1, 2 …until (xk)Tsk<  • Choose k[0,1] and let k=(xk)Tsk/n. • Solve the Newton’s direction (xk, k, sk) • Set (xk+1, k+1, sk+1)=(xk, k, sk) + k(xk, k, sk), where k is chosen to make (xk+1,sk+1)>0

Central path and neighborhood • Feasible set F0={(x,,s) | Ax=b, AT+s=c, (x,s)>0} • Point (x,,s)F0 is on the central pathC if x(i)s(i)=  for all i=1,2,…,n. • An example of the neighborhood of the central path •  is the duality measure (two slides before) • Typically =10−3.

Long-step path-following • Given (x0,0,s0)N− and , min, max. • For k = 0, 1, 2 …until (xk)Tsk<  • Choose k[min,max] and let k=(xk)Tsk/n. • Solve the Newton’s direction (xk, k, sk) • Set (xk+1, k+1, sk+1)=(xk, k, sk) + k(xk, k, sk), where k is chosen to make (xk+1, k+1,sk+1)N−.

Complexity • The long step following method can converge in O(nlog(1/)) iterations (Theorem 14.4) • The most effective interior point method for LP is the predictor-corrector algorithm (Mehrotra 1992)

CS5 3 21 Numerical Optimization