Stabilized column generation

1. Stabilized column generation • Classical column generation often very slow. • Slow convergence (tailing-off effect) • Poor columns in initial stages (head-in effect) • Due to the degeneracy, optimal value of the restricted master problem remains the same during many iterations (plateau effect) • Dual solutions are jumping from one extreme point to another (bang-bang effect) • Intermediate Lagrangian dual bounds do not converge monotonically (yo-yo effect)

2. (Vanderbeck, F., 2005, “Implementing mixed integer column generation”, Column Generation, Kluwer Academic Publishers, Boston, MA.)

3. If we use the optimal dual solution as dual variable values in the column generation, the convergence will be fast. Obtaining optimal dual solution (or close approximation) is important. In dual perspective, column generation is cutting plane method in dual space. We cut off an extreme point in each iteration, which makes the dual solution changes abruptly and makes the convergence slow in the dual space (Kelly’s cutting plane method for convex optimization.) • Any improved method for convex optimization can be used for column generation to obtain dual vector. Idea is to give penalty for the distance of the current best dual solution and the new solution so that the new solution does not deviate much from the current best solution (stabilization). • Remedy: stabilized column generation (du Merle, Villeneuve, Desrosiers, Hansen, 1999, Stabilized column generation, Discrete Mathematics, 194, 229-237. and many others) (See “M. Lübbecke, J. Desrosiers, 2005, Selected Topics in Column Generation, Operations Research, Vol.53, No. 6, pp. 1007-1023” for review of Col. Gen.)

4. (P) min cTx (D) max bT s.t. Ax = b s.t. TA cT x  0 • To overcome degeneracy, may use perturbation. (P ) min cTx s.t. Ax – y- + y+ = b, y- - , y+ + x, y-, y+ 0 • Or may use exact penalty ( 0) (P ) min cTx + y- + y+ s.t. Ax – y- + y+ = b, x, y-, y+ 0 In the dual, dual variables restricted in [-e, e] : prevent bang-bang

5. General form: (P) min cTx - -Ty- + +Ty+ (D) max bT - -Tw- - +Tw+ s.t. Ax – y- + y+ = b, s.t. TA cT y- - , - - w-  - - y+ + - w+  + x, y-, y+ 0 w-, w+ 0 ( - - w-    + + w+ ) • In the dual space, b>0 -w- w+ ’  + - -+ -

6. Let x*, *, ( x*, y-*, y+* ), ( *, w-*, w+* ) be optimal solutions to (P), (D), (P), (D) respectively and v(.) be the optimal value of problem (.). Then (P)  (P) (i.e., y-* = y+* = 0) if one of the following conditions met: (stopping criteria) (i) - = + = 0 (ii) - < * < + • (iii) v(P)  bT*  v(P), hence bT* provides better lower bound than v(P). ( P is a relaxation of P, and * is a feasible solution to (D). )

7. Parameter adjustment (there can be some variants) • If the dual value is too small (  < - ), re-center and enlarge the interval. • If the dual value is within the interval ( -    + ), re-center and reduce the interval. • If the dual value is too large (  > + ), re-center and enlarge the interval.

8. Chebyshev Center Based Column Generation • Chungmok Lee, Sungsoo Park, 2011, Chebyshev center based column generation, Discrete Applied Mathematics 159, 2251-2265. • Chebyshev center: Find the largest Euclidean ball that lies in a polyhedron T. Assume P is bounded and has an interior point. (The center of the optimal ball is called the Chebyshev center of the polyhedron; it is the point deepest inside the polyhedron) • We represent the ball as Consider the simpler constraint that B lies in one halfspaceT, i.e., T. Since supT, have T. Therefore the Chebyshev center can be determined by solving the LP maximize r subject to T Variables are , and r.

9. Chebyshev center was used for convex optimization earlier (Elzinger and Moore (1975), Mathematical Programming, with objective cuts), but never used for column generation. • If a dual feasible solution * (with value ) is given, can add dual objective cut to the dual problem. • The Chebyshev center in the dual polyhedron (with dual objective cut) gives the dual vector for primal column generation. • The Chebyshev center approach can be combined with stabilization.