IEOR 4004 Final Review part II.

1. IEOR 4004 Final Review part II. April 30, 2014

2. Integer Programming max cx Ax = b x ≥ 0 x integer • Standard problems • Knapsack • Fixed-charge Problem (Facility location) • Cutting Stock Problem • Set cover • Travelling Salesman Problem • Job/Machine Scheduling Problems pureIP = all variables integer mixedIP = some variables integer

3. IP formulations • fixed charge (penalty) for a failing a constraint • alternativeconstraints (logical or) • if-then constraints • piece-wise linear objective constraint add constraint penalty for failing the constraint f ≤ My +py f ≤ 0 to the objective y ∈ {0,1} f ≤ 0 f ≤ My y ∈ {0,1} g ≤ 0 g ≤ M(1-y) f ≤ My if f > 0 , then g ≤ 0 y ∈ {0,1} g ≤ M(1-y)

4. Branch-and-Bound max 12x1+ 4x2+ 5x3s.t. 5x1+ 2x2 + 3x3≤ 13 x1,x2,x3≥ 0 max 12x1+ 4x2+ 5x3s.t. 5x1+ 2x2 + 3x3≤ 13 x1,x2,x3≥ 0 and integer • solvethe LP relaxation optimal solution to the LP relaxation gives an upper bound • branch on fractionalvariables (if exist) • how do we (re)-solvethe subproblemsquickly ? • Upper-bounded Dual Simplex • special subproblem(Knapsack, Transportation Problem) fractional solution= solution to the LP relaxation Linear Programming (LP) relaxation drop the integrality Integer Program

5. Upper-bounded Dual Simplex Dually feasible solution Optimal ? 1. check upper bounds 2. check lower bounds given a dually feasiblebasicsolution (all coeffs in z negative) • checkif all basicvariables satisfy their upper bounds if some xifails, if xi should be ≤ B but is > B, then substitute xi = B – xi’ • checkif all basicvariables satisfy their lower bounds if some xi < 0, then use dual ratio test to find xj such that the ratio b/a is smallestpossible where: • a is the coefficient of xj in the equation for xi • (-b) is the coefficient of xj in the equation for z • a is positive pivot xi out and xj into the basis 1. substitute xi = B – xi’ where 0 ≤ xi≤ B 2. pivot (dual ratio test) xi out, xj in the basis x1 = -2 + 2x2 – 4x3 + 3x4 x5 = 1 + x2 + 2x3 – 2x4 z = 5 – 3x2 – 5x3– 4x4 x1 leaves, ratio test: x2: 3 / 2 = 1.5 x4: 4 / 3 = 1.333 x4 enters

6. Branching tree optimal fractional solution not an integersolution branch max 6x1 + 5x2 s.t. 4x1 + 5x2 ≤ 16 8x1 + 5x2 ≤20 x1,x2 ≥ 0 & integer LP relaxation x1 = 1 x2 = 2.4 z = 18 x2 ≤ 2.4 = 2 x2 ≥ 2.4 = 3 x2 ≥ 3 x2 ≤ 2 x1 = 1.25 x2 = 2 z = 17.5 x1 = 0.25 x2 = 3 z = 16.5 optimal solution x1 ≥ 2 x1 ≤ 1 x1 ≥ 1 x1 ≤ 0 infeasible x1 = 2 x2 = 0.8 z = 16 x1 = 0 x2 = 3.2 z = 16 x1 = 1 x2 = 2 z = 16 candidate solution don’t need to branch (no solution here is better than z=16 and we already know a candidate solution of value 16)

7. Example Initial solution x1 = 2.6 x2 = 0 x3= 0 z = 31.2 max 12x1+ 4x2+ 5x3s.t. 5x1+ 2x2 + 3x3+ x4= 13 x1,x2,x3,x4≥ 0 max 12x1+ 4x2+ 5x3s.t. 5x1+ 2x2 + 3x3≤ 13 x1,x2,x3≥ 0 x1 = 2.6 – 0.4x2 – 0.6x3 – 0.2x4 z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4 x2 = 1.5 + 2.5x1’ – 1.5x3– 0.5x4 z = 30 – 2x1’ – x3 – 2x4 x5 = -0.4 – 0.4x2 – 0.6x3 – 0.2x4 z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4 x1’ = -0.6 + 0.4x2 + 0.6x3 + 0.2x4 z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4 x1≥ 3 x1 ≤ 2 x1 = 2 x2 = 1.5 x3= 0 z = 30 infeasible x2≤ 1 x2 ≥ 2 Subproblem: x1 ≥ 3 dictionary is not feasible anymore x1 = 2 x2 = 1 x3= 1/3 z= 29⅔ x1 = 1.8 x2 = 2 x3 = 0 z = 29.6 Repeat How to quickly recover an optimum (without starting over) ? Upper-boundedDualSimplex method x1 ≥ 3 substitute x1 = 3 + x5 x1 ≤ 2 substitute x1 = 2 – x1’ optimal solution recovered x3 ≥ 1 • check upper bounds: OK • check lower bounds: -0.6 = x1’ ≥ 0x1’leaves, ratio test: • x2: 0.8/0.4 = 2 • x3: 2.2/0.6 = 3⅔ • x4: 2.4/0.2 = 12 • check upper bounds: OK • check lower bounds: -0.4 = x5 ≥ 0x5leaves, ratio test: • x2: no constraint • x3: no constraint • x4: no constraint x3 = 0 x1≤ 1 x1= 2 x1’ = 0 x1 = 2–x1’ = 2 x2 = 1.5 x3 = 0 z = 30 x1 = 2 x2 = 1 x3= 0 z= 28 x1 = 2 x2 = 0 x3 = 1 z = 29 x1 = 1 x2 = 4 x3 = 0 z = 28 infeasible not an integersolution branchon x2 ≤ 1 and x2≥ 2 infeasibleLP x2 enters candidate solution optimal candidate solution

8. Knapsack Initial solution max 12x1+ 4x2+ 5x3s.t. 5x1+ 2x2 + 3x3≤ 13 x1,x2,x3≥ 0 and integer x1 = 2.6 x2 = 0 x3= 0 z = 31.2 • optimalgreedystrategy (for LP relaxation) • pick items by value/weight, highest first • take as many items ofthe same type as possible x1≥ 3 x1 ≤ 2 x1 = 2 x2 = 1.5 x3= 0 z = 30 infeasible pick 2 units of item #1 then 1 unit of item #2 and then 1/3 of item #3 x2≤ 1 x2 ≥ 2 substitute x2 = 2 + x5 x1≥ 3 substitute x1 = 3 + x4 x2≤ 1 x2 ≥ 2 x3 ≥ 1 substitute x3 = 1 + x7 max 12x4+ 4x2+ 5x3 + 15s.t. 5x4+ 2x2 + 3x3≤ -2 max 12x1+ 4x5 + 5x3 + 8s.t. 5x1+ 2x5+ 3x3≤ 9 x1 = 2 x2 = 1 x3= 1/3 z= 29⅔ x1 = 1.8 x2 = 2 x3 = 0 z = 29.6 value/ weight 2.4 2 1⅔ max 12x1+ 4x2 + 5x7 + 5s.t. 5x1+ 2x2+ 3x7≤ 10 x3 = 0 do not pick item #3 (1 unit unused in the bag) pick 1.8 units of item #1 x1 ≤ 2 pick 2 units of item #1 and then 1.5 of item #2 (as much as possible) pick 2 units of item #1 x1≤ 1 pick 1 units of item #1 then 2 of item #5 (i.e. 4 units of item #2) x1 = 2 substitute x1 = 2 x3 ≥ 1 x3 = 0 x1≤ 1 x1= 2 x1 = 2 x2 = 1 x3= 0 z= 28 x1 = 2 x2 = 0 x3 = 1 z = 29 x1 = 1 x2 = 4 x3 = 0 z = 28 max 4x5 + 5x3 + 32s.t. 2x5+ 3x3≤ -1 infeasible candidate solution optimal candidate solution

9. Cutting Planes optimal fractional solution max 6x1 + 5x2 s.t. 4x1 + 5x2 ≤ 16 8x1 + 5x2 ≤20 x1,x2 ≥ 0 & integer 4 • only for pureIPs • solve LP relaxation • if not an integeroptimum, make the feasible region smaller by cuttingout fractional points (without removing any integer points) • how to generate cuts ? 3 x1 + x2≤ 3 x1,x2≥ 0 & integer new fractional optimum 2 1 0 1 2 3 4 5 Gomorycuts

10. Gomory cuts optimal solution max 6x1 + 5x2 s.t. 4x1 + 5x2 ≤ 16 8x1 + 5x2 ≤20 x1+ x2≤ 3 x1,x2 ≥ 0 & integer max 6x1 + 5x2 s.t. 4x1 + 5x2 + x3 = 16 8x1 + 5x2 + x4 = 20 x1+ x2 +x5 = 3 x1,x2,x3,x4,x5 ≥ 0 & integer 4 3 optimal fractional solution 2 1 x1 = 1⅔ – ⅓x4 + 1⅔x5 x2 = 1⅓ + ⅓x4– 2⅔x5 x3 = 2⅔ – ⅓x4 + 6⅔x5 z = 16⅔ – ⅓x4 – 3⅓x5 x1 = 1 – x6 + 2x5 x2 = 2 + x6 – 3x5 x3 = 2 – x6 + 7x5 x4= 2 + 3x6– x5 z = 16 – x6 – 3x5 0 1 2 3 4 5 split to wholeand fractionalpart x6 = – ⅔ + ⅓x4 + ⅓x5 z = 16⅔ – ⅓x4 – 3⅓x5 x1 = 1⅔ – ⅓x4 + 1⅔x5 2–⅓ 0+⅓ 1+⅔ negative coeffients x1 – 2x5 – 1 = ⅔ – ⅓x4 – ⅓x5 < 1 ≤ 0 Use DualSimplexto recover the optimum cut positive constant wholepart fractionalpart

11. Dynamic Programming Goal: – valueof the formulafor a targetstate (in the last stage) – saveon calculation by storing the intermediate results fi(j) time (months, years) budget (available resources) optimal substructure property • Break down the problem into stages • each stagehas possible states • next stage depends only on the previous stage • identifypossible decisions/transitionsat each stage • find a formula to evaluate (by recursion) • Evaluate stage by stage fi ( j ) = optimumvalue for the first i stages that ends in statej of stagei 1 1 1 2 2 2 e.g. cost of cheapest production plan for first i months where we end up with j items in inventory at the end of month i 3 3 3 (inventory in month i+1 only depends on production and inventory level in month i) stage 1 stage 2 stage 3

12. Dynamic Programming #1. x1 > 0 thousand $ yields c1(x1)=7x1+2 thousand $ Investments: #2. x2 > 0 thousand $ yields c2(x2)=3x2+7 thousand $ #3. x3 > 0 thousand $ yields c3(x3)=4x3+5 thousand $ c1(0)=c2(0)=c3(0)=0 We can invest up to $6,000. green arrow= best choice of k decisionin stage i = how much we invest into #i (all we need to know is how much money we have availablefor investing) fi( j) = maximumprofitwe can get by investing j thousand dollars into first i investments f0( j) = 0 profit from #i leftover cash + fi-1() fi( j) = j– k ci(k) max { } k ∈ {0,1,...,j} best possible profit with the leftover cash how much we invest in #i 0 0 0 1 1 2 2 3 3 4 4 5 5 6 6 6 #1 #2 #3

13. Traveling Salesman Problem Find a shortestroutethat goes through all cities and comesbackwhere it starts Map of cities Shortest route

14. Traveling Salesman Problem Work backwards – which is the lastvertex visited beforev ? s v f(S,v) = min f(S\{x}, x) + cxv x x ∈ S (x,v) ∈ E initial condition: f(Ø,s) = 0 S\{x} S solution: f(V\{s},s) f(S,v) = length of optimal substructure property a shortestroutestartingat s endingat v and visiting allnodes in S brute force (trying all routes) is O(n!) time and O(1) space complexity: O(n22n) time and space

15. Good luck on the final exam.