Computational Methods for Management and Economics Carla Gomes

1. Computational Methods forManagement and EconomicsCarla Gomes Module 7a Duality

2. Duality • Every maximization LP problem in the standard form gives rise to a minimization LP problem called the dual problem • Every feasible solution in one yields a bound on the optimal value of the other • If one of the problems has an optimal solution, so does the other and the two optimal values coincide • These results have very interesting economic interpretations

3. Bounds on the optimal value Let x1 = the number of doors to produce x2= the number of windows to produce Maximize Z= 3 x1 + 5 x2 subject to x1≤ 4 2x2≤ 12 3x1 + 2x2≤ 18 and x1 ≥ 0, x2 ≥ 0. Lower bound (LB) on Z*  Z* ≥ LB any feasible solution Upper bound (UB) on Z*  Z* ≤UB – how do we come up with upper bounds for Z* for a maximization LP problem?

4. “Guessing” upper bounds

5. A principled way of finding upper-bounds on Z* Dual Problem

6. Primal vs. Dual Problem Maximize Z= 3 x1 + 5 x2 subject to x1 ≤ 4 2x2≤ 12 3x1 + 2x2≤ 18 and x1 ≥ 0, x2 ≥ 0. Minimize W= 4 y1 + 12 y2+ 12 y3 subject to y1 + 3 y3≥ 3 2 y2+ 2 y3 ≥ 5 and y1 ≥ 0, y2 ≥ 0, y3 ≥ 0

7. Primal vs. dual • The dual of a maximization problem is a minimization problem; the dual of a minimization problem is a maximization problem • The m primal constraints are in a one-to-one correspondence with the m dual variables yi, and conversely, the n dual constraints are in a one-to-one correspondence with the n primal variables, xj. • The coefficient at each variable in the objective function, primal or dual, appears in the other problem the RHS of the corresponding constraint.

8. Primal vs. Dual Max Z = c1 x1 + c2x2 + …. + cnxn Subject to: a11 x1 + a12 x2 + … + a1n xn≤ b1 a21 x1 + a22 x2 + … + a2n xn≤ b2 … am1 x1 + am2 x2 + … + amn xn≤ bm Min W = b1 y1 + b2y2 + …. + bmym Subject to: a11 y1 + a21 y2 + … + am1 ym≥ c1 a12 y1 + a22 y2 + … + am2 ym≥ c2 … a1n y1 + a2n y2 + … + amn yn≥ cn x1, x2, …, xn≥ 0 y1, y2, …, ym≥ 0

9. Max Z = cx Max W = bTy Ax≤ b x≥ 0 ATy≤ cT y≥ 0 Primal vs. Dual

10. Primal vs. Dual Problem Maximize Z= 3 x1 + 5 x2 subject to 1 x1 ≤ 4 2 x2≤ 12 3 x1 + 2 x2≤ 18 and x1 ≥ 0, x2 ≥ 0. Minimize W= 4 y1 + 12 y2+ 18 y3 subject to 1 y1 + 3 y3≥ 3 2 y2+ 2 y3 ≥ 5 and y1 ≥ 0, y2 ≥ 0, y3 ≥ 0

11. Wyndor Glass: Primal vs. DualMatrix Notation Min W= 4 12 18y1 y2 y3 Max Z = 3 5x1 x2 Subject to: 3 5 • 0 3 • 0 1 2 y1 y2 y3 • 4 • 12 • 18 ≥ • 1 0 • 0 1 • 3 2 x1 x2 ≤ x≥ 0 y≥ 0

12. Primal-dual table for LP

13. Primal Dual Max Min s.t

14. Every feasible dual solution yields an upper-bound on Z* of the primal and every feasible primal solution yields a lower-bound on Z* of the dual Weak duality property Upper Bounds on Primal Optimum Dual feasible solutions (Minimization) Z* Lower Bounds on Dual Optimum Primal feasible solutions (Maximization)

15. Weak duality property Weak duality property Every feasible dual solution yields an upper-bound on Z* of the primal and every feasible primal solution yields a lower-bound on Z* of the dual

16. Proof: Weak duality property

17. Primal vs. dual Simplex My-wyndor.txt My-wyndor-dual.txt

18. The magic continues … Final tableau - primal • Final row (0) of the • simplex automatically • provides the optimal • values of the • dual variables, y*i , • they correspond to the slack variables in the final row(0) of the • simplex method. Final tableau - dual

19. Strong Duality Property For the proof we’ll use as starting point the final row (0) as generated by the algebraic simplex procedure. We’ll also show how the final row (0) of the simplex algebraic procedure automatically provides the optimal values of the dual variables. In fact we denote by y*i the coefficients that correspond to the slack variables in the final row(0) of the simplex method.

20. Strong duality property Every feasible dual solution yields an upper-bound on Z* of the primal and every feasible primal solution yields a lower-bound on Z* of the dual Weak duality property Upper Bounds on Primal Optimum Dual feasible solutions (Minimization) Z* Primal Opt. = Dual Opt. Lower Bounds on Dual Optimum Primal feasible solutions (Maximization)

21. Strong Duality Property Proof

22. Notation for entries in row 0 of a simplex tableau (textbook) Dual surplus variables Dual decision variables Note: Complete basic solutions for the primal and dual problem can be read directly from row (0)

23. Minimize W= 4 y1 + 12 y2+ 12 y3 subject to y1 + 3 y3≥ 3 12 y2+ 12 y3 ≥ 5 and y1 ≥ 0, y2 ≥ 0, y3 ≥ 0 Introducing surplus vars Minimize W= 4 y1 + 12 y2+ 12 y3 subject to y1 + 3 y3– (z1 – c1) = 3 12 y2+ 12 y3 - (z2 – c2)= 5 and y1, y2 , y3 , (z1 – c1), (z2 – c2) ≥ 0

24. Primal Simplex: Weak Duality and strong duality Note: Complete basic solutions for the primal and dual problem can be read directly from row (0) The different (non-optimal) basic feasible solutions produced by the simplex algorithm in each iteration are feasible from the point of view of the primal but not from the point of view of the dual. Iteration 0: basic dual solution is infeasible since(z1 – c1) and (z2 – c2) are negative (weak duality) Iteration 1: basic dual solution is infeasible because (z1 – c1) is negative (weak duality) Final iteration –basic dual solution is feasible and optimal (Strong Duality).

25. Complementary slackness relationship for complementary basic solutions

27. Simplex method:Complementary solutions property • At each iteration the simplex method simultaneously identifies a CPF solution x for the primal problem and a complementary solution y for the dual problem, where: • cx = yb • If x is not optimal for the primal, than y is not feasible for the dual.

28. Simplex method:Complementary optimal solutions property • At final iteration the simplex method simultaneously identifies an optimal solution x* for the primal problem and a complementary optimal solution y* for the dual problem, where: • cx* = yb*

29. Complementary SlacknessOptimal solutions • Let x1*, x2*, …xn* be a primal feasible solution and a y1*, y2*, …ym* dual feasible solution. • Necessary and sufficient conditions for simultaneous optimality of x1*, x2*, …xn* and y1*, y2*, …ym* are: This result is very useful for checking the optimality of solutions

30. Complementary Slackness Optimal solutions

31. Relationships between the primal and dual problems • If one problem has feasible solutions and a bounded objective function (and therefore an optimal solution), then so does the other problem (both weak and strong duality apply) • If one problem has feasible solutions and an unbounded objective function (therefore no optimal solution) then the other problem has no feasible solutions • If one problem hasno feasible solutionsthan the other problem has no feasible solutions or an unbounded objective function.

32. Primal-Dual Combinations

33. Adapting to Other Forms • What if our problem is not in standard form? • We can always transform it to the standard form and then construct the dual;

35. Dual of the dual Note: it is not important which problem we call dual and which problem we call primal given the symmetry property of the primal dual relationships. In general we call primal the model formulated to fit the actual problem.

36. Shortcut for = constraints

37. Shortcut for dual transformations: = constraints or unconstrained variables We could still do the conversion into the standard format , but a shortcut is available for these two formats. • An equality constraint in the primal should be treated just • like a <= constraint in constructing the dual, except that the • non-negativity constraint for the corresponding dual variable • should be deleted • By the symmetry property, a primal variable not subject to • non-negativity constraints affects the dual problem only • by changing the corresponding inequality constraint to an • equality constraint.

38. Shortcut for dual transformations: >= constraints (Maximization) Consider: • Constructing the dual from this expression would lead to –aij as • coefficients of yi in the functional constraint j (of the form >=) • and a coefficient –bi in the objective function (which is to be • minimized); yi would be subject to a non-negativity constraint • (yi≥ 0); • Cleaner alternative for dual: • Consider y’i = - yi • Express the dual in terms of y’i • (- the coefficients of the variable become aij for • functional constraint j and bi for the objective function • - the constraint in the variable becomes y’i≤ 0;)

39. Sensible, Odd, Byzarre Variable constraints • Maximization/Minimization problems • “sensible” x >=0; • “odd” x unrestricted; • “byzarre” x <=0;

40. Sensible, Odd, Byzarre Constraints • Maximization problems • “sensible” <= • “odd” = • “byzarre” >= • Minimization problems • “sensible” >= • “odd” = • “byzarre” <=

42. Radiation Therapy Example

45. Primal-dual correspondence

46. Complexity of Simplex Method Primal vs. Dual • How long does it take to solve an LP using the simplex method? • Several factors but the most important one seems to be the number of functional constraints. • Computation tends to be proportional to the cube of the number of functional constraints in an LP. • The number of variables is a relatively minor factor (assuming revised simplex method) • The density of the matrix of technological coefficients is also a factor – the sparser the matrix (i.e., the larger the number of zeroes) the faster the simplex method; Real world problems tend to be sparse, i.e., “sparcity” of 5% or even 1%, which leads to fast runs.

47. Question: So if problem A has twice as many • constraints as problem B how much longer takes • to solve problem A in comparison to problem B? Complexity of Simplex Method Primal vs. Dual • But the most important one seems to be the number of functional constraints. • Computation tends to be proportional to the cube of the number of functional constraints in an LP. Problem A takes 8 times longer than problem B

48. Primal vs. Dual? So, the size of the problem, may determine whether to use the simplex method on the primal or dual problem. If the primal has a large number of constraints and a small number of variables it is better to apply the simplex method to the dual (since it will have a small number of constraints).

49. Dual Simplex Method • This method is based on the duality results. • It is a mirror image of the simplex method: • the simplex method deals with primal feasible solutions (but not dual feasible), moving toward a solution that is dual feasible; • the dual method deals with basic solutions in the primal problem that are dual feasible but not primal feasible. It moves toward an optimal solution by striving to achieve primal feasibility as well.