1 / 46

15-853:Algorithms in the Real World

Explore Linear and Integer Programming (LP & IP) concepts, methods like Simplex and Interior Point, applications in industries, and the importance of optimization. Topics include geometric interpretation, formulations, algorithms, and real-world case studies.

hattiew
Download Presentation

15-853:Algorithms in the Real World

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 15-853:Algorithms in the Real World Linear and Integer Programming I Introduction Geometric Interpretation Simplex Method 15-853

  2. Linear and Integer Programming • Linear or Integer programming • minimize z = cTx cost or objective function • subject to Ax = b equalities • x ¸ 0 inequalities • c 2 Rn, b 2 Rm, A 2 Rn x m • Linear programming: • x 2 Rn (polynomial time) • Integer programming: • x 2 Zn (NP-complete) • Extremely general framework, especially IP 15-853

  3. Related Optimization Problems • Unconstrained optimization min{f(x) : x 2 Rn} • Constrained optimization min{f(x) : ci(x) · 0, i 2 I, cj(x) = 0, j 2 E} • Quadratic programming min{1/2xTQx + cTx : aiTx · bi, i 2 I, aiTx = bj, j 2 E} • Zero-One programming min{cTx : Ax = b, x 2 {0,1}n, c 2 Rn, b 2 Rm} • Mixed Integer Programming min{cTx : Ax = b, x ¸ 0, xi2 Zn, i 2 I, xr2 Rn, r 2 R} 15-853

  4. How important is optimization? • 50+ packages available • 1300+ papers just on interior-point methods • 100+ books in the library • 10+ courses at most Universities • 100s of companies • All major airlines, delivery companies, trucking companies, manufacturers, … make serious use of optimization. 15-853

  5. Linear+Integer Programming Outline • Linear Programming • General formulation and geometric interpretation • Simplex method • Ellipsoid method • Interior point methods • Integer Programming • Various reductions of NP hard problems • Linear programming approximations • Branch-and-bound + cutting-plane techniques • Case study from Delta Airlines 15-853

  6. Applications of Linear Programming • A substep in most integer and mixed-integer linear programming (MIP) methods • Selecting a mix: oil mixtures, portfolio selection • Distribution: how much of a commodity should be distributed to different locations. • Allocation: how much of a resource should be allocated to different tasks • Network Flows 15-853

  7. x1 x1’ x2 x1 x3 Linear Programming for Max-Flow 1 • Create two variables per edge: • Create one equality per vertex: x1 + x2 + x3’ = x1’ + x2’ + x3 • and two inequalities per edge: x1· 3, x1’ · 3 • add edge x0 from out to in • maximize x0 7 5 2 2 in out 7 3 3 6 15-853

  8. In Practice • In the “real world” most problems involve at least some integral constraints. • Many resources are integral • Can be used to model yes/no decisions (0-1 variables) • Therefore “1. A subset in integer or MIP programming” is the most common use in practice 15-853

  9. Algorithms for Linear Programming • Simplex (Dantzig 1947) • Ellipsoid (Kachian 1979) first algorithm known to be polynomial time • Interior Pointfirst practical polynomial-time algorithms • Projective method (Karmakar 1984) • Affine Method (Dikin 1967) • Log-Barrirer Methods (Frisch 1977, Fiacco 1968, Gill et.al. 1986) • Many of the interior point methods can be applied to nonlinear programs. Not known to be poly. time 15-853

  10. State of the art • 1 million variables • 10 million nonzeros • No clear winner between Simplex and Interior Point • Depends on the problem • Interior point methods are subsuming more and more cases • All major packages supply both • The truth: the sparse matrix routines, make or break both methods. • The best packages are highly sophisticated. 15-853

  11. Comparisons, 1994 15-853

  12. Formulations • There are many ways to formulate linear programs: • objective (or cost) functionmaximize cTx, orminimize cTx, orfind any feasible solution • (in)equalitiesAx · b, orAx ¸ b, orAx = b, or any combination • nonnegative variablesx ¸ 0, or not • Fortunately it is pretty easy to convert among forms 15-853

  13. Formulations • The two most common formulations: 1 2 slack variables minimize cTx subject to Ax ¸ b x ¸ 0 minimize cTx subject to Ax = b x ¸ 0 e.g. y1 7x1 + 5x2¸ 7 x1, x2¸ 0 7x1 + 5x2 - y1 = 7 x1, x2, y1¸ 0 More on slack variables later. 15-853

  14. Geometric View • A polytope in n-dimensional space Each inequality corresponds to a half-space. The “feasible set” is the intersection of the half-spaces. This corresponds to a polytope The optimal solution is at a corner. • Simplex moves around on the surface of the polytope • Interior-Point methods move within the polytope 15-853

  15. Geometric View x2 • An intersection of 5 halfspaces x2· 10 FeasibleSet Corners Objective Function -2x1 – 3x2 x1 x1 – 2x2· 4 2x1 + x2· 18 15-853

  16. Notes about higher dimensions • For n dimensions and no degeneracy • Each corner (extreme point) consists of: • n intersecting n-1 dimensional hyperplanese.g. 3, 2d planes in 3d • n intersecting edges Each edge corresponds to moving off of one hyperplane (still constrained by n-1 of them) • Simplex will move from corner to corner along the edges 15-853

  17. 1 z ri = z ¢ ei ei pi Optimality and Reduced Cost • The Optimal solution must include a corner. • The Reduced cost for a hyperplane at a corner is the cost of moving one unit away from the plane along its corresponding edge. For minimization, if all reduced cost are non-negative, then we are at an optimal solution. Finding the most negative reduced cost is a heuristic for choosing an edge to leave on 15-853

  18. x2 z = -2x1 – 3x2 x1 Reduced cost example In the example the reduced cost of leaving the plane x1 is (-2,-3) ¢ (2,1) = -7 since moving one unit off of x1 will move us (2,1) units along the edge. We take the dot product of this and the cost function. (2,1) (0,1) 15-853

  19. Simplex Algorithm • Find a corner of the feasible region • Repeat • For each of the n hyperplanes intersecting at the corner, calculate its reduced cost • If they are all non-negative, then done • Else, pick the most negative reduced costThis is called the entering plane • Move along corresponding edge (i.e. leave that hyperplane) until we reach the next corner (i.e. reach another hyperplane)The new plane is called the departing plane 15-853

  20. Step 2 x2 Departing z = -2x1 – 3x2 Step 1 x1 Entering Start Example 15-853

  21. Simplifying • Problem: • The Ax · b constraints not symmetric with the x ¸ 0 constraints.We would like more symmetry. • Idea: • Make all inequalities of the form x ¸ 0. • Use “slack variables” to do this. • Convert into form: minimize cTx subject to Ax = b x ¸ 0 15-853

  22. x5 x1 x4 x3 x2 Standard and Slack Form Standard Form Slack Form slack variables minimize cTx subject to Ax · b x ¸ 0 minimize cTx’ subject to A’x’ = b x’ ¸ 0 |A| = m x n i.e. m equations, n variables |A’| = m x (m+n) i.e. m equations, m+n variables x2· 10 2x1 + x2 + x4 = 18 2x1 + x2· 18 x1 x1 – 2x2· 4 x2 15-853

  23. Example, again x2 x5 x1 x4 x3 x2 x1 The equality constraints impose a 2d plane embedded in 5d space, looking at the plane gives the figure above 15-853

  24. Using Matrices • If before adding the slack variables A has size m x n then after it has size m x (n + m) • m can be larger or smaller than n n m 1 0 0 … 0 1 0 … 0 0 1 … … A = m slack vrs. Assuming rows are independent, the solution space of Ax = b is a n dimensional subspace. 15-853

  25. Gauss-Jordan Elimination Gauss-Jordan elimination 0 0 0 0 1 0 i j 15-853

  26. Simplex Algorithm, again • Find a corner of the feasible region • Repeat • For each of the n hyperplanes intersecting at the corner, calculate its reduced cost • If they are all non-negative, then done • Else, pick the most negative reduced costThis is called the entering plane • Move along corresponding line (i.e. leave that hyperplane) until we reach the next corner (i.e. reach another hyperplane)The new plane is called the departing plane 15-853

  27. Simplex Algorithm (Tableau Method) n • This form is called a Basic Solution • the n “free” variables are set to 0 • the m “basic” variables are set to b’ • A valid solution to Ax = b if reached using Gaussian Elimination • Represents n intersecting hyperplanes • If feasible (i.e. b’ ¸ 0), then the solution is called a Basic Feasible Solution and is a corner of the feasible set I F b’ m current cost 0 r -z reduced costs Basic Vars. Free Variables 15-853

  28. Corner basic variables free variables x5 x1 x4 x3 x2 15-853

  29. Corner x5 x1 x4 x3 x2 15-853

  30. Corner x5 x1 x4 x3 x2 15-853

  31. Corner x5 x1 x4 x3 x2 15-853

  32. Corner x5 x1 x4 x3 x2 Note that in general there are n+m choose m corners 15-853

  33. Simplex Method Again • Once you have found a basic feasible solution (a corner), we can move from corner to corner by swapping columns and eliminating. • ALGORITHM • Find a basic feasible solution • Repeat • If r (reduced cost ) ¸ 0 , DONE • Else, pick column with most negative r • Pick row with least positive b’/(selected column) • Swap columns • Use Gaussian elimination to restore form 15-853

  34. Tableau Method • If r are all non-negative then done n I F b’ current cost 0 r z Basic Variables Free Variables values are 0 reduced costs if all ¸ 0 then done 15-853

  35. Tableau Method • Else, pick the most negative reduced costThis is called the entering plane n I F b’ 0 r z min{ri} entering variable 15-853

  36. Tableau Method • Move along corresponding line (i.e. leave that hyperplane) until we reach the next corner (i.e. reach another hyperplane)The new plane is called the departing plane u I F b’ min positive bj’/uj 1 0 r z departing variable 15-853

  37. Tableau Method • Swap columns • Gauss-Jordan elimination x b’ No longer in proper form x x x x r z swap I Fi+1 bi+1’ Back to proper form 0 ri+1 zi+1 15-853

  38. x5 x1 x4 x3 x2 Example Find corner x1 = x2 = 0 (start) 15-853

  39. Example 18 10 x5 x1 x4 x3 x2 -2 min positive 15-853

  40. 18 10 x5 x1 x4 x3 x2 -2 Example swap Gauss-Jordan Elimination 15-853

  41. 18 10 x5 x1 x4 x3 x2 -2 Example 15-853

  42. Example 18 swap 10 x5 x1 x4 x3 x2 -2 Gauss-Jordan Elimination 15-853

  43. Simplex Concluding remarks • For dense matrices, takes O(n(n+m)) time per iteration • Can take an exponential number of iterations. • In practice, sparse methods are used for the iterations. 15-853

  44. Duality Primal (P): maximize z = cTx subject to Ax · b x ¸ 0 (n equations, m variables) Dual (D): minimize z = yTb subject to ATy ¸ c y ¸ 0 (m equations, n variables) Duality Theorem: if x is feasible for P and y is feasible for D, then cx · yb and at optimality cx = yb. 15-853

  45. Duality (cont.) Optimal solution for both feasible solutions for Dual (maximization) feasible solutions for Primal (minimization) Quite similar to duality of Maximum Flow andMinimum Cut. Useful in many situations. 15-853

  46. Duality Example • Solution to both is 38 (x1=4, x2=10), (y1=0, y2=1, y3=2). 15-853

More Related