AI, OR, and CS

1. Integration of Artificial Intelligence and Operations Research Techniques for Combinatorial ProblemsCarla P. GomesCornell University gomes@cs.cornell.eduKen McAloon and Carol TretkoffILOG{mcaloon,tretkoff}@ilog.com

2. AI, OR, and CS AI OR CS

3. FRAGILE OR Integration of Artificial Intelligence &Operations ResearchTechniques Representations Constraint Languages Logic Formalisms Object-Oriented Prog. Bayesian Nets Rule Based Systems •  •  • Tools Constraint Propagation Systematic Search Stochastic Search •  •  • Pros / Cons Rich Representations Computational Complexity Representations Mathematical Modeling Languages Linear & Non-linear (In)Equalities •  •  • Tools Linear Programming Mixed-Integer Prog. Non-linear Models •  •  • Pros / Cons More Tractable (LP) Primarily Complete Info Limited Representations Combinatorial Problems Planning Scheduling AI THE CHALLENGE AI OR UNIFY APPROACHES TO: SCALE UP SOLUTIONS HANDLE UNCERTAINTY ANALYZE COMPLEXITY (phase transition) EXPLOIT PROBLEM STRUCTURE INCREASE ROBUSTNESS

4. Outline • I. Short Overview of OR • II. Disjunctive Programming and Hybrid Solvers • III. Exploiting Randomization to Solve Hard Combinatorial Problems • IV. Conclusions

5. I. Short OR Overview

6. Outline for Linear Programming and Integer Programming • Standard Form of LP and a Simple Example • Geometric Interpretation of LP • Complexity issues • MIP • Example: Fast Food • Example: Capacitated Warehouse • Example: 911

7. Outline • 1. Short Overview of OR • 2. Constraint Programming • 3. Cooperating Solvers • 4. Disjunctive Programming • 5. Exploiting Randomization to Solve Hard Combinatorial Problems • 6. Conclusions

8. Optimization Technology Evolution 1947 1960 1970 1980 1990 1998 Shifting Bottleneck Primal Simplex LP Interior Point Dispatch Rules CPM PERT First CP Systems Dual Simplex Implementation Constraint Propagation SA, GA, Tabu Constraint-based Scheduling Dual Simplex Barrier LP Barrier Crossover Cooperating Solvers (LP/CP) Global constraints MIP Concurrent Scheduling Parallel LP/MIP Large IPs

9. 1. Short OR Overview

10. Outline for Linear Programming and Integer Programming • Standard Form of LP and a Simple Example • Geometric Interpretation of LP • Complexity issues • MIP • Example: Fast Food • Example: Capacitated Warehouse • Example: 911

11. An LP Story • A factory can produce n products from m parts • For product j it needs aij units of part i • There are bi units of part i available • Each unit of product j sold earns cj • Amount of each product to make is unknown xj  0 • Each part i determines a constraint • ai1x1 + … + ainxn bi • Obvious solution: do nothing • Better: maximize c1 x1 + … + cn xn

12. Standard Forms of LP • A linear program (LP) in standard form (Dantzig 1947) • max cTx • subject to Ax  b • x  0 • Input data: c (n x 1), A (m x n), b (m x 1). • Variables: x (n x 1)

13. Standard Forms of LP • // The objective function • max c1 x1 + … + cn xn • // The constraints • subject to • a11x1 + … + a1nxn b1 • ... • am1x1 + … + amnxn bm • x1  0 , … , xn  0

14. Standard Forms of LP • In OR emphasis is on optimality • Solution means optimal solution • Feasible solution means solution in the ordinary sense

15. Standard Forms of LP • Interpretation of standard form: • xj = amount of product j to make • cj = revenue per unit product j • bi = available amount of component i • aij = units of iused per unit of j produced • The constraints “say”: •  aijxj =  units of i used by j • = units of i used •  bi

16. Project Model Data What are models? • A model is a data-independent abstraction of a problem • A model lets you write down the mathematical representation of a model independently of the data One Problem Instance

17. Products Could be Jewelry Products: Rings and Earrings Components: Gold and Diamonds One ring requires 3 units of Gold, and 1 Diamond One set of earrings requires 2 units of Gold, and 2 Diamonds Total Gold and Diamonds are limited Profit is different for Rings than for Earrings Products = { rings, earrings }; Components = { Gold, Diamonds }; demand = [ [3, 1], [2, 2] ]; stock = [150, 180]; profit = [60, 40];

18. Products Could be Chemicals Products: Ammonium Gas = NH3 Ammonium Chloride = NH4Cl Components: Nitrogen, Hydrogen, Chlorine One unit of Gas requires 1 unit of Nitrogen, 3 units Hydrogen One unit of Chloride requires 1 unit of Nitrogen, 4 units Hydrogen, and 1 unit of Chlorine Total Nitrogen, Hydrogen, Chlorine is limited Profit is different for Gas than Chloride Products = { gas, chloride }; Components = { nitrogen, hydrogen, chlorine }; demand = [ [1, 3, 0], [1, 4, 1] ]; stock = [50, 180, 40]; profit = [30, 40];

19. Data Decision Variables Objective Function Constraints The Problems Have One Model enum Products ...; enum Components ...; float+ demand[Products, Components] = ...; float+ profit[Products] = ...; float+ stock[Components] = ...; varfloat+ production[Products]; maximize sum (p in Products) profit[p] * production[p] subjectto { forall (c in Components) sum (p in Products) demand[p, c] * production[p] <= stock[c] };

20. OR Modeling Systems • OPL • AMPL • 2LP • AIMMS • GAMS • MPL • ILOG Planner • etc

21. The Dual • The dual linear program (von Neumann 1947); • min yTb • subject to yTA  c • y  0 • Variablesy (m x 1) • Awesome Symmetry - • The dual of the dual is the primal

22. Rows and Columns Exchanged • min b1 y1 + … + bm yn • subject to • a11y1 + … + am1ym  c1 • ... • a1ny1 + … + amnym cn • y1  0 , … , ym  0

23. Duality Theorem • Theorem: min yTb = max cTx • Consequence: This turns optimality problem into a feasibility problem in x and y • Ax  b • x  0 • yTA  cT • y  0 • yTb = cTx • Consequence: Enumeration not needed to verify optimality

24. Duality Theorem • Sensitivity Analysis • Consequence: The solution values y* for the y variables yield the Lagrange multipliers of the primal constraints which measure the rate of change of the objective function with respect to the right hand side bounds b • yi * = Z / bi where Z is the optimum • Reference: McAloon and Tretkoff [1996] Wiley

25. Duality • Two different views of the same phenomenon • Point vs Set • Arc vs Node • Momentum vs Position • Vector vs Hyperplane • Landlord vs Renter

26. Simplex and Barrier • The simplex algorithm turns the feasibility problem into a iterative repair process with a powerful evaluation function • The barrier method transforms the LP into a system of differential equations that describe a vector field of flow on the polytope

27. Geometric Interpretation of LP Max: X subject to: -X + Y <= 4 X + 4*y <= 36 2*X + y <= 23 X + Y >= 4 Y >= X + 10 Y (4,8) (0,4) Simplex (10,3) Barrier (4,0) (8,0) X

28. Complexity of Linear Programming • Simplex Method • Worst-case --- exponential (Klee and Minty 72) • Practice --- good performance • Ellipsoid Method • Khachian’s Ellipsoid Method • Worst-case --- polynomial • Practice --- poor performance

29. Complexity of Linear Programming • Interior Point Methods or Barrier Methods • “Karmarkar’s” (and variants) Method • Worst-case --- polynomial • Practice --- good performance

30. Complexity of Linear Programming • Despite its worst case exponential time complexity, the simplex method is usually the method of choice since it provides tools for sensitivity analysis and its performance is very competitive in practice. • Which method performs best is problem dependent.

31. Success Stories • Industrial Planning • Given current resources, decide what to produce in what quantity • Supply Chain Management • Multiperiod planning models that link flow from one period to the next • Network Flow • How best to route goods across a network

32. Assumptions of Linear Programming • Linearity • when violated: ( xy = 50) • Nonlinear programming • Continuity • when violated: (x integral) • (Mixed) Integer programming

33. Assumptions of Linear Programming - continued • No Disjunctive Constraints • when violated: (x  100 or x  0) • Disjunctive programming • Additional 0-1 variables and Big M constraints • Certainty • when violated: (cost c is a random variable) • Stochastic programming

34. Search and MIP • In order to deal with variables that must have integer values in the solution, a search must be performed. • Mixed Integer Programming problems are combinatorial optimization problems and are NP hard • feasibility is NP-Complete • verifying optimality is co-NP-Complete

35. MIP and Combinatorial Optimization • These problems have been attacked by both the AI and OR communities. • In AI, these problems are attacked as CSPs or as Planning Problems. • In OR, they are done as MIPs and use linear relaxation to help guide the search. • The overriding idea in each case is to limit search.

36. Integer Program: All Integer Points in Region

37. Cut to Create Integer Vertex Integer Vertex

38. Example - Fast Food • Question: Is it possible for a male college student to eat at the local fast food outlet and still meet the requirements of a balanced diet? • If so, what is the least he can do it for?

39. Nutritional Requirements • At least 100% of vitamins A, C, B1, B2, niacin, calcium and iron • At least 55 grams of protein • At most 3000 milligrams of sodium • At most 30% of the calories can come from fat • Nutritional information is available from fast food outlets

40. College Student’s Requirements • At least 2000 calories a day • No more than 3 servings of any one food • Milk only with cereal and not as a stand-alone drink

41. Fast Food - MIP Model • We will have variables Servk to represent the number of servings of item k in the plan. • The variable Servk will have to take an integer value for the solution to be valid. • The objective function: Z for cost

42. Fast Food - MIP Model • Let foodk,j represent the percent of RDA of nutrient j in a serving of item k • The for each nutrient j, we have a constraint •  foodk,j Servk 100 • k

43. Fast Food - MIP Model • Let sodiumk represent the amount of salt in a serving of item k • For salt we have the constraint •  sodiumk Servk  3000 • k • Similarly for fat

44. Fast Food - MIP Model • Let costk represent the cost of a serving of item k • For the objective function we have the defining constraint •  costk Servk = Z • k

45. Fast Food - Solution • With a MIP solver and a way to input these constraints we ask for • a solution that makes the variables Servk integral • and which minimizes Z

46. MIP Solution Technique • What the MIP solver does is to carry out a branch and bound search guided by • the linear relaxation • the solution to the problem with the integrality requirements relaxed • Initialize the global variable best_so_far to 1000 (or something else very big).

47. At a Node • Compute a solution to the linear relaxation which minimizes Z yielding z*. Prune this node if • z*best_so_far , • If all values of Servk are integral, this is a solution. Setbest_so_far =z*. Save this node.

48. Branching at a node • Choose a variable Servk whose value s* is not integral. • Typical heuristic: most non-integral variable • Create two child nodes, • add Servkfloor(s*) • add Servkceil(s*)

49. Good News • The linear relaxation can prune nodes before all variables Servk are forced to be integral. • Surprisingly often a node “high in the tree” will turn up with all relevant variables integer. Here’s why • A solution to the LP is at a vertex • A vertex is defined as the simultaneous solution of the equality form of n linearly independent constraints • Many of these constraints are integer bounding constraints yielding X = integer

50. Arboreally Speaking • Breadth first search is often preferred - it visits the “smallest” number of nodes needed to find and verify the optimal solution - analogous to A* • If the linear relaxation is tight • | z*linear - z*integral | is relatively small • then z*linearis an excellent evaluation function