1 / 43

15-381 / 681 Instructors: Fei Fang (This Lecture) and Dave Touretzky feifang@cmu

Artificial Intelligence: Representation and Problem Solving Optimization (3): (Mixed) Integer Linear Programming. 15-381 / 681 Instructors: Fei Fang (This Lecture) and Dave Touretzky feifang@cmu.edu Wean Hall 4126. Recap. Convex optimization is a convex function and is a convex set

cicily
Download Presentation

15-381 / 681 Instructors: Fei Fang (This Lecture) and Dave Touretzky feifang@cmu

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Artificial Intelligence: Representation and Problem SolvingOptimization (3): (Mixed) Integer Linear Programming 15-381 / 681 Instructors: Fei Fang (This Lecture) and Dave Touretzky feifang@cmu.edu Wean Hall 4126

  2. Recap • Convex optimization • is a convex function and is a convex set • Gradient descent leads to global optimum • Nonconvex but continuous optimization • Gradient descent leads to local optima • Linear program • is a linear function and is a convex polytope • This section: is a linear function and is defined by a set of linear constraints + integer constraints s.t. Fei Fang

  3. Outline • (Mixed) Integer Linear Program • How to solve a MILP • Formulate/Convert to MILP Fei Fang

  4. (Mixed) Integer Linear Program: Definition • (Mixed) Integer Linear Program: • A special case of non-convexoptimization problem • An optimization problem whose optimization objective is a linear function and feasible region is defined by a set of linear constraints + integer constraints • Integer Linear Program (ILP): s.t. Can be minimization Can be or Fei Fang

  5. (Mixed) Integer Linear Program: Example • Example: Maximize Profit in Manufacturing s.t. Earphone Charger Fei Fang

  6. Binary Linear Program: Definition • Binary Linear Program (BIP): • All variables are restricted to take value or s.t. Fei Fang

  7. Binary Linear Program: Example • 0-1 Knapsack • Maximum weight • How to select items to maximize total value? Fei Fang

  8. Binary Linear Program: Example • 0-1 Knapsack • indivisible items. Item has weight , value . • Maximum weight is • How to pick the items to maximize total value? Fei Fang

  9. Outline • (Mixed) Integer Linear Program • How to solve a MILP • Formulate/Convert to MILP Fei Fang

  10. LP Relaxation • LP relaxation of an MILP or BLP is the LP with the same linear constraints LP Relaxation MILP s.t. s.t. BLP LP Relaxation s.t. s.t. Fei Fang

  11. Quiz 1: LP Relaxation • Let be the optimal solution and the optimal value of a MILP. Let be the optimal solution and the optimal value of the LP relaxation. Which of the following are true? • A: • B: if it is a maximization problem • C: if it is a minimization problem • D: if it is a maximization problem • E: if it is a minimization problem LP Relaxation MILP s.t. s.t. Fei Fang

  12. Quiz 2: (Mixed) Integer Linear Program • Which are optimal solutions? • None of above s.t. Fei Fang

  13. How to solve • Is it sufficient to consider integer points around the corresponding LP solution? No! • Solution to the MILP can be arbitrarily far from the solution of the corresponding LP (relaxing all the integer constraints) • Sometimes even hard to find a feasible integer point Fei Fang

  14. How to solve • For Binary Linear Program (BLP) • Discrete optimization problem • Naïve approach: Enumerate all the possibilities • Can apply uninformed or informed search • Natural tree structure s.t. Fei Fang

  15. Depth-First Search for BLP s.t. Fei Fang

  16. Recall: Informed Search function BEST-FIRST-SEARCH (problem, EVAL-FN) returns a solution sequence inputs: problem, a problem EVAL-FN, an evaluation function Queuing-Fn←a function that orders nodes by EVAL-FN return GENERAL-SEARCH (problem, Queuing-Fn) • search: Best-first-search with EVAL-FN where is an admissible heuristic for node • path cost up to node , =heuristic estimate of distance to go • In a minimization problem, never overestimates the distance to go • In a maximization problem, never underestimates the value of the remaining path • Optimal for tree search function GREEDY-SEARCH (problem) returns a solution or failure return BEST-FIRST-SEARCH (problem, h) Fei Fang

  17. How to solve • For Binary Linear Program (BLP) • Discrete optimization problem • Naïve approach: Enumerate all the possibilities • Can apply uninformed or informed search • Natural tree structure • Vanilla branch and bound (BnB)=Best-first-search with EVAL-FN optimal value of LP relaxation at node Fei Fang

  18. Evaluation Function in Branch and Bound LP relaxation at node • optimal value of LP relaxation at node • LP relaxation provides a upper bound of the objective value of any solution with a path through if it is a maximization problem (lower bound if minimization) • Never underestimates the real value of the whole path given the path goes through for maximization problem (never overestimate the total cost for minimization) • (path value up to ) is an admissible heuristic s.t. Fei Fang

  19. How to solve • For Binary Linear Program (BLP) • Discrete optimization problem • Naïve approach: Enumerate all the possibilities • Can apply uninformed or informed search • Natural tree structure • Vanilla branch and bound (BnB)=Best-first-search with EVAL-FN optimal value of LP relaxation at node • Optimality guaranteed Fei Fang

  20. How to solve • For Binary Linear Program (BLP) • Discrete optimization problem • Naïve approach: Enumerate all the possibilities • Can apply uninformed or informed search • Natural tree structure • Vanilla branch and bound (BnB)=Best-first-search with EVAL-FN optimal value of LP relaxation at node • Optimality guaranteed • BnB with upper bound and lower bound (not required) • Terminate early if we know the remaining nodes are not as good as a binary solution we have already found Fei Fang

  21. How to solve • For MILP (not required) • BnB: For each integer variable, branching a node by considering and where is a non-integer value • Branch and cut: use cutting planes (which are hyperplanes) to separate current non-integer solution and integer solutions Fei Fang

  22. How to solve • Practically efficient solvers: Cplex, Gurobi, intlinprog (MATLAB), SCIP solver Fei Fang

  23. Outline • (Mixed) Integer Linear Program • How to solve a MILP • Formulate/Convert to MILP Fei Fang

  24. Constraint satisfaction problem • TWO+TWO=FOUR Fei Fang

  25. Constraint satisfaction problem • TWO+TWO=FOUR Variables: , , ,, , , , , s.t. Fei Fang

  26. Formulate a Problem as a MILP • Sudoku • Fill a 9×9 grid with digits so that each column, each row, and each of the nine 3×3 subgrids that compose the grid (also called "boxes", "blocks", or "regions") contains all of the digits from 1 to 9 https://en.wikipedia.org/wiki/Sudoku Fei Fang

  27. Formulate a Problem as a MILP • Sudoku • Variables indicate if we place number in row , col • is the set of (row,col) index pairs of the groups (rows, columns, and subgrids) • For example, s.t. Fei Fang

  28. Formulate a Problem as a MILP • Kidney Exchange Fei Fang

  29. Formulate a Problem as a MILP • Kidney Exchange • Given directed graph , where each node represent a patient-donor pair, and an edge means donor of node can give one kidney to patient of node • Find a set of disjoint cycles so as to maximize the number of nodes covered 6 2 1 4 5 3 7 8 Fei Fang

  30. Quiz 3: Kidney Exchange • Given the graph below, what is the maximum number of patients that can get a kidney through kidney exchange assuming the length of each cycle should be less than or equal to 3? • A: 3 • B: 6 • C: 7 • D: 8 6 2 1 4 5 3 7 8 Fei Fang

  31. Formulate a Problem as a MILP • Kidney Exchange • Given directed graph , where each node represent a patient-donor pair, and an edge means donor of node can give one kidney to patient of node • Find a set of disjoint cycles so as to maximize the number of nodes covered Hint: enumerate all the cycles 6 2 1 4 5 3 7 8 Fei Fang

  32. 8-Queens Problem • No queen in the same row, column and diagnal • : whether there is a queen on row col s.t. Fei Fang

  33. 8-Queens Problem • Recall in the first lecture about optimization, we had • Can we rewrite it using only linear constraints + integer constraints? Variables : row index of the queen in column s.t. Fei Fang

  34. 8-Queens Problem s.t. Convert constraints (We omit the range of index below) s.t. or or Fei Fang

  35. Formulate a Problem as a MILP • 8-Queens Problem • Introduce variables , ,set s.t. or s.t. or or Fei Fang

  36. General techniques for conversion • At-most-one constraint: At most one of the options can be chosen • Add variable to represent whether or not option is chosen • Add constraint Fei Fang

  37. General techniques for conversion • Constraint feasibility constraint: If , then • Choose a large enough constant such that is always satisfied (imagine ) • Add constraint Fei Fang

  38. General techniques for conversion • Alternative Constraints: or • Introduce some large enough constants , such that always hold • Introduce binary variables , • Add constraints Fei Fang

  39. General techniques for conversion • Conditional Constraints • If , then • Equivalent to or • Apply techniques for alternative constraints Fei Fang

  40. General techniques for conversion • Bilinear term where and • Introduce new variable • Add constraints Fei Fang

  41. Summary Optimization Problems Convex Programs Mixed Integer Linear Programs Gradient Descent Branch and Bound Linear Programs Simplex Fei Fang

  42. (Mixed) Integer Linear Program: Additional Resources • Textbook • Applied Mathematical Programming, Chapter 9 • By Bradley, Hax, and Magnanti (Addison-Wesley, 1977)  • http://web.mit.edu/15.053/www/AMP.htm • Online course • https://ocw.mit.edu/courses/sloan-school-of-management/15-083j-integer-programming-and-combinatorial-optimization-fall-2009/index.htm Fei Fang

  43. Acknowledgment • Some slides are borrowed from previous slides made by J. Zico Kolter and Ariel Procaccia Fei Fang

More Related