1 / 28

Linear and Nonlinear Optimization

Linear and Nonlinear Optimization. Ashish Goel Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A . http://www.stanford.edu/~ashishg http://www.stanford.edu/class/msande211 /. 1 st Day Questions.

tatiana-blackwell
Download Presentation

Linear and Nonlinear Optimization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Linear and Nonlinear Optimization Ashish Goel Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/~ashishg http://www.stanford.edu/class/msande211/ Lecture #1; Based on slides by Yinyu Ye

  2. 1st Day Questions • 6 Homeworks (best five), 1 Midterm (Oct 28, 1 Final) • Regular Students: 35% H + 25% M + 40% F • Extra credit: 5-10%% • Greater than 100% => A+ • No difference in taking 3 or 4 units • No formula for cutoff between A/B etc. • The more fun we all have, the more A’s we will give out. • Required Textbook: Optimization and Algorithms (PDF will be available) • Website: http://web.stanford.edu/class/msande211 • Programming ≠ Computer Science. The software use will help: Mostly an Excel and a “PAPER AND PENCIL” class! Extra credit Matlab assignments. • Friday’s problem sessions • CA team Lecture #1; Based on slides by Yinyu Ye

  3. Introduction to Optimization • Often considered the common goal of management science & engineering. • Maximize or Minimize f(x) for all x ϵ some set X • Applications in: • Applied Science, Engineering, Economics, Finance, Medicine, Statistics, Business • General Decision and Policy Making • The famous Eighteenth Century Swiss mathematician and physicist Leonhard Euler (1707-1783) proclaimed that “…nothing at all takes place in the Universe in which some rule of maximum or minimum does not appear.” Lecture #1; Based on slides by Yinyu Ye

  4. The Prototypical Optimization Problem Minimize: f(x) Subject to: h1(x) = 0 ... hm(x) = 0 g1(x) < 0 ... gr(x) < 0 The Function could be: x1+2x2, x2+2xy+2y2, xln(x)+ey, |x|+max{x,y}, etc Lecture #1; Based on slides by Yinyu Ye

  5. An Example: Maximum Flow How much flow can travel from A to B, given that each of the directed connecting routes have flow limits? 12 13 6 5 A B 5 11 4 10 Lecture #1; Based on slides by Yinyu Ye

  6. Maximum Flow by Inspection 11 12 2 4 13 6 15 15 5 1 6 11 4 3 5 10 9 Lecture #1; Based on slides by Yinyu Ye

  7. An Example: Maximum Flow Cut value from Source site to Sink site=17 12 13 6 5 A B 5 11 4 10 Lecture #1; Based on slides by Yinyu Ye

  8. An Example: Maximum Flow Cut value from Source site to Sink site=17 12 13 6 5 A B 5 11 4 10 Lecture #1; Based on slides by Yinyu Ye

  9. An Example: Maximum Flow Cut value from Source site to Sink site=15 12 13 6 5 A B 5 11 4 10 Lecture #1; Based on slides by Yinyu Ye

  10. outflow • inflow 12 2 4 • x12 13 6 • (k12) 5 1 6 5 • x13 • (k13) 11 4 3 5 10 • x61 • (k61) ∞ Lecture #1; Based on slides by Yinyu Ye

  11. 3 Main Categories in Optimization • Linear Optimization (Programming) • Search Algorithms • Interior-Point Algorithms • Unconstrained Optimization • Constrained Optimization Other Classifications: Quadratic, Convex, Integer, Mixed-Integer, Binary, etc. Lecture #1; Based on slides by Yinyu Ye

  12. Issues in Optimization • Problem Size • Small – by hand • Medium – by software • Large –by decomposition • Algorithm Complexity • Convergence • Local Convergence Time • Insight more than just the solution? • Solution structure properties • Sensitivity analysis • Alternate formulations Lecture #1; Based on slides by Yinyu Ye

  13. What do you learn? • Models – the art • How we choose to represent real problems • Theory – the science • Necessary and Sufficient Conditions that must be true for the optimality of different classes of problems. • Algorithms – the tools • How we apply the theory to robustly and efficiently solve problems and gain insight beyond the solution. Lecture #1; Based on slides by Yinyu Ye

  14. Section I: Linear Programming • Why do we study LP’s • Not because solving non-linear problems are too difficult • But because real-world problems are often formulated as linear equations • Either because they indeed are linear • Or because it is unclear how to represent them and linear is an intuitive compromise Lecture #1; Based on slides by Yinyu Ye

  15. LP, Nobel Prize,… Lecture #1; Based on slides by Yinyu Ye

  16. … and National Medal of Science Lecture #1; Based on slides by Yinyu Ye

  17. Art of Modeling & Vocabulary • Decision Variables x ϵ Rn, yet to be decide • Coefficients, c ϵ Rn, that are given and fixed • Objective inner product f =cTx: Rn ⇥R • Constraint Set S ⊂ Rn • Feasible solution x ϵ S • Optimal solution x* ϵ S • Optimal value z* = f (x*) Lecture #1; Based on slides by Yinyu Ye

  18. x2 • x1 • x1 • x1 LP Example 1: Production Management Lecture #1; Based on slides by Yinyu Ye

  19. Abstract Model • x11 • 1 • 1 • 2 • 2 • 3 • 3 • x34 • 4 LP Example 2: Transportation Lecture #1; Based on slides by Yinyu Ye

  20. LP Example 3: Support Vector Machine • ai • bj {y: yTx + x0 = 0} x is the normal direction or slope vector and x0 is the intersect Lecture #1; Based on slides by Yinyu Ye

  21. LP Example 3: Is Strict Separation Possible Are there x and x0 such that the following (open) inequalities are all satisfied Are there x and x0 such that the following inequalities are all satisfied for arbitrarily small ε. Divide x and x0 by ε., the problem can equivalently reformulated. This is a special LP, called linear feasibility problem. Lecture #1; Based on slides by Yinyu Ye

  22. LP Example 4: Electric Vehicle Charging Schedule Lecture #1; Based on slides by Yinyu Ye

  23. LP Example 4: When Discharge is Allowed Lecture #1; Based on slides by Yinyu Ye

  24. Linear Programming Abstraction Lecture #1; Based on slides by Yinyu Ye

  25. Abstract Linear Programming Model Lecture #1; Based on slides by Yinyu Ye

  26. Coefficient matrix • Obj. vector decision vector • RHS vector LP in Compact Matrix Form Lecture #1; Based on slides by Yinyu Ye

  27. Some Facts of Linear Programming • Adding a constant to the objective function does not change the optimality • Scaling the objective coefficients does not change the optimality • Scaling the right-hand-side coefficients does not change the optimality but the solution gets scaled accordingly • Reordering the decision variables (together with their corresponding objective and constraint coefficients) does not change the optimality • Reordering the constraints (together with their right-hand-side coefficients) does not change the optimality • Multiplying both sides of an equality constraint by a constant does not change the optimality • Pre-multiplying both sides of all equality constraints by a non-singular matrix does not change the optimality Lecture #1; Based on slides by Yinyu Ye

  28. Homework • Before next class read Chapters 1, 2, and 3 in Optimization and Algorithms Lecture #1; Based on slides by Yinyu Ye

More Related