1 / 36

The New Faces of Nonlinear Programming

The New Faces of Nonlinear Programming. Jorge Nocedal Optimization Technology Center Argonne-Northwestern. New Problems, New Algorithms, New Software. Traditional Applications: solve larger problems, more robustness New classes of applications Advances in modeling languages: AMPL , …

darice
Download Presentation

The New Faces of Nonlinear Programming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The New Faces of Nonlinear Programming Jorge Nocedal Optimization Technology Center Argonne-Northwestern

  2. New Problems, New Algorithms, New Software • Traditional Applications: solve larger problems, more robustness • New classes of applications • Advances in modeling languages: AMPL, … • Automatic differentiation • Interior Methods • Test problems (CUTE, COPS) • New packages: LOQO, KNITRO,. • Internet Optimization: NEOS server McMaster

  3. Argonne Northwestern NEOS Server USER MINOS. SNOPT, FILTER, LANCELOT LOQO, KNITRO McMaster

  4. Semi-infinite Optimization Mixed Integer Nonlinearly Constrained Optimization Mixed Integer Linear Programming Nonlinearly Constrained Optimization Semidefinite & Second Order Cone Programming Linear Programming Unconstrained Optimization Linear Network Optimization Complementarity Problems Nondifferentiable Optimization Stochastic Linear Programming Global Optimization Application-specific Optimization McMaster

  5. Part I:New Classes of Problems Instead of new algorithms/software/ adapt existing techniques • Equilibrium constraints: (T. Luo) • Bi-level programming • Complementarity constraints • Semi-definite programming (??) • PDE-constrained optimization • Differential algebraic systems McMaster

  6. Nonlinear Optimization Formulation • Theme: constraints involve a difficult computation/simulation. • Limitations of this formulation? • Logic constraints McMaster

  7. Equilibrium Constraints • Structurally difficult • No strictly feasible direction • Algorithmically… y x McMaster

  8. Optimization problem with equilibrium constraints not stable  cannot apply NLP algorithms to it • Confirmed by experimental evidence (??) • Reality: software not capable of dealing with degeneracy, not sufficiently robust • Theoretical mistake: lack of stability does not imply practical problems. Structural degeneracy. • Active Set SQP (Leyffer et al) • Interior Methods: solve perturbed problem McMaster

  9. origin destination Traffic Assignment • For each origin-destination pair (o,d) we have: • qod: demand (in terms of flow) between o and d • K : index set of paths from o to d • fk : flow along path k, for each k in K • ck(f): cost of travel along path k (usually time), for each path k in K • λ = λ(qod) : minimum possible travel cost between o and d • Vector x of link flows, • Efficiency parameters (capacity, speed limit) given at link level • The path flows and costs are aggregated (based on x) through adjacency matrix A • Need constraints for demand satisfaction and conservation of flow • Many origin-destination pairs may exists in the network McMaster

  10. origin destination Network Design (Continuous Equilibrium) • Improvements to network: Traffic Network Design • Discrete: add lanes, links • Continuous: link capacity expansion • Boyce (1979) ed, • Continuous capacity? • Complex interaction between System Optimal and User Equilibrium is recognized -> bilevel programming (e.g. Abdulaal-LeBlanc) McMaster

  11. origin destination Example of Continuous Equilibirium Network Design • Given network: G=(N,A) • Find additions yi to capacities ci of links i in A • So that:Cost of improvement and efficiency of network is minimized An equlibrium flow McMaster

  12. KKT Conditions • Together with demand satisfaction and conservation of flow, • we need to demand EQUILIBRIUM, which in this case looks like: • λ = λ(qod) : minimum possible travel cost between o and d • If there is flow on path k (fk > 0): • path k is a minimum cost path (ck = λ) • If path k is relatively expensive (ck > λ): • no one uses this path (fk = 0) McMaster

  13. Partial Differential Equations and Optimization (Tsai, Byrd,N) Navier-Stokes equations Desired flow Mems flaps Determine position of mems flaps to optimize Quality of exhaust flow Phase II: boundary control McMaster

  14. Partial Differential Equations (PDEs) • Systems that evolve in space (several dimensions) and time are described by PDEs • Solution: function u(x,t) – infinite-dimen prob • More space dimen.: great computational and storage cost McMaster

  15. Success of PDE simulations 3DLarge Eddy Simulation around an airfoil McMaster

  16. Fluid flow described by Navier-Stokes • Solution of nonlinear PDEs • Newton-Krylov • Sequence of meshes, Krylov (FGMRES)-(full) • multigrid (Krylov smoother). • Parallel computing to obtain high resolution McMaster

  17. Now optimize! • Robustness of PDE solvers: millions of variables, hundreds of processors, multiple physical interactions • Introduce free parameters • Finite-dimensional formulation system of PDEs McMaster

  18. State-of-the-art algorithms • KKT system Newton-Lagrange • Active set: SNOPT, FilterSQPfactor subset of A, reduced Hessian • Interior: LOQO, KNITROfactor • Algorithms must accept iterative solution of constraint linearization. Av A’v McMaster

  19. Unconstrained reformulation • Linearize constraints • Eliminate state variables xs (basic) • Minimize w.r.t. controlsxd (non-bas) • New problem Modern optimization SQP: McMaster

  20. Weather Forecasting - Oceanography • = state of atmosphere, • Observations: Time windows i: length = a few time steps • Short integration: from initial condition Problem: unknown McMaster

  21. Nonlinear Least Squares Problem • Background field • Observations • Background covar • Obs covar • Time Constraints eliminated, no bounds, inequalities 3 Spaces: grid point, spectral, observation McMaster

  22. Part III:New Algorithms • New applications • New methods • New software • New tools (modeling languages, automatic differentaition) McMaster

  23. Part IIIAdvances in NLP Algorithms:Active Set SQP Before 1998: • Active Set SQP software: highly complex • Many dense, substandard versions • Quasi-Newton (SNOPT, MINOS) Present: • Filter, Second derivatives (FilterSQP) • SNOPT second derivatives in progress • Can SQP compete with Interior Methods? Future: • Linear Programming Based (Dundee, Northwestern.) McMaster

  24. Interior Methods Terlaky Newton’s method to KKT conditions of equal-problem: Reformulate to avoid rational functions: primal-dual Backtrack (difficulties!) Update barrier parameter Initial point strategy-failures McMaster

  25. Nonlinear Interior Methods Approach I: LOQO,OPINEL,BOEING,IPOPT,… • Modify W, • Merit Function/Filter Approach II: (KNITRO) enforcement Eliminate constraints-null space approach Approximate solution Iterative solution by conjugate gradients Merit function McMaster

  26. The Future (summer 2003) KNITRO 3.0 Active Interior Iterative Direct McMaster

  27. LP-EQP based on a Penalty Approach Equality constrained quadratic program LP Working set W L-1 penalty EQP McMaster

  28. Stepcomputation dc: min quadratic model of merit function: Cauchy point Dogleg approach EQP by projected CG dLP deqp d dc xk McMaster

  29. Rationale for Integration • Active set approach: LP + EQP (Fletcher) shares EQP solution with Interior Algor. • preconditioning • Final active set identification • Warm starts • Share interfaces, stop tests, testing, object c. • Both trust region methods • New algorithms… McMaster

  30. Remarks on:New Software—Interior Methods • Change of barrier parameter • Guiding principles (increase/decrease) • Scale invariance, initial point • Global convergence • When to attempt superlinear convergence • Seemingly superior to active set SQP codes • Only choice for very large (reduced space) • Versatile • 1st derivs only; do/not factor Hessian • Iterative vs direct solvers • Feasible/infeasible modes LOQO, KNITRO McMaster

  31. Tests Sets: CUTE (850 problems) COPS McMaster

  32. On many practical applications…

  33. Final Remarks • Extension of core NLP algorithms/sofware instead of special-purpose methods • Investigation of limitations: degeneracies, multilevels • Robustness (fundamental algorithmic) • More iterative options • How far can NLP methods scale up? McMaster

  34. L1 Merit Function L1 linear program and penalty parameter selection • Try to find minimum s.t. residuals = zero • W= constraints with zero residuals • Can achieve robustness and efficiency McMaster

  35. Interior Method- Direct • Solve primal-dual system • Retain trust region • Revert to Interior Iterative Step is too long Inertia is not correct Step is rejected by merit function Adaptive barrier parameter (Morales, Orban) dn Slack bound McMaster

  36. Interior Method (Iterative, CG) ProjectedCG: Infeasible slacks = barrier function Primal method, no multipliers McMaster

More Related