1 / 13

Algorithms for Nonlinear Optimization - CSE543T

Study various nonlinear optimization algorithms, their applications, and challenges. Gain hands-on experience with optimization packages. Understand mathematical modeling and solver selection. Course covers unconstrained, convex constrained, and general constrained optimization.

carloswilliams
Download Presentation

Algorithms for Nonlinear Optimization - CSE543T

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSE543T: Algorithms for Nonlinear OptimizationYixin Chen, PhDProfessorDepartment of Computer Science & EngineeringWashington University in St Louis Spring, 2016

  2. Nonlinear Programming (NLP) problem Minimize f(x) (optional) Subject to h(x) =0 g(x) <= 0 Characteristics: • variable space X: continuous, discrete, mixed, curves • Decision variables, control variables, system inputs • function properties • Closed form, evaluation process, stochastic, …

  3. Why non-linear? • Deal or no deal? You can choose • Take $2M and go home; Or, • Deal: could get $0.1 or $5M • What will you choose?

  4. Why study nonlinear optimization? • Optimization is everywhere • Constraints are everywhere • The world is nonlinear (curvature and local optima) • Plenty of applications • Investment, networking design, machine learning (NN, HMM, CRF), data mining, sensors, structural design, bioinformatics, medical treatment planning, games • Nonlinear optimization is difficult • Curse of dimensionality, among many other reasons

  5. Example: Linear Regression

  6. Logistic Regression • Y=0 or 1 • Where nonlinear functions clearly make more sense • P(Y=1) = 1/(1+exp(-wTx)) • Find w to maximize ∏i=1..n P(Yi=1)Yi (1-P(Yi=1))(1-Yi)

  7. SVM • Minimize • || w|| • subject to: • yi(w xi - b) ≥ 1

  8. Sensor network optimization • Variables • Number of sensors • Locations of sensors • Functions • Minimize the maximum false alarming rate • Subject to minimum detection probability > 95% • Highly nonlinear functions • Based on a Gaussian model and voting procedure • Computed by a Monte-Carlo simulation

  9. Sodoku

  10. Goals of the course • Introduce the theory and algorithms for nonlinear optimization • One step further to the back of the blackbox • Able to hack solvers • But not too much gory details of mathematics • Hands-on experiences with optimization packages • Know how to choose solvers • Understanding of the characteristics of problems/solvers • Mathematical modeling • Modeling languages • CSE applications • Help solve problems in your own research

  11. Overview of the course • Unconstrained optimization (6 lectures) • Convex constrained optimization (3 lectures) • General constrained optimization (12-14 lectures) • Penalty methods • Lagrange methods • Dual methods

  12. Assignments and grading • Three or four homework (40%) • One project (30%) • In-class presentations • Reports • One final exam (30%)

More Related