Optimization Problems

1. Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室

2. Content • Introduction • Definitions • Local and Global Optima • Convex Sets and Functions • Convex Programming Problems

3. Optimization Problems Introduction 大同大學資工所 智慧型多媒體研究室

4. General Nonlinear Programming Problems objective function constraints

5. Local Minima vs. Global Minima objective function constraints local minimum global minimum

6. Local optimality  Global optimality Convex Programming Problems objective function constraints convex f (x) gi(x) concave hj(x) linear

7. Local optimality  Global optimality Linear Programming Problems objective function constraints linear f (x) a special case of convex programming problems gi(x) linear hj(x) linear

8. Local optimality  Global optimality Linear Programming Problems objective function constraints linear f (x) gi(x) linear hj(x) linear

9. Integer Programming Problems objective function constraints linear f (x) gi(x) linear hj(x) linear

10. The Hierarchy of Optimization Problems Nonlinear Programs Convex Programs Linear Programs (Polynomial) Integer Programs (NP-Hard) Flow and Matching

11. Optimization Problems • General Nonlinear Programming Problems • Convex Programming Problems • Linear Programming Problems • Integer Linear Programming Problems

12. Optimization Techniques • General Nonlinear Programming Problems • Convex Programming Problems • Linear Programming Problems • Integer Linear Programming Problems Continuous Variables Continuous Optimization Combinatorial Optimization Discrete Variables

13. Optimization Problems Definitions 大同大學資工所 智慧型多媒體研究室

14. Optimization Problems

15. Optimization Problems Minimize cost c: FR1 Define the set of feasible points F

16. F: the domain of feasible points c: F R1 cost function A global optimum Definition:Instance of an Optimization Problem (F, c) Goal: To findf Fsuch that c( f) c(g) for allgF.

17. Definition:Optimization Problem • A set of instances of an optimization problem, e.g. • Traveling Salesman Problem (TSP) • Minimal Spanning Tree (MST) • Shortest Path (SP) • Linear Programming (LP)

18. Traveling Salesman Problem (TSP)

19. Traveling Salesman Problem (TSP) • Instance of the TSP • Given ncities and an nn distance matrix [dij], the problem is to find a Hamiltonian cycle with minimal total length.

20. Minimal Spanning Tree (MST)

21. Minimal Spanning Tree (MST) • Instance of the MST • Given an integern > 0and an nn symmetric distance matrix [dij], the problem is to find a spanning tree on n vertices that has minimum total length of its edge.

22. Linear Programming (LP) minimize Subject to

23. Linear Programming (LP) minimize Subject to

24. Linear Programming (LP) minimize Subject to

25. Example:Linear Programming (LP) minimize Subject to

26. x3 x2 x1 Example:Linear Programming (LP) minimize Subject to v3 c(v3) = 6 The optimal point is at one of the vertices. c(v2) = 4 c(v1) = 8 v2 v1 The optimum

27. x3 c1=4 c2=2 x2 c3=3 x1 Example:Minimal Spanning Tree (3 Nodes) minimize Integer Programming Subject to x1{0, 1} x3{0, 1} x2{0, 1}

28. x3 c1=4 c2=2 x2 c3=3 x1 Some integer programs can be transformed into linear programs. Example:Minimal Spanning Tree (3 Nodes) minimize Linear Programming Subject to x1{0, 1} x3{0, 1} x2{0, 1}

29. Optimization Problems Local and Global Optima 大同大學資工所 智慧型多媒體研究室

30. For combinatorial optimization, the choice of N is critical. Neighborhoods Given an optimization problem with instance (F, c), a neighborhood is a mapping defined for each instance.

31. TSP (2-Change) gN2(f ) f F

32. TSP (k-Change)

33. MST gN(f ) f F • Adding an edge to form a cycle. • Deleting any edge on the cycle.

34. minimize Subject to LP

35. (F, c) an instance of an optimization problem Given N neighborhood Local Optima f F is called locally optimum with respect to N (or simply locally optimum whenever N is understood by context) if c(f )  c(g)for allgN(f ).

36. c small 1 0 F Local Optima F = [0, 1]  R1 Global minimum C Local minimum Local minimum A B

37. Decent Algorithm f = initial feasible solution While Improve(f )   do f = any element in Improve(f ) return f Decent algorithm is usually stuck at a local minimum unless the neighborhood N is exact.

38. Exactness of Neighborhood Neighborhood N is said to be exact if it makes Local minimum  Global Minimum

39. F = [0, 1]  R1 c Global minimum C Local minimum Local minimum A B 1 0 F N is exact if   1. Exactness of Neighborhood

40. TSP N2:not exact Nn: exact

41. gN(f ) f F • Adding an edge to form a cycle. • Deleting any edge on the cycle. N is exact MST

42. Optimization Problems Convex Sets and Functions 大同大學資工所 智慧型多媒體研究室

43. A convex combination of x, y. A strictconvex combination of x, y if   0, 1. Convex Combination x, y Rn z = x +(1)y 0    1

44. z = x +(1)y Convex Sets 0    1 S Rn is convex if it contains all convex combinations of pairs x, y S. convex nonconvex

45. z = x +(1)y Convex Sets 0    1 S Rn is convex if it contains all convex combinations of pairs x, y S. n = 1 S is convex iff S is an interval.

46. Convex Sets Fact: The intersection of any number of convex sets is convex.

47. a convex set a convex function if c x y Every linear function is convex. Convex Functions S Rn c:S R c(x +(1)y)  c(x) + (1)c(y), 0    1 c(x) + (1)c(y) c(y) c(x) c(x +(1)y) x +(1)y

48. a convex set a convex function on S a real number Lemma S c(x) is convex. t Pf) Let x, y  St x +(1)y S c(x +(1)y) c(x) + (1)c(y)  t + (1)t = t x +(1)y St

49. Level Contours c = 5 c = 4 c = 3 c = 2 c = 1

50. a convex set a concave function if Every linear function is concave as well as convex. Concave Functions S Rn c:S R c is a convex