Lecture: 5 Optimization Methods & Heuristic Strategies

1. Lecture: 5 Optimization Methods & Heuristic Strategies Ajmal Muhammad, Robert Forchheimer Information Coding Group ISY Department

2. Outline • Introduction to Optimization & Intractable Problems • Linear Programming • Heuristic • Metaheuristic • Linear Programming Formulation Shortest path Problem Max Flow Problem Wavelength Assignment Problem

3. Optimization Problem Problem that aims to find the best solution from all feasible solutions The best solution can be the minimum or maximum solution Examples: • Find the route from point A to point B that takes the shortest times • Determine how a production factory can maximize its profit using limited materials

4. The Travelling Salesman Problem (TSP) A classic combinatorial optimization problem Given n cities on a map, find the shortest route that visits all the cities once, and starts and ends at the same city Starting and ending in Paris, which route allow us to visit major cities with the least amount of travelling ?

5. The 1-Dimensional Bin Packing Problem Given n items of different (1D) sizes, and given some fixed- capacity bins, pack the items into a minimum number of bins Is this the minimal number of Bins needed for these items?

6. TSP Growth Rates • A route around a map can be represented as a permutation of the n cities: E.g. for 5 cities, [B,A,C,E,D] means “start at city B, then go to city A, then city C, then E, then D, and return to city B” • Given n cities, there is a total of n! permutations • Some permutations represent the same routes - there are actually ½(n-1)! different routes in total • To return the provably optimal solution to any instance of the TSP the algorithm needs to check the majority of – if not all – possible routes • The number of routes grows exponentially quickly making the problem intractable:

7. Linear Programming (LP) An optimization problem in which the objective function and all the constraints are expressed as linear functions even if just one of them is not a linear function, the problem is not an LP problem Linear programming problem Nonlinear programming problem

8. LP Formulation Matrix expression for LP formulation

9. Nomenclature in LP Problem Boundary is a constraint that express the upper or lower bound of an inequality or equality Feasible region is an area delineated by the boundaries Corner point is an intersection of the boundaries

10. Example Let z= x+ymax zRewrite as y= -x + z

11. Solution by Moving y = -x +z

12. Simplex Method

13. Heuristics • Provide a shortcut to solve difficult problems. • Used under limitedtime and/or information to make a decision. • Problem-dependent techniques, i.e., adapted to the problem at hand to take advantage of the particularities of the problem. • Lead to a good decision most of the time. Heuristics are good when • Making a spur-of-the-moment decision • Having limited information and cannot obtain more • The decision is not that important

14. Heuristics Heuristics are bad when • Having plenty of time and information to make an important decision • Need to be right 100% of the time Examples of heuristics • Deciding to eat at restaurant B rather than restaurant A only because B has more cars in its parking lot • When the level in the fuel tank drops to 1/2 tank or less, always filling up the fuel tank at the very next re-filling station

15. Metaheuristic Meta: in an upper level Heuristic: to find A metaheuristic is formally defined as an iterative generation process which guides a subordinate heuristic by combining intelligently different concepts for exploring and exploiting the search space, learning strategies are used to structure information in order to find efficiently near-optimal solutions. A metaheuristic is a function that consists of heuristic subroutines. It is a compound heuristic.

16. Fundamental Properties of Metaheuristics • Metaheuristics are strategies that guide the search process to avoid being trapped in confined areas of the search space • The goal is to efficiently explore the search space in order to find (near)-optimal solutions • Techniques which constitute metaheuristic algorithms range from simple local search procedures to complex learning processes • Metaheuristic algorithms are approximate and usually non-deterministic • Metaheuristics are not problem-specific

17. Metaheuristic Examples • Genetic algorithms (1960s/1970s), Evolutionary strategy (Rechenberg & Swefel 1960s), Evolutionary programming (Fogel et al. 1960s). • Simulated annealing (Kirkpatrick et al. 1983), Tabu search (Glover 1980s), Ant colony optimization (Dorigo 1992), Genetic programming (Koza 1992), Particle swarm optimization (Kennedy & Eberhart 1995), Differential evolution (Storn & Price 1996/1997), • Harmony search (Geem et al. 2001), Honeybee algorithm (Nakrani & Tovey 2004), ..., Firefly algorithm (Yang 2008), Cuckoo search (Yang & Deb 2009), ...

18. Genetic Algorithms: TSP Example

19. Genetic Algorithms: TSP Example

20. Genetic Algorithms: TSP Example

21. Genetic Algorithms: TSP Example

22. Genetic Algorithms: TSP Example

23. Ant Colony Optimization (in one Slide) A. Ants in a pheromone trail between nest and food; B. an obstacle interrupts the trail; C. ants find two paths to go around the obstacle; D. a new pheromone trail is formed along the shorter path.

24. LP Formulation for Shortest Path Problem Network is represented by directed graph G(V,E), where V is the set of vertices (nodes) & E is the set of links Link from node i to node j is expressed by (i,j); dij is the link cost of (i,j) Xpqij: traffic volume from node pto node qrouted through (i,j)

25. Max flow Problem Network model with link capacities Max flow routing from node 1 to node 6 ILP formulation for max flow problem ? Assignment

26. Wavelength Assignment (WA) Problem Optical path requests Graph construction for coloring

27. LP Formulation for Graph Coloring Let W be a set ofλ, where W = {λ1,λ2,··· ,λ|W|} Let xλv and yλ, be binary variables. Ifλ is assigned to a path corresponding v, xλv = 1, otherwise xλv = 0. Ifλ is used at least one time yλ=1, otherwise yλ=0.

28. Heuristic: Largest Degree First (LDF) Step 1: select the uncolored vertex with largest degree. Step 2: choose the min indexed color from the colors that are not used by the adjacent vertices. Step 3: color the selected vertex using the color chosen in step 2. Step 4: if all vertices are colored, LDF stops. otherwise, LDF returns to step 1. Wavelength assignment process of LDF