Nature’s Algorithms

1. Nature’s Algorithms David C. Uhrig Tiffany Sharrard CS 477R – Fall 2007 Dr. George Bebis

2. What are “Nature’s Algorithms”? • Combinatorial Optimization Problems • No easy way to examine all combinations • Use algorithms based on scientific principles, nature, and genetics • This does not give optimal solutions, but is more practical than an exhaustive search

3. Example: Traveling Salesman • The Traveling Salesman problem is one of the most famous combinatorial optimization problems: • Given a set of cities (nodes), find the shortest path required to visit all cities exactly once.

4. Example: Traveling Salesman • There are (n-1)!/2 possible tours for n cities. • 2520 tours for 8 cities • 653,837,184,000 tours for 16 cities • Would require a computer nearly 8 days to compute them all! (for 1/1,000,000 sec per tour generation) • Not practical to compute all tours…

5. Example: Location Allocation • Given a set of facilities, each of which can serve a certain number of nodes on a graph, place the facilities in such a way that the distance between each node and the facility is minimized. • Police stations and population densities • Schools and student populations

6. Example: Location Allocation

7. Issues with Combinatorial Optimization Problems • Large set of discrete solutions • Number of solutions grows exponentially • In the Travelling Salesman problem, the solution size grows by n*(n-1)! for each city that is added. • In the Location Allocation problem, the solution size is affected by both the number of nodes but also the connections between nodes and the number of (and size of) facilities.

8. Evolution and the Genetic Algorithm • Mimics the process of evolution in developing a solution. • Begins with a large set of solutions to the problem in a candidate pool. • Each of these algorithms is tested for ‘fitness’

9. Evolution and the Genetic Algorithm • After the fitnesses are calculated, a new set of solutions is randomly picked from the ‘most fit’ algorithms. • Mimics Natural Selection • Next, two algorithms are paired together and swap pieces of each solutions • Then each algorithm changes (mutates) slightly

10. Evolution and the Genetic Algorithm • This pattern of evaluation, reproduction, swapping, and mutating mimics the way DNA is processed during sexual reproduction. • This process continues until a satisfactory solution is much better than the rest of the population of solutions.

11. Metallurgy and the Simulated Annealing Problem • In metallurgy, when molten metal cools too quickly it can lose certain properties, such as strength or magnetism. • Annealing is the process of slowly cooling, which can help retain or amplify desired traits. • In Computer Science, Simulated Annealing Algorithms are usually an improvement on “Greedy Local Search” algorithms.

12. Metallurgy and the Simulated Annealing Problem • Start with a proposed solution. • Change the solution slightly • i.e. Swap two cities in the traveling salesman problem. • Check to see if the new solution is better than the original. If not, undo the change. • Repeat steps 2 and 3 until it is apparent that any small change makes the solution worse.

13. Metallurgy and the Simulated Annealing Problem • A greedy local searchwill only find LocallyOptimal Solutions. • This also occurs with Genetic Algorithms when mutations stop producing positive changes.

14. Metallurgy and the Simulated Annealing Problem • Try to improve on Greedy Local Searches by taking occasional risks by accepting a worse solution. • Compute ΔC, the difference between the cost of the current solution and the proposed solution. Then:for Random(r), 0 < r < 1, if r < e -ΔC/TWhere T is the metaphoric “temperature” that is iteratively decreased until an acceptable temperature is reached.

15. Metallurgy and the Simulated Annealing Problem • Theoretically, the temperature should be lowered proportionally to 1/log(n), but this rate is too slow. • Realistically, the temperature is lowered at a linear or exponential rate. • Even with this decreased cooling time, simulated annealing has proven to be very effective in finding a balance between exploring every solution and avoiding locally optimal results.

16. Economics • The idea of “commodities” and “consumers” competing for them • Prices are determined by competition • Consumers buy what they can afford • Prices are influenced by consumers

17. Computer Networks • Consumer = Process • Competing commodities: processor time, access to network printer, etc

18. Example • Two terminals want to share two database files • The files will run slower if they are accessed from the other location than from the local location • Decision should be made based on how much of each file should be stored at each terminal

19. Solution • Give each file a starting price • Give each terminal a fixed amount of money • With constraints of the amount of money, terminals ask for a certain portion of each file • Price changes based on the proportion of how much of the files were requested • Done until equilibrium is reached

20. Simulated N-Body Algorithm • Study of moving particles in physics • Particles are allowed to move around the graph • To minimize potential energy in the system, particles tend to spread out • A “boundary” is formed to keep the particles from being pushed to the outer edge

21. Aggregation and Decomposition • Every particle is grouped into a larger particle with combined mass • Smallest particle is removed • Small and the largest particle moved to where the potential energy is the lowest • Take next smallest particle and do the same • Result: reasonably good configuration

22. Particle Physics • Simulation used to solve traveling salesman problems • Use of charged particles connected by electric bands

23. How Does It Work? • Number of Cities = Number of negatively- charged particles = Number of positively-charged particles • Positive particles are placed in a circle • Solution: • Particles are set free • When particle hits a city-stops • When all have stopped- elastic string traces a path through each city