1 / 47

New Optimization Methods in Geometric Computing

New Optimization Methods in Geometric Computing. By Paul Zimmons CS 258 Fall 2000. Preface. Optimization techniques are optimization techniques whether applies to geometry or otherwise The topics I will cover have applications in both geometric and non-geometric computation.

gin
Download Presentation

New Optimization Methods in Geometric Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New Optimization Methods in Geometric Computing By Paul Zimmons CS 258 Fall 2000

  2. Preface • Optimization techniques are optimization techniques whether applies to geometry or otherwise • The topics I will cover have applications in both geometric and non-geometric computation

  3. Basis for this Talk • The basis for this talk was the book New Ideas in Optimization by David Corne, Marco Dorigo, and Fred Glover (UK book)

  4. Overview • Current Optimization • Ant Colony Methods • Differential Evolution • Immune System Methods • Swarm Method

  5. What is Optimization? • “doing better”– NIIO • Different possible solutions and some measure of the quality of those solutions • Real problems are ‘hard’ • Can’t guarantee to find the best solution in acceptable amounts of time

  6. Types of Optimization • Problem specific methods and very general • The methods discussed here are based on the idea of stochastic iterative search • General method

  7. General Method • 1. Generate and evaluate an initial collection of candidate solutions S • 2. Produce and evaluate a collection of new candidate solution S’ by making random changes to selected members of S • 3. Replace some members of S with some of the members of S’ and then back to 2

  8. Why Do This? • Simple to Implement • Very General • No function assumptions • Very Successful • Relatively little effort • Easily tailored to use problem specific knowledge

  9. Ant Colony Optimization • Inspired by ant colonies • Equates the notion of a candidate solution with the route taken by an ant between two places • Ants leave a trail in which better solutions leave a stronger trail • The trail affects future ants

  10. Ant Algorithm • ‘Multi-agent system’ in which each member acts like an ant or a super-ant • Good for routing, traveling salesman problem, etc. • Path Planning

  11. History • Goss et al (1989) studied ants in a laboratory • Ant colony was given access to food linked to the nest by two bridges of different lengths • Ants must choose one way or the other • Ants converged on the smaller path

  12. Experimental Setup Nest Food

  13. More History • Autocatalytic effect • Indirect communication and differential path length • Ants deposit pheromones on the ground • When they reach a decision, they are biased by the amount of pheromone

  14. Simple Ant Colony Opt. • G = (N,A) is a graph with n = |N| nodes • Source node f and destination node d and path length is given by # of hops • Each arc (i,j) has a variable ij which is called the artificial pheromone trail • Pheromone trails are read and written by ants • Intensity is proportional to utility

  15. More S-ACO • At each node, information on the node or the outgoing arcs are used stochastically to decide the next move • For ant k at node i uses pheromone trails ij to compute the probability of moving to node jNi as the next move

  16. More S-ACO • Ants deposit pheromone on the arcs they use • ij(t)  ij(t) +   • Autocatalysis and differential path length will again favor the smaller path • BUT this can converge inappropriately to a suboptimal path

  17. Better Pheromone Model • Similar to real pheromones, artificial pheromone trails can evaporate • Decrease the pheromone trails exponentially •   (1-) , (0,1] • Multiply for each iteration of the algorithm • Pheromone trails encode a long-term memory about the whole search process

  18. S-ACO Problems • Increasing the complexity by introducing multiple shortest paths makes the algorithm less stable and effective • Parameter choices are more important • More sophisticated approaches are necessary • Ex. Increase the pheromone by the ‘goodness’ of the solution or give ants memories • Good for discrete optimization

  19. S-ACO Notes • Good quality solutions can usually only emerge from collective interaction among the ants • Each ant makes use only of private information and local information about the node it is visiting • Ants do not adapt, they adaptively modify the way the problem is represented and perceived by other ants

  20. When to use Ants • NP-hard problems where the search space is exponential in the dimension of the problem representation • Cases where the properties of the graph change over time concurrently with the optimization process • Problems where the computational architecture is spatially distributed (networks) because ants are inherently distributed, asynchronous, and multi-agent

  21. Performance • Better than general heuristics like evolutionary computation or simulated annealing • Adding a local search makes the algorithm competitive with more sophisticated approaches • Does not beat the best TSP out there (Iterated Lin-Kernighan, 1997) • Better than most for network problems (cause of the changes)

  22. Token Geometry App • Path planning with moving objects (??) • Sophisticated game engine where CPU/Graphics/Memory resources must be dynamically allocated • Camera path and choosing best reference images • Nodes are images and edges are possible choices

  23. Differential Evolution • “Differential Evolution (DE) is a simple yet powerful population-based, direct-search algorithm for globally optimizing functions defined on totally ordered spaces, including especially functions with real-valued parameters.”

  24. When to Use DE • Problem specific methods will usually do better when compared to general methods • But no deterministic algorithm does well with functional pathologies

  25. What can go wrong? • Non-linearity • High dimensionality • Multi-modality (multiple local optima) • Parameter interaction • Constraints • Flatness • Non-Differentiability

  26. Generic Evol. Alg. • Initialize population • Evaluate fitness • Do(until we win or are tired) • Mutate the population • Recombine the mutants to keep the population size down • Evaluate fitness • Select successors

  27. DE Overview • Overview of algorithm • Encoding • Population and parameters • Mutation • Selection • Recombination

  28. DE Algorithm • R1, r2, r3  {1,2,…,NP} • (randomly selection : r1r2r3I) • Jrand = int(randi[0,1)*D)+1 • For(j=1, j  D; j=j+1) • If(randi[0,1] < CR  j=jrand) • Uj,I,G+1=vj,I,G+1=xj,r3,G+F*(xj,r1,G-xj,r2,G) • Else uj,i,G+1 = xj,i,G

  29. Pictorially

  30. Evaluating a Vector Vi,G+1=Xr3,G+F*(Xr1,G- Xr2,G) = potential child F*(Xr1,G- Xr2,G) Ui,G+1 Ui,G+1 Xr3,G Ui,G+1 Xi,G Vi,G+1 Xr1,G- Xr2,G Xr1,G Xr2,G

  31. Ex. Ellipse Again

  32. Global Mutations • Say two ellipsoids • Usually a search will converge on one of them and ignore the other • In DE since all the object vectors are mutated by the same distribution, if object vectors occupy both ellipsoids then the distribution will contain a mixture • Inclusion of large scale mutations from vectors form both ellipses give some form of globally correlated mutation

  33. Pictorially

  34. DE Results • Telescope design with DE and ray tracer • DE chooses curvature, number of lenses, type of lenses • Mechanical problems • Discrete and continuous solutions used the same parameter choices • Function approximation, curve constraints, surface constraints, inverse problems

  35. Immune System Method • I’m not a doctor • Immune system is a distributed system with several functional components in strategic locations

  36. Artificial Immune Sys. • Emulate some facet of the immune system • Learning classifier • Recruitment mechanism • Similar to biological evolution • Feedback • Discover and maintain coverage of diverse patterns • Solutions are drawn from a library of separate gene segements (put the pieces together)

  37. Applications • Virus detection • Scheduling • Antibodies evolve that recognize common patterns of job sequences • Those patterns are used to produce schedules • Optimizing structures (truss)

  38. Graphics Engine as Immune System • Objects are the antibodies • Trying to dispose of them as quickly as possible • Keep a library of drawing routines • Make populations of drawing different parts or the whole object by these different routines • glVertex vs. Display List vs. glArrays vs glStream

  39. GE as IS • When a routine is found to draw the object quickly, remember the kind of object • # vertices, vertex colors, textures • This is kind of like molecules designed for one type of antibody or generalizing to many types • Keep a population of these good solutions (to prevent explosion of drawing programs, dictates type) • Kind of like learned pattern recognition of geometric objects

  40. Particle Swarm (1995) • Social interaction is crucial to human cognition • PS models the exploration of problem space by a population • Individual’s successes influence the searches of their peers

  41. Moving in Hyperspace • For each iteration, an individual’s position is changed by adding a velocity v to its coordinates • Xi(t) = Xi(t-1) + Vi(t) • Vi(t)=Vi(t-1)+(Pi-Xi(t-1)) • If f(xi) < pbesti then pi = xi •  is random

  42. On To Groups • A neighborhood is defined for an individual • Can use whole population (gbest) or a subset (lbest) • An individual looks at the best solution so far pg • (pg-xi) is the distance from individual to best (vector of social influence)

  43. Revised Movement • Vi(t)=Vi(t-1)+1(Pi-Xi(t-1))+2(PG-Xi(t-1)) • Xi(t) = Xi(t-1) + Vi(t) • vMax is max velocity else the system might explode • 1 , 2 usually less than 2 1(Pi-Xi(t-1)) 2(PG-Xi(t-1)) Vi(t-1)

  44. Parameters • Dimensionality • Works better on high than low • Number of Individuals • Inverse relationship between the size of the population and the number of iterations • i should be < 2 • VMAX– might be able to eliminate

  45. More Parameters • Using global best is simpler • Local neighborhood is slower but less susceptible to local minima • Similar ‘cultures’ within the population can emerge • Settle on a local optima that is ‘good enough’ • Inertia Weight • Controls the impact of previous velocities • Can balance global and local • Can decrease over time to focus the search

  46. Applications • Generally used for optimizing high dimensional functions • Very good at making neural nets • Same type of geometric applications (function optimization, complex bounding surfaces…)

  47. Conclusion • We have seen 4 different approaches to the same task of optimization using stochastic methods • Randomness can be  smart • The theory behind these methods is still immature • The application to graphics is still open to debate

More Related