1 / 33

FDA- A scalable evolutionary algorithm for the optimization of ADFs

FDA- A scalable evolutionary algorithm for the optimization of ADFs. By Hossein Momeni. Factorized Distributed Algorithm. Outline. Factorization Theorem FDA Analysis of FDA for large populations Boltzmann and Truncation selections Finite and critical population Numerical results LFDA.

clio
Download Presentation

FDA- A scalable evolutionary algorithm for the optimization of ADFs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FDA- A scalable evolutionary algorithm for the optimization of ADFs By Hossein Momeni

  2. Factorized Distributed Algorithm Outline • Factorization Theorem • FDA • Analysis of FDA for large populations • Boltzmann and Truncation selections • Finite and critical population • Numerical results • LFDA Of 47 Iran University of Science and Technology November 2006

  3. Factorized Distributed Algorithm Introduction • In a deceptive function the global optimum x=(1,…,1) is isolated. • Neighbors of the second best fitness value x=(0,…,0) have large fitness value • GAs are deceived by the fitness distribution • Most Gas will convergence to x=(0,…,0) Of 47 Iran University of Science and Technology November 2006

  4. Factorized Distributed Algorithm Solutions • Mathematical methods are suitable to optimize deceptive functions • Consider additively decomposed functions (ADF) • Sj are non-overlapping substrings of X with k elements • This class of functions is of great theoretical and practical importance • Optimization of an arbitrary in this space is NP complete Of 47 Iran University of Science and Technology November 2006

  5. Factorized Distributed Algorithm ADFs Optimization Approaches • Adaptive recombination • Explicit detection of relations (kargupta&Goldberg, 97) • Dependency trees(Baluja&Davies, 97) • Bivariate marginal distributions (pelikan&Muhleinbein,98) • Estimation of Distributions(Muhlenbein et all,1997) Of 47 Iran University of Science and Technology November 2006

  6. Factorized Distributed Algorithm ADF • Definition: An additively decomposed function (ADF) is defined by: • For theoretical analysis, use Boltzmann Distribution Of 47 Iran University of Science and Technology November 2006

  7. Factorized Distributed Algorithm Gibbs or Boltzmann distribution • Definition: The Gibbs or Boltzmann distribution of a function f is defined for u>=1 by • is partitionfunction • larger function value f(x) and larger p(x) • Such a search distribution is suitable for an optimization problem • exponential computation Of 47 Iran University of Science and Technology November 2006

  8. Factorized Distributed Algorithm Reduce of B.D. computation • Approximate the Boltzmann distribution (simulated Annealing) • Look for ADFs with distribution computation in Polynomial time • factorize distribution into a product of marginal and conditional probabilities (used by FDA) Of 47 Iran University of Science and Technology November 2006

  9. Factorized Distributed Algorithm Input sets for Factorization theorem Definition: if S={s1,s2, …, sl} for i=1, 2,…, l then In the decomposable graphs theory: di histories bi residuals ci separators Of 47 Iran University of Science and Technology November 2006

  10. Factorized Distributed Algorithm Factorization Theorem Theorem1: Let p(x) be a Boltzmann distribution on X If then Of 47 Iran University of Science and Technology November 2006

  11. Factorized Distributed Algorithm FDAr S0: set t=0, generate (1-r)*N>>o point randomly and r*N points (Equation 16) S1: selection S2: Compute using selected points S3: Generate a new population S4: If termination criteria is met, Finish S5: Add the best point of previous generation to generated points (elitist) S6: Set t=t+1, Go to Step2 Of 47 Iran University of Science and Technology November 2006

  12. Factorized Distributed Algorithm Analysis of Factorization Algorithm • The computational Complexity depends on the factorization and population sizeN • Number of function evaluations: FE=GENe*N GENe is the number of generation till Convergence p(x,t+1)=p(x,t) • The computational Complexity of computing N new search points is • The Computational Complexity of computing probability is Of 47 Iran University of Science and Technology November 2006

  13. Factorized Distributed Algorithm Analysis of … (Contd) • Computation of FDA depends on: • Number of decomposition functions (l) • Size of the defining sets (si) • Size of selected point (M) • An infinite population is needed to exactly computation • Should use a minimal population size N* in a numerical efficient FDA • Computation of N* is a difficult problem for any search method using a population of points Of 47 Iran University of Science and Technology November 2006

  14. Factorized Distributed Algorithm FDA-FAC • S0: set i=1, is non-linear sub-function • S1: compute • S2: Select sk which has maximal overlap with and • S3: if no set is found go to step 5 • S4: Set if i<L go Step1 • S5: Compute the factorization using Eq. 6 with sets Of 47 Iran University of Science and Technology November 2006

  15. Factorized Distributed Algorithm Generation of Initial Population • Normally the initial population is generated randomly • with ADF, initial point can be generated with this information. • Generate subsets with high local fitness values • Distribution is an approximation of • Conditional probabilities are computed using local fitness functions Of 47 Iran University of Science and Technology November 2006

  16. Factorized Distributed Algorithm Generation of Initial Population…. • The larger u, the steeper distribution • if u=1 the distribution is uniform. • if function Onemax(n)=∑xi then • FDA computes span=1 and u=10 Of 47 Iran University of Science and Technology November 2006

  17. Factorized Distributed Algorithm Generation of Initial Population…. • if function Onemax(n)=∑xi then • FDA computes span=1 and u=10 • There will be 10 times more 1s than 0s in the initial population • Such an initial population might not give a B.D. • Only half of the population is generated by this method • Other half is generated randomly Of 47 Iran University of Science and Technology November 2006

  18. Factorized Distributed Algorithm Convergence of FDA • If points are selected base on Bol. Distribution convergence of FDA is proved. • The distribution ps of selected points is given by: • If p(x,t) is B.D. then ps(x,t) is B.D. • FDA computes new search points according to Of 47 Iran University of Science and Technology November 2006

  19. Factorized Distributed Algorithm • Theorem2 : If the initial points are distributed according to with u>=1, then for FDA the distribution at generation is given by with Tip: B. Selection with fixed basis v>1 defines an annealing schedule with that t is number of generation Theorem3 remains valid for any annealing schedule with Of 47 Iran University of Science and Technology November 2006

  20. Factorized Distributed Algorithm • Theorem 3(Convergence): Let be the set of optima, then base on Theorem 2 : • FDA with B. selection is exact simulated annealing algorithm. • simulated annealing is controlled by 2 parameters: N(T) and annealing schedule • N can be called population size Of 47 Iran University of Science and Technology November 2006

  21. Factorized Distributed Algorithm Truncation Selection Vs B. selection • Numerically truncation selection is easier to implement • With truncation threshold ד the best ד*N individual are selected. • Conditional probabilities of selected point is: • Based on factorization theorem to generate new search points : • Problem: After Truncation selection the distribution is not B.D. therefore: • With this inequality that this makes a convergence proof difficult. Of 47 Iran University of Science and Technology November 2006

  22. Factorized Distributed Algorithm Theoretical Analysis for Infinite populations • For analysis two linear function will be investigated: • OneMax has (n+1) different fitness value which are multinomial D. • Int has 2n different fitness value. • For ADFs the multinomial distribution is typical • The distribution generated by Int is more special • Both functions is linear, therefore can use following factorization: Of 47 Iran University of Science and Technology November 2006

  23. Factorized Distributed Algorithm • Theorem 4 For B. selection with basis v the probabilities distribution for OneMax is given by: • Number of generations to generate the optimum is given by: Of 47 Iran University of Science and Technology November 2006

  24. Factorized Distributed Algorithm • Theorem 5 For Truncation selection ד with selection intensity Iד the marginal probability p(t) obeys for OneMax • The approximate solution of this equation is : Where • The number of generations till convergence is given by: Of 47 Iran University of Science and Technology November 2006

  25. Factorized Distributed Algorithm Of 47 Iran University of Science and Technology November 2006

  26. Factorized Distributed Algorithm Comparison Truncation & B. selection • T.S. need more number of generation to convergence than B.S. • GENe is of order for B.S. and for T.S. is • If basis v is small (e.g. v=1.2) T.S. convergence is faster Of 47 Iran University of Science and Technology November 2006

  27. Factorized Distributed Algorithm • B.S. with fixed v gives an annealing schedule of Of 47 Iran University of Science and Technology November 2006

  28. Factorized Distributed Algorithm • FDA with truncation selection generates a B.D. with annealing schedule • The annealing schedule depends on the average fitness and the variance of the population. Of 47 Iran University of Science and Technology November 2006

  29. Factorized Distributed Algorithm Of 47 Iran University of Science and Technology November 2006

  30. Factorized Distributed Algorithm Of 47 Iran University of Science and Technology November 2006

  31. Factorized Distributed Algorithm • For Int the B.D. is concentrated around the optimum • The selected population has a small diversity • In finite population this cause a problem, some genes will get fixed to wrong alleles Of 47 Iran University of Science and Technology November 2006

  32. Factorized Distributed Algorithm Analysis of FDA for Finite Populations In finite population, convergence of FDA can be Probabilistic Of 47 Iran University of Science and Technology November 2006

  33. Factorized Distributed Algorithm Analysis of FDA for Finite Populations Cumulative fixation probability for Int(16) Truncation Selection vs. Boltzmann selection with v=1.01 Of 47 Iran University of Science and Technology November 2006

More Related