230 likes | 396 Views
To STOP or not to STOP. By I. E. Lagaris. A question in Global Optimization. Contributions. Research performed in collaboration with Ioannis G. Tsoulos . PhD candidate, Dept. of CS, Univ. of Ioannina. Searching for “Local Minima”. One-Dimensional Example Exhaustive procedure :
E N D
To STOP or not to STOP By I. E. Lagaris A question in Global Optimization Department of Computer Science, University of Ioannina Ioannina - GREECE
Contributions Research performed in collaboration with Ioannis G. Tsoulos . PhD candidate, Dept. of CS, Univ. of Ioannina Department of Computer Science, University of Ioannina Ioannina - GREECE
Searching for “Local Minima” One-Dimensional Example Exhaustive procedure: From left to right minimization-maximization repetition. Department of Computer Science, University of Ioannina Ioannina - GREECE
Level plots in 2-D Searching for “Local Minima” Two-Dimensional Example “Egg holder” The exhaustive technique used in one-dimension, is not applicable in two or more dimensions. Department of Computer Science, University of Ioannina Ioannina - GREECE
The “MULTISTART” algorithm • Sample a point x from S • Start a local search, leading to a minimum y • If y is a new minimum, add it to the list of minima • Decide“ to STOP or not to STOP ” • Repeat If the decision is right, the iterations will not stop before all minima inside the bounded domain S are found. Department of Computer Science, University of Ioannina Ioannina - GREECE
The set of all points that when a local search is started from, concludes to the same minimum. • Formally: • The RA depends strongly on the local search (LS) procedure. • The measure of an RA is denoted by m(Ai). The “Region of Attraction” (RA) Department of Computer Science, University of Ioannina Ioannina - GREECE
ASSUMPTIONS … • Deterministic local search. Implies non-overlapping basins. • Sampling is based on the uniform distribution. Implies that a sampled point belongs to Ai with probability: • There is no zero-measure basin, i.e. Department of Computer Science, University of Ioannina Ioannina - GREECE
If can be calculated, then a rule may be formulated based on the space coverage: w, being the number of minima discovered so far. Coverage based stopping rule i.e.: STOP when c→1 Department of Computer Science, University of Ioannina Ioannina - GREECE
note that: Estimating m(Ai) Let L be the number of the performed local searches and Li those that ended at yi. An estimation then, may be obtained by: Unfortunately this estimation is useless in the present framework, since it will always yield: c=1 Department of Computer Science, University of Ioannina Ioannina - GREECE
Double Box Consider a box S2 that contains S and satisfies: Sample points from S2, and perform local searches only from points contained in S. L, now stands for the total number of sampled points. Department of Computer Science, University of Ioannina Ioannina - GREECE
and → 1 → 0 and Implementation • Keep sampling from S2 until N points in S are collected. (N =1 for Multistart) • At iteration k, let Mk be the total number of sampled points (kN of them in S). • STOP if: last indicates the iteration during which the latest minimum was discovered Department of Computer Science, University of Ioannina Ioannina - GREECE
Multistart performance with Double Box, for a range of p values Department of Computer Science, University of Ioannina Ioannina - GREECE
Observables rule • This rule relies on the agreement of values of observable (i.e. measurable) quantities, to their expected asymptotic values. • The number of times Li that minimum yi is found, is compared to its expected value. • yi are indexed in order of their appearance. Hence y1 requires one application of the LS, y2 requires additional n2 applications, y3 additional n3 … • Let the number of the recovered minima so far be denoted by w. Department of Computer Science, University of Ioannina Ioannina - GREECE
Expectation values The expectation value of the number of times the ith minimum is found, at the time when the wthminimum is recovered for the first time, is recursively given by: An estimation that may be used is: Department of Computer Science, University of Ioannina Ioannina - GREECE
Keep trying … Suppose that after having found w minima, there is a number (say K) of consecutive trials without any success, i.e. without finding a new minimum. The expected number of times the ith minimum is found at that moment is given recursively by: Department of Computer Science, University of Ioannina Ioannina - GREECE
The Observables’ criterion The quantity: Tends asymptotically to zero. Hence, STOP if: Department of Computer Science, University of Ioannina Ioannina - GREECE
“Expected Minimizers” Rule • Based on estimating the number of local minima inside the domain of interest. • The estimation is improving as the algorithm proceeds. • The key quantity is the probability thatlminima are found after m trials. • Calculated recursively. Department of Computer Science, University of Ioannina Ioannina - GREECE
then the probability of finding l minima in m trials is given by: If stands for the probability to recover in a single trial, Probability that one of the first l minima is found again. Probability that a new minimum is found other than Probabilities Note that: Department of Computer Science, University of Ioannina Ioannina - GREECE
We use the estimate: The corresponding variance is given by: Expected values The expected value for the number of minima, estimated after m trials is given by: The RULE STOPif: Department of Computer Science, University of Ioannina Ioannina - GREECE
Uncovered fraction of space: Zieliński (1981) STOP if: Boender & Rinnooy Kan (1987) Estimated number of minima: STOP if: Boender & Romeijn (1995) Probability all minima are found: STOP if: Other rules Department of Computer Science, University of Ioannina Ioannina - GREECE
MULTISTART Department of Computer Science, University of Ioannina Ioannina - GREECE
TMLSL Department of Computer Science, University of Ioannina Ioannina - GREECE
Conclusions • The new rules improve the performance at least for problems in our benchmark suite. • Proper choice of the parameter p, for different methods is important. • Remains to be seen if performance is also boosted in other practical applications. Department of Computer Science, University of Ioannina Ioannina - GREECE