Dynamic Batch Bayesian Optimization

1. Dynamic Batch Bayesian Optimization JavadAzimi, Ali Jalali, Xiaoli Fern Oregon State University University of Texas at Austin In NIPS 2011, Workshop in Bayesian optimization, experimental design and bandits: Theory and applications

2. Bayesian Optimization (BO) • Finding the Maximizerof an unknown function by requesting a small set of • function evaluations (experiments) – experiments are costly • BO assumes prior over – select next experiment based on posterior Current Experiments Select Single/Multiple Experiment Gaussian Process Surface Run Experiment(s)

3. Traditional Approaches • Sequential: • Only one experiment is selected at each iteration • Pros: Performance is optimized • Cons: Can be very costly when running one experiment takes long time • Batch: • experiments are selected at each iteration • Pros: times speed-up comparing to sequential approaches • Cons: Can not performs as well as sequential algorithms

4. Batch Performance (Azimi et.al NIPS 2010) • Given a sequential policy, it chooses a batch of samples which are likely to be selected by the sequential policy. k=5 k=10

5. Motivation • Given a sequential policy, is it possible to simultaneously, • select a batch of experiments • approximately preserve the sequential policy performance. • Size of the batch can change at each time step • Dynamic batch size

6. Proposed Idea: Big Picture • Based on a given prior (blue circles) and an objective function (G), is selected • To select the next experiment, , we need, which is not available • The statistics of the samples inside the red circle are expected to change after observing at • Set the Gvalues for all samples inside the red circleas their upper bound value • If the next selected experiment is outside of the red circle, we claim it is independent from. x2 x3 x1

7. Problems • Which samples statistics are changed after selecting an/a set of experiment? • How can we upper bound the objective function G?

8. Gaussian Process (GP) • GP is used to model the posterior over the unobserved samples in BO • Statistical prediction for each point by a normal random variable rather than deterministic prediction • The posterior variance is independent from the observation

9. Definition • Unobserved set of points • Corresponding Outputs • Any point : z z z

10. GP Theorems

11. Expected Improvement (EI) • Our algorithm inputs a sequential policy to compete with. • We choose Expected Improvement (EI) as criterion • our approach extends to other policies • EI simply computes the expected improvement after sampling at each point

12. Dynamic Batch • samples are asked at each iteration. • if the selected samples are independent from each other. • The first selected sample, is the same as sequential • Choice of the second point depends on • Setting (maximum possible value) EI of the next step is upper bounded • The next sample is selected, if it is not inside the red circle (not significantly effected by )

13. Dynamic Batch: Algorithm

14. Experimental Results: Setting • GP with squared exponential kernel is used as the model • We set nl =20(total number of experiments), and nb=5 (maximum batch size) • The average regret over 100 independent runs is reported where regret is: • Speedup of each framework is reported which is the percentage of experiments asked in batch mode. • ε=0.02 for 2-3 dimensional and 0.2 for higher dimensional frameworks • An alternative and more realistic approach is to set M=(1+ α) ymwhich means (100* α)% improvement at each iteration.

15. Experimental Results: Results

16. Experimental Results: Speedup vs Budget

17. Conclusion and Future works • Conclusion • The proposed dynamic batch approach selects variable number of experiments at each step • The selected experiments are approximately independent from each other • The proposed approach approximately preserves the sequential performance • Future Works • Theoretical analysis of the distance between selected samples in batch and sequential approach. • The analysis of choose of epsilon in performance