The ZOOM Minimization Package

1. The ZOOM Minimization Package David SachsMark FischlerFermi National Accelerator Laboratory

2. Minimization Package • Object-oriented C++ library for minimization • Minuit capabilities, and more • Suitable for use • In stand-alone programs • As part of Root applications • In other frameworks • The OO design allows the Minimizer to ‘play well with others’ • As central part of a fitter; provided as examples will be: • an unbinned maximum likelihood fitter • a binned chi-squared fitter,

3. Benefits of OO Design • User convenience • Domain concepts reflected more directly in code • Natural to encapsulate function to be minimized • Avoidance of global variables allows • multiple simultaneous minimizations • use in multi-threaded applications • Extensibility • Separation of concepts allows for independent replacement • Benefits from any revision of code: • Find features that “should have” been there • Sometimes you uncover flaws… yes, even in Minuit

4. Brief Survey of C++ Minimizers • Other C++ packages are available, but do not use modern C++ • and so lack the OO advantages • Other Projects include • The MINUIT Project [SEAL/MathLib] • Hand-translation of Minuit (James and Winkler) • ROOT minimization/fitting • Hand-smoothed automated translation of Minuit

5. An Example of Use double f(const std::vector<double>& x) { return x[0]*x[0]+ //… some function of // x[0] thru x[4] } int main() { using namespace Minimization; const int Ndimensions = 5; Problem m(UserFunction(Ndimensions, f)); m.minimize(); cout << m.currentPoint() << endl; }

6. Directions of Extensibility • class Terminator • When do we stop improving the minimum? • class Domain • How do we restrict the allowed values of the parameters? • class Algorithm • How do we refine existing minimization methods? • How do we add new minimization techniques?

7. Terminators • A terminator is an object that can be asked whether or not minimization is complete • it must be an instance of a class derived from class Terminator • Several standard terminators are provided • These can be combined by ORs and ANDs • Terminators typically make use of the state of the minimization • Users can easily create their own terminator classes

8. An Example Terminator class NCalls : public Terminator { int n; public: explicit NCalls(int limit):n(limit) {} ... TerminationType finished (const ProblemState& p) { return p.functionCallCount >= n ? TTStop : TTContinue; } };

9. Domains • A domain is an object that maps internal coordinates to external coordinates and vice versa • It must be an instance of a class derived from Domain • Custom domain classes must provide • Mapping from external M-space to “interior” unrestricted N-space, and vice versa • Gradient calculation • One domain is provided: RectilinearDomain • independent [-, ], [a,b], [-, b], [a, ] intervals • maps for [a,b] intervals match those in Minuit

10. Some Possible Custom Domains • Orthogonal domain, but with sigmoid mappings instead of arcsine mappings • Superior for some cases, worse for others • Multiple probability space • All parameters must be non-negative • Parameters must sum to 1 • Interior (or surface) of N-Sphere • Lots of possible mappings • But watch out if minimum lies very near a mapping singularity

11. Algorithms • An algorithm is an object that encapsulates a minimization technique • It must be an instance of a class derived from Algorithm • Custom algorithm classes can be written by experts. • Two algorithms are provided: • Migrad • Simplex • Other algorithms will be provided: • Seek and others completing Minuit collection • Further algorithm implementation possibilities: • Fumili • biConjGradStab • LEAMAX

12. Adding a New Algorithm • We might wish to add a Simplex-like algorithm but with: • improved estimation of the distance to the minimum (edm) • strategies to avoid false convergence • The mathematics of finding the best strategy is still hard • OO design makes adding the algorithm straightforward, once the proper strategy is discovered

13. The Simplex Algorithm • The algorithm: • N+1 points in N-space form a simplex • Repeatedly reflect the “worst” point about the centroid of the remaining points • Sometimes shrink if that improves function • Sometimes expand if reflected point is best point • When estimated distance to minimum is small, stop • Problem: The edm estimate formed by simplex “is largely fantasy.”

14. Discovery of a Flaw in Minuit • Simplex can converge to a non-minimum • Even on well-behaved (quadratic) functions! • When starting a large distance from the true minimum • How does this happen? • Simplex becomes thin, aligned with an equipotential • There is a good direction to expand off that line but… • simplex shrinks before it finds that direction and… • triggers convergence criterion based on the false edm • If the edm were big at that point, we would at least know something was wrong • The fix is still not trivial but knowing a meaningful edm is a start

15. Apparent Convergence MINUIT Minimization The simplex algorithm is stopped when it indicates it is near the function minimum.

16. Convergence Failure MINUIT Minimization MINUIT’s simplex indicates convergence while still far from it. The error is fixed in Minimization

17. Conclusion • Minimization package available from • http://cepa.fnal.gov/aps/minimization.shtml • Standard build mechanism (configure/make) • Modern, Object Oriented C++ • Suitable for some multi-threaded operation • Designed to be extensible to new • Termination criteria • Domains (parameter restrictions) • Algorithms • Corrects a flaw in Minuit • In the Simplex algorithm

18. The ZOOM Minimization Package David SachsMark FischlerFermi National Accelerator Laboratory