Automated Discovery of Numerical Approximation Formulae via Genetic Programming

1. Automated Discovery of Numerical Approximation Formulae via Genetic Programming Matthew Streeter Lee A. Becker Worcester Polytechnic Institute

2. Motivations • Approximations have value in formal mathematics and industrial settings • Discovery of approximation formulae requires human insight or numerical analysis technique (f.e. Taylor series, Padé approximations) • Genetic programming provides general automated technique with potential improvement over existing methods

3. Evaluating Approximations • Given cost and error, utility function assigns value to an approximation • Reasonable utility function assigns higher value to approximations with lower error and cost • Pareto front represents set of approximations which are best under some reasonable utility function

4. Experimental Approach • Calculated cost of each expression by assigning fixed cost associated to each primitive operator • GP system returns Pareto front with respect to cost and error as result of a run • Used parameter settings from (Koza 1992) including populations size of 500, but with generation limit of 100 • Took combined Pareto front for population histories 50 independent runs

5. Rediscovery of Harmonic Number Approximations • Hn Sigma(i=1,n,1/i) • Asymptotic expansion: Hn =  + ln(n) + 1/(2n) - 1/(12n2) + 1/(120n4) - . . . ( 0.57722) • Function set: {+, *, RCP, RLOG, SQRT, COS} • Fitness cases taken as first 50 points of Hn series • Candidate approximations simplified through Maple

6. Evolved Harmonic Number Approximations • Candidate solutions 8-10 are variations on first two terms of asymptotic expansion Terms of Asymptotic Expansion • Candidate solutions 2-7 are variations on first 3 terms • Euler’s constant discovered as: (RCP(SQRT(* 4.67956 RLOG(1.90146))))

7. Rational Polynomial Approximations to Functions of a Single Variable • Function set: {*,+,/,-} • Evolved approximations to 5 common functions: ln(x), sqrt(x), arcsinh(x), exp(-x), tanh(x) • Re-evaluated Pareto front through Maple cost and error procedures • Approximated over large interval to give evolutionary technique the advantage

8. Pareto fronts for approximations to ln(x)

9. f(x) = sqrt(x) f(x) = arcsinh(x) f(x) = exp(-x) f(x) = tanh(x)

10. Approximations of Functions of Multiple Variables • Possible to use single-variable techniques by combining and nesting approximations • This cannot be done for all functions, f.e. f(x,y) = x^y • Genetic programming used to evolve approximating surface for f(x,y) = x^y over area 0 <= x <= 1, 0 <= y <= 1

11. Target function: f(x,y) = x^y Evolved expression: f(x,y) = x/(y2+x-xy3)

12. Evolutionary Refinement of Approximations • New fitness formula: 1/(1+[error multiplier]*[error]) • Refined approximations evolved to sin(x) over interval [0, /2] • Refined 3 approximations whose error function looked simple • Original Pareto front contains 7 approximations; first 4 are refined

13. Refined Approximations to sin(x) over [0, /2]

14. Summary and Conclusions • Rediscovered terms of asymptotic expansion for Harmonic numbers • For common mathematical functions approximated over a large interval, evolved solutions are superior to Padé approximations under some reasonable utility function • Evolved approximations can be refined • Evolved approximations for functions of multiple variables to which Padé approximation cannot be applied

15. Future Work • Iterative refinement, refinement of evolved approximations using numerical analysis technique & vice versa • Larger population size • Use of seed individuals corresponding to existing Padé approximations, Taylor series (see late-breaking paper, “Toward a Better Sine Wave”) • More recent GP features: automatically defined functions, architecture-altering operations • Ideal application would be to discover a genuinely new approximation formula