Tuning GENIE Earth System Model Components using a Grid Enabled Data Management System

1. Tuning GENIE Earth System Model Components using a Grid Enabled Data Management System Andrew Price, Gang Xue, Andrew Yool, Dan Lunt, Tim Lenton, Jasmin Wason, Graeme Pound, Simon Cox and the GENIE team. http://www.genie.ac.uk/ UK e-Science – All Hands Meeting 3rd September 2004

2. Outline • Introduction • Scientific aims of GENIE • e-Science tools • Data Management System • Geodise Toolboxes • OPTIONS Design Search and Optimisation • Results • Future work • Conclusions UK e-Science - All Hands Meeting, Nottingham, 2004

3. Introduction • The GENIE project is developing a Grid-based system to: • Flexibly couple together state-of-the-art components to form a unified Earth system model • Execute the resulting model on the Grid • Share the distributed data produced in simulations • Provide high-level open access to the system, creating and supporting virtual organisations of Earth system modellers UK e-Science - All Hands Meeting, Nottingham, 2004

4. ice-age ice-age ice-age ice-age Scientific Aims • Orbital parameters affect incident radiation and climate • Biological and geological processes interact with, and feedback upon, the climate (via, for instance, CO2) UK e-Science - All Hands Meeting, Nottingham, 2004

5. 3D atmosphere Atmospheric CO2 3D ice sheets 2D sea ice 2D land surface 3D ocean Land vegetation Ocean biogeochemistry Ocean sediments Atmosphere – Bristol’s IGCM3 Ocean – SOC’s GOLDSTEIN Land – Met. Office’s TRIFFID Land ice – Bristol’s GLIMMER Ocean biogeochemistry and sediments – UEA’s BioGEM The target GENIE Model UK e-Science - All Hands Meeting, Nottingham, 2004

6. Initial GENIE experiments • Initial studies in GENIE performed parameter sweeps to investigate the properties of the model UK e-Science - All Hands Meeting, Nottingham, 2004

7. e-Science Tools • Data Management System (augmented version of the Geodise Database System) • Matlab scripting environment • Geodise Toolboxes • XML Toolbox • OPTIONS Design Search and Optimisation package • Template and Example scripts UK e-Science - All Hands Meeting, Nottingham, 2004

8. Globus Server Database Web Services Jython Location Service Authorisation Service Portal Metadata Archive & Query Services Data Management System Client Grid Geodise Database Toolbox Jython Functions Java Client Code SOAP Apache Axis Matlab Functions Metadata Database CoG GridFTP XML Schema UK e-Science - All Hands Meeting, Nottingham, 2004

9. Jython Local Resources (GT2) Imperial Condor Pool Southampton Condor Pool Grid Computation National Grid Service (GT2) Oxford Leeds RAL Manchester Java CoG Flocked Condor Pools UK e-Science - All Hands Meeting, Nottingham, 2004

10. Geodise Toolboxes UK e-Science - All Hands Meeting, Nottingham, 2004

11. Scripting a Tuning Study MATLAB functionRMS_Error = cgoldstein(params) optimum = fminsearch( … @cgoldstein, params, … ) GENIE Database gd_query(results) Grid Resource gd_putfile(CG binary) CG binary gd_putfile(config file) config file gd_jobsubmit(RSL) gd_getfile(results file) results file gd_archive(results) return RMS_Error UK e-Science - All Hands Meeting, Nottingham, 2004

12. Matlab Optimisation Toolbox % ************************ % Specify a starting point % ************************ parameters = [ 0.5 ]; % ************************ % Perform the minimisation % ************************ optimum = fminsearch( @cgoldstein_1D, parameters, optimisation_parameters ) % ************************ % Specify a starting point % ************************ parameters = [ 420 5000000 ]; % ************************ % Perform the minimisation % ************************ optimum = fminsearch( @cgoldstein_2D, parameters, optimisation_parameters ) UK e-Science - All Hands Meeting, Nottingham, 2004

13. OPTIONS • Matlab interface to the Options design exploration system • http://www.soton.ac.uk/~ajk/options/welcome.html • State of the art design search and optimisation algorithms • Design of Experiment methods • Response Surface Modelling • Over 30 search methods including: • Adaptive Random Search (ADRANS), Powell's Direct Search (PDS), • Simplex Method (SIMP), Genetic Algorithm (GA), • Simulated Annealing (SA), Evolutionary Programming (EP) UK e-Science - All Hands Meeting, Nottingham, 2004

14. Grid Computation OptionsMatlab optjobparallel.m GENIE Database objfun.m objfun_parse.m National Grid Service (GT2) Local Resource (GT2) Oxford Leeds RAL Manchester UK e-Science - All Hands Meeting, Nottingham, 2004

15. OptionsMatlab • >> OptionsInput = createOptionsStructure(4.0) • DNULL: -777 • OLEVEL: 2 • MAXJOBS: 100 • NVRS: 12 • VNAM: {'SCLTAU' 'INVDRAG' 'OCNHORZDF' ... } • LVARS: [1.3000 2.0000 2500 ... ] • UVARS: [2.1000 4.8000 5700 ... ] • VARS: [1.7000 3.4000 4100 ... ] • ONAM: 'RMSERROR' • OMETHD: 4.0000 • DIRCTN: -1 • NITERS: 1000 • OPTFUN: 'cgoldstein_12D' • OPTJOB: 'optjobparallel' • GA_NPOP: 100 • >> OptionsOutput = OptionsMatlab(OptionsInput); Available Optimisation Methods: 1.1 for OPTIVAR routine ADRANS 1.2 for OPTIVAR routine DAVID 1.3 for OPTIVAR routine FLETCH 1.4 for OPTIVAR routine JO 1.5 for OPTIVAR routine PDS 1.6 for OPTIVAR routine SEEK 1.7 for OPTIVAR routine SIMPLX 1.8 for OPTIVAR routine APPROX 1.9 for OPTIVAR routine RANDOM 2.1 for user specified routine OPTUM1 2.2 for user specified routine OPTUM2 2.3 for NAG routine E04UCF 2.4 for bit climbing 2.5 for dynamic hill climbing 2.6 for population based incremental learning 2.7 for numerical recipes routines 2.8 for design of experiment based routines 3.11 for Schwefel library Fibonacci search 3.12 for Schwefel library Golden section search 3.13 for Schwefel library Lagrange interval search 3.2 for Schwefel library Hooke and Jeeves search 3.3 for Schwefel library Rosenbrock search 3.41 for Schwefel library DSCG search 3.42 for Schwefel library DSCP search 3.5 for Schwefel library Powell search 3.6 for Schwefel library DFPS search 3.7 for Schwefel library Simplexsearch 3.8 for Schwefel library Complexsearch 3.91 for Schwefel library two­membered evolution strategy 3.92 for Schwefel library multi­membered evolution strategy 4 for genetic algorithm search 5 for simulated annealing 6 for evolutionary programming 7 for evolution strategy UK e-Science - All Hands Meeting, Nottingham, 2004

16. Twin-Test Experiment Attempt to recover a known state of the model using a Genetic Algorithm. Performed 10 generations of a 100 member population. Then applied a local Simplex search of the best candidate. Population too small to find optimal solution – suitable for finding local minima UK e-Science - All Hands Meeting, Nottingham, 2004

17. Model Sea Surface Temperatures Model Air Temperatures NCEP Sea Surface Temperatures NCEP Air Temperatures Tuning using Observational Data • Apply the same method but calculate the RMS error statistic by comparing the model state with NCEP observational data. • The lack of a land surface in the model means tuning cannot match the observational data. UK e-Science - All Hands Meeting, Nottingham, 2004

18. IGCM3 Atmosphere Model • The objective function is a weighted sum of the RMS differences between the model state and NCEP data. • Winter and Summer averages for a number model fields. UK e-Science - All Hands Meeting, Nottingham, 2004

19. IGCM Results • 25% reduction in error statistic compared to default parameters • Similar result to a parallel study performed using the Ensemble Kalman Filter • Model physics insufficient to perfectly match observational data. UK e-Science - All Hands Meeting, Nottingham, 2004

20. e-Science Summary UK e-Science - All Hands Meeting, Nottingham, 2004

21. Conclusions • Provided the environmental scientist with a toolset for tuning GENIE models: • Scripting environment • Database repository • Computational Grid interface • Suite of generic optimisation algorithms • A Global minimum can reliably be found in low dimensional problem space. • For higher dimensional problems, the tools are appropriate for locating local minima in the state space. UK e-Science - All Hands Meeting, Nottingham, 2004

22. Coordinator: Tim Lenton – CEH Edinburgh Principal investigator: Paul Valdes – Bristol Research Team and Collaborators: James Annan – FRSGC, Japan Chris Brockwell – UEA Norwich David Cameron – CEH Edinburgh Peter Cox – Hadley Centre (UKMO) Neil Edwards – Bern, Switzerland Murtaza Gulamali– London e-Science Centre Julia Hargreaves – FRSGC, Japan Phil Harris – CEH Wallingford Dan Lunt – Bristol Bob Marsh– SOC Andrew Price – Southampton e-Science Centre Andy Ridgwell – UBC, Canada Ian Rutt – Bristol Gang Xue – Southampton e-Science Centre Andrew Yool – SOC The GENIE Team Management Team: Melvin Cannell – CEH Edinburgh Trevor Cooper-Chadwick – Southampton e-Sci. Centre Simon Cox – Southampton e-Sci. Centre John Darlington – London e-Science Centre Richard Harding – CEH Wallingford Tony Payne – Bristol John Shepherd – SOC Andrew Watson – UEA Norwich Thanks to Steven Newhouse UK e-Science - All Hands Meeting, Nottingham, 2004