1 / 30

An update on BFG, The Bespoke Framework Generator

An update on BFG, The Bespoke Framework Generator. Graham Riley (& Rupert Ford, STFC) Coupling Workshop 2013 @ Boulder, Colorado - February 20 th -22 nd 2013. Overview. Why BFG? What is BFG? Not a coupler! What is new? New implementation in python, faster

daryl
Download Presentation

An update on BFG, The Bespoke Framework Generator

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An update on BFG,The Bespoke Framework Generator Graham Riley (& Rupert Ford, STFC) Coupling Workshop 2013 @ Boulder, Colorado - February 20th-22nd 2013

  2. Overview • Why BFG? • What is BFG? Not a coupler! • What is new? • New implementation in python, faster • Interface to other languages (‘R’ and GAMS in IAMs) • Component- and program-compliance • Portal and BFG utilities • the power of metadata! • Parallel model support (and Infrastructure API) • Model ‘export’ – step to interoperability... • to couple to existing models using other technologies, such as ESFM or OASIS • References & Future

  3. Why BFG? • Community-based coupled modelling (still) emerging: • From Centres of expertise… • in and across science domains (e.g. ESM, IAM, Flood modelling etc.) • Distributed across institutions • Implied: need for rapid exploration of use of models; sharing of models etc. • Requirement for flexibilityto choose appropriate models

  4. Why BFG? • This historic development of science has led to several coupling technologies (infrastructures/frameworks): • Shared memory, MPI, TDT, MCT, OASIS, ESMF, CESM, FMS… • (and other infrastructure, e.g. for parallel models etc.) • Requirement for flexibility to choose appropriate supporting technology • Similar situation for the community which led to MPI standard?

  5. Why BFG? • Choice of computing systems: • Many options… • (Parallel) laptop/desktop, cluster, cloud, supercomputers… • Rapid developments in hardware technology • Requirement for flexibility to use available computing systems efficiently

  6. What is BFG? Aims… • Isolation of science code (i.e. models) • Protect science code from changes in supporting software technologies and hardware (as far as possible) • Flexibility in composition and deployment • Allow models to be put together rapidly to create new coupled models (as science permits!) • Allow the models to be deployed into program units to make best use of computing resources • Allow the coupled model to use the most appropriate existing coupling technology (or technologies) • Provide this flexibility through metadata descriptions and code generation (BFG2) • See other benefits of metadata-based approach…

  7. What is BFG2? • A metadata description of a coupled model • Capturing both its static structure and run-time, dynamic behaviour. • The individual models described have to conform to certain, minimal, rules (see next). • A set of tools that use the metadata • BFG2 - generates ‘wrapper’ code that can be compiled with model code to build an implementation. • Using a choice of existing (coupling) technologies etc. • Visualise the coupled model (static composition and dynamic – control schedule – behaviour) • Other utilities – see the BFG portal later…

  8. What is a BFG2-compliant model • Component compliance (fine-grain models) • Essentially a subroutine (in Fortran) • Science code communication • by argument or ‘in-place’ put() and get() • Control ‘outside’ the model • Program compliance • A model already in an existing program • Communication by put() and get() • Other BFG calls (e.g. end_of_step()) • Possibly embedded control • (Support composition of fine-grain models and aggregation of program-based models

  9. A component-compliant model module atmos use bfg, only : put,get implicit none private public :: init,iteration,finalise contains subroutine init(arg1,arg2,...) end subroutine init subroutine iteration(arg1,arg2,...) call get(data3,tag3) call put(data4,tag4) end subroutine iteration subroutine finalise(arg1,arg2,...) end subroutine finalise end module atmos • ‘Standard’ init, run, finalise • Couping by argument • Tags generated by BFG • Couping by in-place put()/get() • The ‘tag’ connects to the metadata! • Key to BFG

  10. BFG2 model ‘wrapping’ • Component-compliance Container, Control (“Driver” layer), ArgPass Data, ArgPass Wrapper Model Code(s) InPlace Calls Concurrency Target Coupling Infrastructure - Existing code/library code - BFG-generated code

  11. Flexible Deployment bio-geo chem T T ocean atm transformer T T model sea ice program unit/deployment unit

  12. A program-compliant model • Stand alone program program atmos use bfg, only : put,get, bfg_init, bfg_eos, bfg_finalise implicit none call bfg_init() call get(data1,tag1) call put(data2,tag2) do i=1,nts call get(data3,tag3) call work(...) call put(data4,tag4) call bfg_eos() end do call get(data5,tag5) call put(data6,tag6) call bfg_finalise() end program atmos • Generated ‘target’ technology init. • Only put()/get() in the ‘run’ section • Calls to tell BFG about control structure

  13. Model(s) Name, language Type Science/transformer Coupling data As args: in, out, inout As put/get timestep Composition Coupling description What connects to what Priming data Which connections are first started Controls sequential or concurrent model execution Overview of BFG2 metadata • Deployment • Mapping of models to ‘threads of control’ and main programs • to sequence units (“this model runs before that” in a ‘thread’) • to deployment units (-> the sequence units in a main program) • Schedule • Iteration control of model execution • Nested loop structures allowed • Target coupling technology Think of SPMD model All together, the metadata captures the run-time behaviour of the coupled model

  14. What is BFG2? - summary script model Description xml coupled xml Input xml Output Composition xml Deployment xml Templates & xslt xslt • Metadata + generative approach Code and scripts Python engine

  15. The Power of Metadata - BFG Portal • For examples of the use of metadata • Prototype, in development • bfg.cs.man.ac.uk/ • Demos available on request! • Several utilities exist • Validation • Upload metadata and run BFG • Access to some (small) examples • Visualisation of composition and control/schedule • Model stub generation (for test purposes) • Makefile generation • …

  16. Simple ESM…Composition andSchedule views

  17. Future? • Trend towards a common, shared infrastructure for ESM (and beyond)? • Similar situation for the community which led to MPI? • And/Or – go for Framework interoperability • BFG export is a first exploration • (we have a design for BFG import too) • Use of (and extension of) CIM metadata • Out of METAFOR project • More uses of metadata • Including support for performance analysis, for example. • BFG as a Domain Specific Language…

  18. Web References • BFG web page – including papers • http://cnc.cs.man.ac.uk/projects/bfg • BFG portal: • http://bfg.cs.man.ac.uk/ • A BFG Wiki (from the GENIE project) • https://source.ggy.bris.ac.uk/wiki/GENIE_BFG • IS-ENES,EU project deliverables • D8.3, “Towards Flexible Construction of ESMs using BFG”, March 2012 • D8.5 – “ditto”, final report, March 2013

  19. Thanks to our sponsors…

  20. Towards parallel models (D8.5) • Support for parallel models on quasi-uniform grids • Wider than coupling • Looking at other infrastructure support too • E.g. halo exchange (from MCT and ESMF) • Prototype “manUgen” solution • Mainly targeting MCT • but some ESMF and OASIS3-MCT (for coupling) tests • Developed a prototype Infrastructure API • Metadata + configuration of run-time system approach • plus some code generation

  21. Infrastructure API (D8.5) • A minimal API for use by model developers • Metadata-driven • Target coupling and halo exchange • Support implementation of API in any of a number of existing technologies • Select ‘best of breed’

  22. Parallel Model Support (D8.5)

  23. Code Generation vs Configuration (D8.5) Metadata describing coupling and partition etc. Per model ModelInfo structure captures specific requirements – for halo exchange and coupling etc. Models use ‘tag’ in calls to access ModelInfo

  24. Composite models – BFG2 export (D8.3) BFG atmos components dyn conv rad Export to OASIS Export to ESMF Atmos (OASIS) Ocean (OASIS) Ocean (ESMF) Atmos (ESMF) Coupling via OASIS Coupling via ESMF Existing Ocean models

  25. ERMITAGE use case LPJmL – veg./hydrology model GENIE ‘fast’ ESM model (or emulator) MAgPIE – land use model TIAM/REMIND – enegy/economic models

  26. An IAM example

  27. ‘Import’ of models • Aim: to enable models written to use different existing coupling technologies to talk to each other • Using a selected communication mechanism…

  28. BFG import BFG-generated ‘adaptor’ code BFG Adaptor BFG Adaptor Ocean (OASIS) ? Atmos (ESMF) Atmos (ESMF) Import both models Coupling via OASIS Coupling via BFG-generated ‘target’ Coupling via BFG-generated ESMF Coupling via BFG-generated TARGET E.g. web services Target could be: (almost) anything BFG will support… … MPI, GridMPI, OASIS3/4, ESMF, web services…

  29. BFG import… BFG Adaptor Ocean (OASIS) Atmos (BFG-gen) Coupling via OASIS Coupling via BFG-generated ‘target’ Coupling via BFG-generated ‘target’

  30. Summary • Working with real ESM codes takes a lot of effort • Experience with GENIE and JULES • Using ‘toy’ models to demonstrate capability and potential • Motivation from current ESMs is small • Few models (but growing number) • Large cost of change • Seeking use cases for export and import • Examples of community collaboration (c.f. NEMO) • Case for import made in ERMITAGE project • An emerging infrastructure requiring long term support for the community

More Related