Virtual Geophysics Laboratory: Empowering Geoscience Research Effortlessly

1. Virtual Geophysics Laboratory (VGL) VGL v1.1 Launch Ryan Fraser, Terry Rankine, Joshua Vote, Lesley Wyborn, Ben Evans, Robert Woodcock February 2013 CSIRO | Minerals down under flagship

2. Geo-information bedrock surficial mineral geophysical geochemical geochronologic hyrdrogeological $ Gather data, process it, publish resultsSimple, isn’t it? data knowledge Virtual Geophysics Laboratory v1.1 | Ryan Fraser

3. Let’s map it out for Geophysics … Hardcopy of data Proprietary Package Maths Package Get handed field data Visualise data Transform to a regular grid Download results Crop data to area of interest Reformat data for processing SSH Client Configure job and start processing Upload data to HPC resource Virtual Geophysics Laboratory v1.1 | Ryan Fraser

4. The Past • Compile raw data using proprietary FORTRAN • Also use other proprietary software • Transform to a regular grid using more software • MATLAB, Intrepid, ER Mapper, ESRI ArcGIS, QGIS, etc • Crop data spatially • eg: everything in Victoria • Transform data into a file format that can be read by proprietary scientific code • This is usually done with some handwritten python or c • There is no version control, code is often rewritten / redone • Upload data to HPC • Manually enter input parameters/start job • Where did my results go, how do I visualise? Virtual Geophysics Laboratory v1.1 | Ryan Fraser

5. There seems to be a problem… • Reproducibility – there is none • What was the input of your model? • What transformations occurred? • It’s a manual process • Time consuming • Error prone • Expensive • Licensing costs • Effort/time Virtual Geophysics Laboratory v1.1 | Ryan Fraser

6. Scientific workflow – Virtual Geophysics Laboratory (VGL) • Automated - Simplify access and integration • Reproducible process • Open access (less expense) • Allow scientists to focus on the science and not the tech • Exploiting the Cloud and HPC • Operational or education tool • Automates and massively expands geophysics computational capacity via the Cloud Virtual Geophysics Laboratory v1.1 | Ryan Fraser

7. V(what)GL • VEGL – Virtual Exploration Geophysics Laboratory • One primary science collaboration • One primary workflow • One collection of geophysical data sets • VGL - Virtual Geophysics Laboratory • NeCTAR funded activity • Collaboration with multiple partners (CSIRO, NCI, GA, UQ, Monash, ANU) • Supporting multiple workflows • New data types and collections • New Use – Not just exploration. Done Virtual Geophysics Laboratory v1.1 | Ryan Fraser

8. Let’s map it out… Hardcopy of data Proprietary Package Maths Package Get handed field data Visualise data Transform to a regular grid Download results Crop data to area of interest Reformat data for processing SSH Client Configure job and start processing Upload data to HPC resource Virtual Geophysics Laboratory v1.1 | Ryan Fraser

9. …to this Virtual Geophysics Laboratory Select spatial bounds Build “science” from existing libraries Discover raw data Run job Collect and publish results Virtual Geophysics Laboratory v1.1 | Ryan Fraser

10. Introducing The Virtual Geophysics Laboratory Virtual Geophysics Laboratory v1.1 | Ryan Fraser

11. Powered by the Spatial Information Services Stack Common Platform Environment, Water Marine • Groundwater Geology Geophysics 11 | Virtual Geophysics Laboratory v1.1 | Ryan Fraser

12. VGL: Opportunities • Free for research use • Supporting multiple workflows • Model Registry (3D) – researcher contributed models • New Scientific Codes – Underworld, eScript, UBC, Airborne EM inversion codes + others (relatively “easy” to add) • Data sets from GA: National Airborne Geophysical DB including • Gravity, Radiometric, AEM, Magnetics • Data sets from State Surveys • Broad application and generic implementation • Scalable Infrastructure (Cloud) Virtual Geophysics Laboratory v1.1 | Ryan Fraser

13. What’s left? • More data, more codes, more workflows! • BYO Resource (cloud allocation) • Confidential Data • Pawsey Centre, NCI, commercial clouds Virtual Geophysics Laboratory v1.1 | Ryan Fraser

14. VGL is Power by: Thank you VGL acknowledges funding from the NeCTAR project http://www.nectar.org.au NeCTAR is an Australian Government project conducted as part of the Super Science initiative and financed by the Education Investment Fund. Ryan FraserResearch Manager t +61 8 6436 8760 e ryan.fraser@csiro.au w siss.auscope.org VGL is a collaboration between: VGL is supported by: University of Queensland, Monash University and the Australian National University VGL Launch Event is sponsored by:

15. Introducing The Virtual Geophysics Laboratory Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

16. No Installation Necessary! Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

17. Don’t Compute on your Laptop Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

18. Flexibility What I want to do ≠ How I want to do it ≠ Where I want to do it Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

19. Provenance Don’t be this guy... Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

20. To Quickly Recap • VGL is a different kind of workflow engine... • No installation necessary • Doesn’t process on your machine • Very flexible • Records results and the process Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

21. Time for a live demo... Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

22. VGL is Power by: VGL acknowledges funding from the NeCTAR project http://www.nectar.org.au NeCTAR is an Australian Government project conducted as part of the Super Science initiative and financed by the Education Investment Fund. VGL is a collaboration between: VGL is supported by: University of Queensland, Monash University and the Australian National University VGL Launch Event is sponsored by: http://vgl.auscope.org

23. Data discovery Layers discovered via remote registries Layers consist of numerous remote data services Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

24. Data discovery Some data services support subsetting Some data services support reformatting e.g. CSV, NetCDF, GeoTIFF Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

25. Data discovery Some data is only registered with flat files Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

26. Data processing A variety of different scientific codes are already available in the form of “Toolboxes” Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

27. Data processing Input files are passed directly into the cloud Further input files can be uploaded. Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

28. Data processing The steps so far have been building an environment to run a processing script Either write your own... ...or build from existing templates Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

29. Managing results - provenance All of a job’s outputs are also accessible Each job has a lifecycle that can be managed Successful jobs can have their entire process captured in a ISO 19115 ‘provenance record’ Presentation title | Presenter name

30. VGL is Power by: VGL acknowledges funding from the NeCTAR project http://www.nectar.org.au NeCTAR is an Australian Government project conducted as part of the Super Science initiative and financed by the Education Investment Fund. VGL is a collaboration between: VGL is supported by: University of Queensland, Monash University and the Australian National University VGL Launch Event is sponsored by: http://vgl.auscope.org