1 / 30

Virtual Geophysics Laboratory (VGL)

Virtual Geophysics Laboratory (VGL). VGL v1.1 Launch Ryan Fraser, Terry Rankine, Joshua Vote, Lesley Wyborn, Ben Evans, Robert Woodcock. February 2013. CSIRO | Minerals down under flagship. Geo-information. bedrock. surficial. mineral. geophysical. geochemical. geochronologic.

calvinu
Download Presentation

Virtual Geophysics Laboratory (VGL)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Virtual Geophysics Laboratory (VGL) VGL v1.1 Launch Ryan Fraser, Terry Rankine, Joshua Vote, Lesley Wyborn, Ben Evans, Robert Woodcock February 2013 CSIRO | Minerals down under flagship

  2. Geo-information bedrock surficial mineral geophysical geochemical geochronologic hyrdrogeological $ Gather data, process it, publish resultsSimple, isn’t it? data knowledge Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  3. Let’s map it out for Geophysics … Hardcopy of data Proprietary Package Maths Package Get handed field data Visualise data Transform to a regular grid Download results Crop data to area of interest Reformat data for processing SSH Client Configure job and start processing Upload data to HPC resource Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  4. The Past • Compile raw data using proprietary FORTRAN • Also use other proprietary software • Transform to a regular grid using more software • MATLAB, Intrepid, ER Mapper, ESRI ArcGIS, QGIS, etc • Crop data spatially • eg: everything in Victoria • Transform data into a file format that can be read by proprietary scientific code • This is usually done with some handwritten python or c • There is no version control, code is often rewritten / redone • Upload data to HPC • Manually enter input parameters/start job • Where did my results go, how do I visualise? Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  5. There seems to be a problem… • Reproducibility – there is none • What was the input of your model? • What transformations occurred? • It’s a manual process • Time consuming • Error prone • Expensive • Licensing costs • Effort/time Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  6. Scientific workflow – Virtual Geophysics Laboratory (VGL) • Automated - Simplify access and integration • Reproducible process • Open access (less expense) • Allow scientists to focus on the science and not the tech • Exploiting the Cloud and HPC • Operational or education tool • Automates and massively expands geophysics computational capacity via the Cloud Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  7. V(what)GL • VEGL – Virtual Exploration Geophysics Laboratory • One primary science collaboration • One primary workflow • One collection of geophysical data sets • VGL - Virtual Geophysics Laboratory • NeCTAR funded activity • Collaboration with multiple partners (CSIRO, NCI, GA, UQ, Monash, ANU) • Supporting multiple workflows • New data types and collections • New Use – Not just exploration. Done Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  8. Let’s map it out… Hardcopy of data Proprietary Package Maths Package Get handed field data Visualise data Transform to a regular grid Download results Crop data to area of interest Reformat data for processing SSH Client Configure job and start processing Upload data to HPC resource Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  9. …to this Virtual Geophysics Laboratory Select spatial bounds Build “science” from existing libraries Discover raw data Run job Collect and publish results Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  10. Introducing The Virtual Geophysics Laboratory Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  11. Powered by the Spatial Information Services Stack Common Platform Environment, Water Marine • Groundwater Geology Geophysics 11 | Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  12. VGL: Opportunities • Free for research use • Supporting multiple workflows • Model Registry (3D) – researcher contributed models • New Scientific Codes – Underworld, eScript, UBC, Airborne EM inversion codes + others (relatively “easy” to add) • Data sets from GA: National Airborne Geophysical DB including • Gravity, Radiometric, AEM, Magnetics • Data sets from State Surveys • Broad application and generic implementation • Scalable Infrastructure (Cloud) Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  13. What’s left? • More data, more codes, more workflows! • BYO Resource (cloud allocation) • Confidential Data • Pawsey Centre, NCI, commercial clouds Virtual Geophysics Laboratory v1.1 | Ryan Fraser

  14. VGL is Power by: Thank you VGL acknowledges funding from the NeCTAR project http://www.nectar.org.au NeCTAR is an Australian Government project conducted as part of the Super Science initiative and financed by the Education Investment Fund. Ryan FraserResearch Manager t +61 8 6436 8760 e ryan.fraser@csiro.au w siss.auscope.org VGL is a collaboration between: VGL is supported by: University of Queensland, Monash University and the Australian National University VGL Launch Event is sponsored by:

  15. Introducing The Virtual Geophysics Laboratory Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  16. No Installation Necessary! Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  17. Don’t Compute on your Laptop Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  18. Flexibility What I want to do ≠ How I want to do it ≠ Where I want to do it Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  19. Provenance Don’t be this guy... Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  20. To Quickly Recap • VGL is a different kind of workflow engine... • No installation necessary • Doesn’t process on your machine • Very flexible • Records results and the process Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  21. Time for a live demo... Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  22. VGL is Power by: VGL acknowledges funding from the NeCTAR project http://www.nectar.org.au NeCTAR is an Australian Government project conducted as part of the Super Science initiative and financed by the Education Investment Fund. VGL is a collaboration between: VGL is supported by: University of Queensland, Monash University and the Australian National University VGL Launch Event is sponsored by: http://vgl.auscope.org

  23. Data discovery Layers discovered via remote registries Layers consist of numerous remote data services Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  24. Data discovery Some data services support subsetting Some data services support reformatting e.g. CSV, NetCDF, GeoTIFF Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  25. Data discovery Some data is only registered with flat files Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  26. Data processing A variety of different scientific codes are already available in the form of “Toolboxes” Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  27. Data processing Input files are passed directly into the cloud Further input files can be uploaded. Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  28. Data processing The steps so far have been building an environment to run a processing script Either write your own... ...or build from existing templates Virtual Geophysics Laboratory | Ryan Fraser / Josh Vote

  29. Managing results - provenance All of a job’s outputs are also accessible Each job has a lifecycle that can be managed Successful jobs can have their entire process captured in a ISO 19115 ‘provenance record’ Presentation title | Presenter name

  30. VGL is Power by: VGL acknowledges funding from the NeCTAR project http://www.nectar.org.au NeCTAR is an Australian Government project conducted as part of the Super Science initiative and financed by the Education Investment Fund. VGL is a collaboration between: VGL is supported by: University of Queensland, Monash University and the Australian National University VGL Launch Event is sponsored by: http://vgl.auscope.org

More Related