1 / 18

eSDO Algorithms, Visualization and Data Access

eSDO Algorithms, Visualization and Data Access. Elizabeth Auden 27 September 2006 HELAS Workshop Nice. What is eSDO?. eSDO is an e-science project funded by PPARC to make SDO data and algorithms available to the UK solar community using the virtual observatory.

avel
Download Presentation

eSDO Algorithms, Visualization and Data Access

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. eSDO Algorithms, Visualization and Data Access Elizabeth Auden 27 September 2006 HELAS Workshop Nice

  2. What is eSDO? eSDO is an e-science project funded by PPARC to make SDO data and algorithms available to the UK solar community using the virtual observatory. eSDO Duration: 36 months, October 2004 – October 2007 SDO Launch: October 2008 RESEARCH - Phase A: 1 October 2004 – 30 September 2005 IMPLEMENTATION - Phase B: 1 October 2005 – 30 September 2007 Current eSDO Project Documentation: http://www.mssl.ucl.ac.uk/twiki/bin/view/SDO

  3. eSDO Workpackages I. Algorithms • Code: feature recognition, local helioseismology, global helioseismology • Grid integration: AstroGrid and JSOC II. Data centres • AstroGrid interface with US JSOC data centre • AstroGrid interface with UK (or European?) data centre III. Visualization tools • Catalogues • SDO Streaming tool • On-the-fly image galleries and movies

  4. eSDO Project Team • MSSL: Coronal Image Analysis • Elizabeth Auden, project manager • Len Culhane, PI • Lidia van Driel-Gesztelyi • Mike Smith • RAL: Event and Feature Recognition • Richard Harrison • Andrzej Fludra • Vito Graffagnino • Sheffield: Local Helioseismology • Mike Thompson • Sergei Zharkov • Birmingham: Global Helioseismology • Yvonne Elsworth • Bill Chaplin • Thierry Toutain US Links: JSOC, Joint Operations Centre Stanford University Lockheed Martin

  5. AlgorithmsCode Feature Recognition / Event Analysis • Coronal loop detection, MSSL • Non-linear magnetic field extrapolation, MSSL • Helicity computation, MSSL • Small event detection, RAL • CME dimming region recognition, RAL Local Helioseismology • Perturbation map generation, Sheffield • Subsurface flow analysis, Sheffield • Local helioseismology inversions, Sheffield Global Helioseismology • Mode parameters analysis, Birmingham • Mode asymmetry analysis, Birmingham Solar image from TRACE mission. Coronal loops marked in blue.

  6. AlgorithmsGrid Integration JSOC • SDO pipeline at Stanford University / Lockheed Martin • eSDO algorithms deployed as pipeline modules (automated or user-run) • Data will be obtained locally from JSOC archive AstroGrid CEA • UK hosted algorithms hosted as web services • Accessible through AstroGrid workflows • Data transferred from MySpace, remote data centre or user’s local storage SolarSoft • Algorithm C modules wrapped in IDL • Distribution through MSSL SolarSoft gateway • Data transferred to algorithm from user’s local storage

  7. AlgorithmsAstroGrid Deployment 9 algorithms have been deployed as CEA applications

  8. Data CentresUS Data Centre Primary US Data Store • Files stored in SUMS (Storage Unit Management System) at Stanford University with metadata in distributed DRMS (Data Resource Management System) • DSA / DRMS integration for grid data access investigations • VSO (Virtual Solar Observatory) primary US access AstroGrid / VSO Interface • VSO queries in AstroGrid workflows through STAP (Simple Time Access Protocol) web service US / UK Network Latency Tests • Test expediency of file transfer mechanisms • Test between Stanford, MSSL, UCL, RAL

  9. Data CentresPossible UK Data Centre Storage • 30 TB disc cache • 60 day rolling cache HMI/AIA • True cache for HMI / AIA • DRMS / SUMS Transfer • Export formats from JSOC • Data search – DSA • Data transfer - CEA • Test installation MSSL Investigation of DRMS / SUMS

  10. Data CentresPossible European Data Centre TLAs: DRMS: Data Resource Management System (JSOC) SUMS: Storage Unit Management System (JSOC) DSA: DataSet Access (AstroGrid) • EU Framework 6 support for European helioseismology data and algorithms • Possible full mirror of SDO data in Europe? • DSA / DRMS interface for AstroGrid access to full mirror of SDO data in Europe? DRMS DRMS DRMS SUMS SUMS US Europe DSA UK UK datasets

  11. Data CentreseSDO / Solar-B Prototype: system perspective (Local) XML AstroGrid HelioScope (Local) STAP service URLs UKSSDCADS file system Note: AstroGrid components (JES, DSA, MySpace, Workbench) may be located anywhere.

  12. Data CentresUser access through AstroGrid AstroGrid workbench: STAP service access through HelioScope, DSA access through the Task Launcher. Task Launcher: locate DataSet Access components, submit ADQL queries STAP service will point to URLs in UK, US or Germany. DSA will integrated with DRMS (Data Resource Management System) in UK or Germany. HelioScope: solar data results from Simple Time Access Protocol services.

  13. Visualization ToolsCatalogues Small Event / CME Dimming catalogue • Generated from continuously running of small event detection / CME dimming region recognition algorithms • Tables of event / CME statistical information • Likely to be generated in UK Helioseismology Mode Parameters catalogue • GONG month of data generated from mode parameters analysis algorithm • Likely to be generated in US Catalogues can be accessed through ADQL (Astronomical Data Query Language) query to DSA.

  14. Visualization ToolsSDO streaming tool User Facilities • Pan in space P • Zoom in space P • Pan in time • Zoom in time Data Products • LoS Magnetogram • HMI continuum map • AIA images Future Work • FFT / wavelet analysis for coronal wave investigations

  15. Visualization ToolsOn-the-fly Image Galleries and Movies Thumbnail Extraction P • Extract thumbnails in US from HMI and AIA science products Image Gallery Generation P • User specifies start time, stop time, cadence, data product • Image gallery generated on the fly in web browser Movie Generation • User specifies start time, stop time, cadence, data product • MPEG movie generated on the fly Find data with AstroGrid HelioScope. Use PLASTIC to load data into visualization tools.

  16. Int’l Virtual Observatory Alliance Defines standards for VO schemas and coordinates interoperable software between virtual observatories VOTech: EU Framework 6 funded Technology / infrastructure for European virtual observatory UK-funded e-science project Provide access to SDO data, algorithms and visualization tools through AstroGrid. UK-funded Virtual Observatory Infrastructure for the UK: data access, applications, virtual storage, workflows, computational resources US-funded Virtual Observatory Infrastructure to search, obtain and analyze solar data eSDO and the Virtual Observatory eSDO has links with AstroGrid, VOTech, VSO, and the IVOA.

  17. Future Work Phase B implementation development now in progress: • Summer / Autumn 2006: • Algorithm integration with AstroGrid; test integration with JSOC • Unit and science testing for algorithms • Add STAP service client to visualization tools • Add movie functionality to streaming tool • Investigate DRMS integration with DSA • 2007: • Completed algorithms • Full algorithm integration with JSOC and SolarSoft • Coronal wave analysis in SDO streaming tool • Work with HELAS to explore European SDO data centre • Catalogues

  18. Further Information Links eSDO: http://www.mssl.ucl.ac.uk/twiki/bin/view/SDO Astrogrid:http://www.astrogrid.org VOTech: http://wiki.eurovotech.org/bin/view/VOTech/ IVOA: http://www.ivoa.net Software to try AstroGrid Workbench: http://www2.astrogrid.org/desktop/index HelioScope: included with workbench eSDO Streaming Tool: http://www.mssl.ucl.ac.uk/twiki/bin/view/SDO/StreamingTool Contacts eSDO: Elizabeth Auden, eca@mssl.ucl.ac.uk Astrogrid Solar Science: Silvia Dalla, s.dalla@manchester.ac.uk

More Related