1 / 34

Madrigal A Virtual Observatory for AMISR Science

Madrigal A Virtual Observatory for AMISR Science. John Holt and Bill Rideout MIT Haystack Observatory AMISR Science Workshop, 2006. What is Madrigal?. A collection of standards-based local databases that share metadata:. What is the local Madrigal database?. User. Web services API

casey
Download Presentation

Madrigal A Virtual Observatory for AMISR Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MadrigalA Virtual ObservatoryforAMISR Science John Holt and Bill Rideout MIT Haystack Observatory AMISR Science Workshop, 2006

  2. What is Madrigal? A collection of standards-based local databases that share metadata:

  3. What is the local Madrigal database? User • Web services API • Python API • Matlab API • Scripts • From anywhere on internet Web interface Database standard – Cedar file format Metadata standard – Madrigal standard Madrigal derivation engine Real-time and historical data

  4. How is Madrigal a Virtual Observatory? • Searches from local Madrigal site can link to any other site User chooses whether the search is local or includes all Madrigal sites Implemented via shared, standard metadata

  5. In what other way is Madrigal database a virtual observatory? • Entire database of multiple types of instruments can be searched at once using a complex query • Example: Give me all measurements of Te and Ti on April 3, 2004 between longitudes 280 and 310 and on closed field lines • Queries can be built on any parameter, measured or derived (such as Tsyganenko model)

  6. How is this done? • All registered Madrigal databases share standard metadata • Queries use metadata to filter out experiments that don’t need to be analyzed (fast) • Then the remaining experiment is analyzed at the data level (slower) • How much can be filtered quickly depends on what’s in the metadata

  7. Madrigal can hold data from many types of instruments • ISR • GPS TEC

  8. Madrigal derivation engine adds to VO feel Madrigal data file Spatial and temporal conversions Geophysical data (IMF, Kp, DST. etc) Magnetic field models / MSIS Empirical ISR models Madrigal derivation engine The user sees the file as containing many additional parameters Example: Te from ISR (measured) Te from empirical model (derived)

  9. Underpinnings of Madrigal • Cedar database format • Defines parameters – allows instruments to add parameters unique to their instruments • File format requires start time, end time, instrument • For each measurement time, 1D or 2D data • New parameters can be added to format by working with NCAR • Madrigal metadata • Only instrument and time range – no spatial or parameter information

  10. Strengths of Cedar database format • Unambiguous time in the prolog • Use of standard parameters allows the derivation of many other parameters not in the data file - *very important* • Human intervention not needed to interpret data • Madrigal’s core is a list of methods in a standard form with input parameters and output parameters, and an engine that chooses which methods to call • Easily extensible by adding new methods • Example – less than 48 hours after request, Tsyganenko field model added to Madrigal • Allows model, geophysical data to appear to be in every data file • Includes IGRF, MSIS, CGM, Tsyganenko, Kp, Dst, F10.7, IMF, etc. • Makes the normally hard metadata problem easy

  11. How hard is Madrigal to install? • Arecibo experience – Madrigal installed and data imported in < 2 weeks

  12. Summary • A key to the success of AMISR will be prompt, unified and user-friendly access to data from the radar, its supporting instrumentation and other radar facilities • Madrigal is a virtual observatory with numerous desirable features • Open Source – you can contribute • Well-defined metadata standard • Real-time capability • interactive Web interface • Global search capability • Provision for including information such as html pages, publications, figures, movies and images • Interactive plotting • Complete Web-services interface • High-level user friendly tools for entering new datasets • Help in setting up a new Madrigal site or entering new datasets into Madrigal is available

  13. Thank You

  14. Summary of lessons learned • A robust, well-defined data format with time and space information part of the standard makes building virtual observatories far easier than the top down approach • The Cedar database format is a strong platform to build on • The existence of such a standard makes possible an open-source tool such as Madrigal to be easily deployed for any new instrument, saving development time

  15. Two approaches to VO’s VO Custom interfaces Top down: Build custom interfaces to existing databases VO Uniform interfaces Bottom up: Encourage adoption of standard local database software (i.e., Madrigal) Both approaches can be pursued

More Related