1 / 14

What does LOFAR have to do with the Virtual Observatory (VO)?

What does LOFAR have to do with the Virtual Observatory (VO)?. LOFAR Science Day 16 December 2003 Melbourne David Barnes The University of Melbourne. VO is Standards (1). data description metadata, content tags, context tags IVOA: VOTable, UCD WGs data retrieval

sydney-payne
Download Presentation

What does LOFAR have to do with the Virtual Observatory (VO)?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What does LOFAR have to do with the Virtual Observatory (VO)? LOFAR Science Day 16 December 2003 Melbourne David Barnes The University of Melbourne

  2. VO is Standards (1) • data description • metadata, content tags, context tags • IVOA: VOTable, UCD WGs • data retrieval • http, ftp, web services, replica catalogs • IVOA: Data Access Layer, Grid & WS WGs • data models • structure and relationships in data • IVOA: Data Modelling WG

  3. VO is Standards (2) • data and service compilation and indexing • registries, resource description • IVOA: Registry WG • finding data and services • VO Query Language, SQL, brokers, agents • IVOA: VO Query Language WG • workflow and interface • portals, portlets, authentication, authorisation • IVOA: no WG yet

  4. VO is Grid Computing • local, national, global compute grids • transparent, secure access to processors • job submission & monitoring, workflows • local, national, global data grids • replica catalogues (caching) • global filesystems • local, national, global networks • move code to data or data to code • realtime client-server processing, analysis and visualisation

  5. VO Utopia (1) • diverse data (eg. X-ray photon event lists and SPH cosmo simulation) • found in standard VO registries • described using standard vocabulary • tagged with pointers into a standard astronomy data model • accessed via standard mechanism eg. SOAP query to Web Service • provided in standard storage format, eg. VOTable, HDF, FITS, … • data sets are interoperable

  6. VO Utopia (2) • diverse tools (eg. source extractor, volume visualiser, PCA algorithm) • found in standard VO registries • described using standard vocabulary • tagged with pointers into a standard astronomy data model • accessed via standard mechanism eg. SOAP query to Web Service • read and writestandard storage format, eg. VOTable, HDF, FITS, … • tools are interoperable • tools AND data are interoperable

  7. VO Utopia (3): LOFAR • naif: registry search : LOFAR exists! : free data is available! • traditionalist: query LOFAR archive via VO, download images and analyse at home • aspirant: use on-line VO LOFAR pipeline to recalibrate intermediate data product/s and re-image with modified visibility weights • braveheart: incorporate any or all LOFAR data products in VO workflows exploiting several other data sets at once

  8. VO Utopia (4): LOFAR • Grid: this data is popular in France : replicate data product @ CDS : provide transparently to UK users • Grid: this data is owned by David and in propietary period : you can’t have it yet • Grid: you have requested 4 TB of data : click here to upload and run your code at the data centre rather than fetching for days. • Telescope: “I am just a long-queue data archive” : search for data returns null result : would you like to queue an observing request based on your search metadata?

  9. LOFAR data products will • be large … accurate and standardised metadata will reduce unnecessary downloads and processing • be heavily qualified … pointers into a global data model will help software and astronomers make valid use of the data • LOFAR should make a substantial contribution to data modeling in the VO • experience widely varying usage patterns … VO and Grid will provide intelligent caching and data distribution mechanisms

  10. adopting VO buys: • functionality: new and emerging VO tools and services, plus “wrapped” legacy software, immediately available • interoperability: all VO tools and services work with and understand LOFAR data to the best of their ability • archive & data centre: “reference” implementations of standards will provide “data centre in a box” software • wider user community

  11. Reality… • VO standards are moving targets • VOTable stores metadata and small catalogues; most “real” data still stored in legacy formats (FITS, HDF, …) • Tools implementing or supporting VO standards are rare! • Portals, workflows, embracing the Grid paradigm, …, are all in the future

  12. end ( unless you care about LOFAR data rates … )

  13. LOFAR data:baseband input & output • total baseband data rate is ~20 Terabits/s • 13000 antennae, dual pol • 64 Msamples/s • 12-bit sampling • total station data rate is ~150 Gigabits/s • 100 stations • 16 beams • 4 Msamples/s (2 MHz bandwidth) • 6-bit sampling

  14. LOFAR data:correlator input & output • central correlator input rate is ~150 Gigabit/s • output rate is ~10 GB per cycle • 5000 baselines (100 stations) • 2000 channels (2 MHz bandwidth, 1kHz resolution) • 16 beams • 4 polarisation products • 32-bit IEEE floats • ~36 TB/hr for 1s samples • ~3.6 TB/hr for 10s samples

More Related