VO Data Access Layer

1. VO Data Access Layer IVOA Cambridge, UK 12 May 2003 Doug Tody, NRAO

2. Data access/analysis portal • Link between client (user) data analysis software and the VO • Enables distributed multiwavelength data access and analysis • Key longer-term goals: • Integrate everyday data analysis with VO • Achieve a scalable VO architecture (same software used in multiple contexts)

3. DAL Scope: Software • Service data access protocols • Catalog, image, spectrum, etc. • Service frameworks • Virtual data generation and manipulation, grid computing • Client side integration • API, Web Services client, components, etc. • Test cases and demonstrations Denotes first year priority

4. DAL Scope: Types of data • Source catalog • Image • Spectrum, SED • Time series • Event and visibility data • Key challenges • DAL depends upon registry, query, UCD, DM, representation, etc. • Keep it simple; lower the bar for service implementors

5. Goals for this meeting • Agree on DAL scope, WG scope • this will be our kick-off meeting • Strawman data service architecture • what services are required? (e.g., extend image model, add spectra) • Define roadmap for next year • drivers, priorities, goals of the partners • define Y1 standards development efforts • test cases and demos • Example • SIA V1.1+V2, SSA V1, better RG/DM/UCD/QL integration, web services, test cases

7. DAL Agenda

8. Goals for this meeting • Agree on DAL scope, WG scope • this will be our kick-off meeting • Strawman data service architecture • what services are required? (e.g., extend image model, add spectra) • Define roadmap for next year • drivers, priorities, goals of the partners • define Y1 standards development efforts • test cases and demos • Example • SIA V1.1+V2, SSA V1, better RG/DM/UCD/QL integration, web services, test cases

9. Existing DAL Prototypes • Cone search, SIA (NVO) • Jodrell Bank – imaging online from Merlin data (Anita Richards) • CVO – WFPC assoc, ROSAT, 2QZ (spectroscopic survey) – generic DA service for all data (query is separate); different service instances, obey same interface • AUS/VO – HIPASS, ATCA archive, MACHO, SUMSS, 2DF GRSS – Web query i/f

10. Existing DAL Prototypes • AVO • SIA, IDHA (all images) • AstroGrid • HDX (Starlink) – mainly data model, not much data access yet • Also solar data, space physics data

11. DAL Portal Concept • Link between client (user) data analysis software and the VO • Enables distributed multiwavelength data access and analysis • Key longer-term goals: • Integrate everyday data analysis with VO • Achieve a scalable VO architecture (same software used in multiple contexts)

12. DAL Scope: Software • Service data access protocols • Catalog, image, spectrum, etc. • Service frameworks • Virtual data generation and manipulation, grid computing • Client side integration • API, Web Services client, components, etc. • Test cases and demonstrations Denotes first year priority

13. DAL Scope: Types of data • Dataset (add later) • Query for any type of data in a given ROI • Use other data access services to drill-down to actual data • Source catalog • May need simplified catalog query in DAL portal • More sophisticated catalog operations elsewhere in VO • Image • 2D sky images, spectral data cubes, long slit spectra • Access to raw data via VO framework • Data model issues, representation (FITS, Graphic, XML) • Sparsely sampled images including IFU data • Spectrum, SED • Time series (later?) • Event and visibility data (later?) • Key challenges • DAL depends upon registry, query, UCD, DM, representation, etc. • Keep it simple; lower the bar for service implementors

14. DAL Architecture • Data Access Services • Multiple services • One service for each view of the data • At a high level may not need to differentiate data • Integration of common elements of services • Can sometimes view the same data via different services • E.g., and event list or visibility dataset viewed as an image • Simple services for most common cases

15. Application DAL Client Adapter Grid Service Analysis Component Data Access Framework Data Mediator Component VO Data Access Layer Architecture Service Protocols Client Portal DAL Service Simple Server

16. DAL Services • Key common elements • Registry (caches resource and service data) • Query syntax and capabilities • Metadata • Data models • Data representation (e.g., VOTable)

17. First cut at priorities for Y1 • SIA V1.1 • Same as now with minor enhancements • Simple spectral access • Emphasize 1D and SED • SIA V2.0 • General image model (spectral data cubes etc.) • Explore ways to structure metadata (table vs hierarchical) • Better integration with VO standards • Query standards, registries, MD(UCDs), DM, data representations • First steps for event, visibility data • E.g., metadata standards to publish raw data

18. Spectral Data Access • Goals / Use Cases • discover, retrieve, display spectra for some object • discover, retrieve, display SED for some object • 1D science spectrum for analysis, e.g., classification • multi-wavelength spectral combination • Generate spectra on the fly from grism/radio/HE data • ??

19. Spectral Data Access • Priorities • 1D science spectra • SEDs • spectral image cubes (2 spatial + 1 energy) • 2D science spectra (slit spectra) • ??

20. Spectral Data Access • Key Issues • general 3/4D image model vs simple 1D/SED • two services versus 1 • FITS vs XML (VOTable) • both should be available for 1D spectra, SEDs (also graphic) • Would need a new FITS standard for 1D spectra to use FITS here • ??

21. SIA Evolution • Interface concepts • roadmap: V1.0, 1.1; V2.0 • separation of query/discovery and access • types of services: atlas, pointed, cutout, mosaic • image generation parameters: query ROI supplies default • full specification of IGP possible for mosaic services

22. SIA Evolution • Feedback on SIA V1.0 • Aladin/CDS integration (Francois)

23. SIA – Proposed Enhancements • Registry integration • Image Attributes • Image provenance and identification - 1 • Spectral bandpass (already present) - 1 • Spatial bandpass - 3 • Resolution - 1 • Limiting flux - 2 • Image type (future- v2) • Time of observation (1) • Default for case where there are multiple versions of same dataset • Image Query • Use of image attributes to refine query (e.g., band) • POS, SIZE generalization? Coordinate frameworks

24. SIA – Proposed Enhancements • Others • Support image compression in access protocol

25. Second cut at priorities for Y1 • SIA V1.1 • Same as now with minor enhancements • Simple spectral access • Emphasize 1D and SED • SIA V2.0 • General image model (spectral data cubes etc.) • Explore ways to structure metadata (table vs hierarchical) • Better integration with VO standards • Query standards, registries, MD(UCDs), DM, data representations • First steps for event, visibility data • E.g., metadata standards to publish raw data • Web services versions of DAL services

26. Drivers/Priorities • IAU demo July 03 • Add registry support • Add Aladin, other SIA services • Integrate SIA registry into clients (Aladin, DIS, etc.) • AAS, AVO demos January 04 • Web services versions of services?