Virtual Observatory Interfaces reused in the Virtual Atomic and Molecular Data Centre

1. Virtual Observatory Interfaces reused in the Virtual Atomic and Molecular Data Centre Guy Rixon Institute of Astronomy University of Cambridge September 2009

2. Rixon: VObs interfaces reused in VAMDC VAMDC reusing VObs IT Too complicated? Too simplistic? Virtual Observatory VAMDC Paradigms, Standards, Implementations Start from scratch? WHY?

3. Rixon: VObs interfaces reused in VAMDC Past, present and future Usefulness Time 2009 2010 2011 2012 2013 Era of Separate web-sites VAMDC project ??? IT support for use of A & M data does not start and end with the VAMDC project.

4. Rixon: VObs interfaces reused in VAMDC Goals • Open • Easy to add users (and user communities) • Extensible • Easy to add data and applications • Composable • Easy to combine with other software • Sustainable • Easy (cheap) to maintain • Finished in < 3 years

5. Rixon: VObs interfaces reused in VAMDC Goal conflicts Extensible, composable, cheap: pick any two... “Good Software Takes Ten Years. Get Used To it” – Joel Spolsky

6. Rixon: VObs interfaces reused in VAMDC How does the Virtual Observatory do? • Tries to be open, extensible, composable, etc. • Web services • Standard protocols • Resource discovery via registry • Workflow support via data grids

7. Rixon: VObs interfaces reused in VAMDC Open and Composable GUI Script Workflow Etc. HTTP calls across WAN Vobs service Vobs service Vobs service Vobs service Web (HTTP) services

8. Rixon: VObs interfaces reused in VAMDC IVOA standard interfaces Public Private Extensible: adding data VObs clients Site A Site B Site C

9. Rixon: VObs interfaces reused in VAMDC My personal apps Generic apps & utilities My personal apps Standard Apps & utilities My personal apps Standard Apps & utilities IVOA standard interfaces Public Private Open & extensible: adding applications My community's apps My community's apps My community's apps The IVOA “API library” “The Virtual Observatory”

10. Rixon: VObs interfaces reused in VAMDC Extensible and sustainable: registry • Resources registered in on-line directory • Scientists can discover meaning of resources • Software can discover interface details • No need to update software to use new resources Discover, choose Publish, describe

11. Rixon: VObs interfaces reused in VAMDC What can be registered? Hypothetical extension by VAMDC

12. Rixon: VObs interfaces reused in VAMDC Structure of registration document Resource identity “Dublin core”: meaningful to science users Used to select services. Curation Content synopsis Capability (TAP) Each Capability describes a major interface Lists access URL(s) and support for optional parts of interfaces. Machine readable detail; drives UIs. Capability (VAMDC) Tableset Describes the tables available for query. Machine-readable detail; drives query building UIs.

13. Rixon: VObs interfaces reused in VAMDC Native XML DB New(ish) technology Few implementations XQuery Trivial to add new schemata Relational vs. XML storage Registries have been built on both relational and native-XML databases. • Relational: • Robust, mature • Choice of RDBMS • SQL query • Hard to add new schemata XQuery + schema flexibility wins every time.

14. Rixon: VObs interfaces reused in VAMDC Publishing, harvesting, searching Virtual Observatory VAMDC “Full” registry VAMDC users search here All registrations “harvested” to full registry VAMDC registry VAMDC providers register Services here NVO (STSci) Publishing registry Vizier Publishing registry CASU Publishing registry Etc.

15. Rixon: VObs interfaces reused in VAMDC Extensible and composable: data grid My app. Slower network Service A Service B Service C Fast network VOSpace VOSpace

16. Rixon: VObs interfaces reused in VAMDC ISO PHT HST FOS IUE ISO LWS HST INES HUT EUVE UIs: VOSpec • Multi-wavelength spectral analysis tool developed at ESAC by ESA-VO • Create a SED with spectra from: • a wide range of different providers registered in the VO Registry • local data (FITS, ASCII, VOT) • Many fitting/ plotting functions http://esavo.esac.esa.int/vospec/

17. Rixon: VObs interfaces reused in VAMDC VOSpec cont. • VOSpec provides access to atomic and molecular line Databases (SLAP): • NIST Atomic Spectra • CHIANTI (X-rays) • CIELO (XMM-Newton)‏ • IASD (ISO)‏ • LERMA (molecular)‏ • STSCI (UV) • on-the-fly identification of transitions associated to a given observed feature.

18. Rixon: VObs interfaces reused in VAMDC UIs: Taverna

19. Rixon: VObs interfaces reused in VAMDC UIs: VODesktop

20. Rixon: VObs interfaces reused in VAMDC VODesktop: actions on resources

21. Rixon: VObs interfaces reused in VAMDC VODesktop: query builder

22. Rixon: VObs interfaces reused in VAMDC VODesktop/SAMP: UI integration

23. Rixon: VObs interfaces reused in VAMDC VODesktop: data-grid access

24. Rixon: VObs interfaces reused in VAMDC Looter's checklist • Web-service paradigm • Registry paradigm • Data-grid paradigm • VODesktop • Other UIs/desktop applications (as applicable) • Service standards (as applicable) • Service implementations (as applicable)

25. Rixon: VObs interfaces reused in VAMDC IVOA standard services

26. Rixon: VObs interfaces reused in VAMDC The data-model issue Requests Results Web-service shell Elaborate data-model makes it hard to repurpose the service, especially if domain-specific Data-model For reuse, prefer services with little or no data model Resource adaptor What the client needs

27. Rixon: VObs interfaces reused in VAMDC TAP: Table Access Protocol Requests: SQL Results: VOTable/HTML/CSV/??? Web-service shell TAP data-model is very generic: “relational data” Data-model Resource adaptor: async. queries on JDBC Works with any DB schema Relational database

28. Rixon: VObs interfaces reused in VAMDC TAP: possible use in VAMDC year 1 VAMDC App (desktop) Deal with schemata Variations here TAP One per VAMDC site TAP service Reuse same implementation on each VAMDC site Describe DB schemata in registry Sites provide these DB

29. Rixon: VObs interfaces reused in VAMDC TAP: possible evolution Requests: XQuery Results: XSAMS Web-service shell Data-model Replace the TAP data-model with something VAMDC-specific Resource adaptor: async. queries on JDBC Extend this for VAMDC Works with any DB schema Relational database

30. Rixon: VObs interfaces reused in VAMDC VOSpace • Storage accessible to users and services • Useful in workflows, grid jobs • Can use simple storage or data grid VOSpace VOSpace VOSpace Site A Site B Site C Site C iRODS iRODS iRODS

31. Rixon: VObs interfaces reused in VAMDC SLAP: Simple Line Access Model Requests Results: VOTable Web-service shell Data-model SLAP data-model is very simple: “just enough for astronomy” Resource adaptor Needs a particular schema Relational database

32. Rixon: VObs interfaces reused in VAMDC CEA & grid • CEA = Common Execution Architecture • Describe, register, discover, run server-side apps: grid • “A standard for everything else” • Powerful: • CEA UI runs any registered app • Allows long-running jobs (asynchronous execution) • Can be connected to the Grids, e.g. EGEE • Integrates with VOSpace, Taverna workflow

33. Rixon: VObs interfaces reused in VAMDC If all else fails... • ...design a new service type! • Need to specify: • Service semantics • Web-service protocol (e.g. REST) • Query language (e.g. XQuery) • Representation of results (e.g. XSAMS) • XML schema for registration • Consider following IVOA “DAL” architecture

34. Rixon: VObs interfaces reused in VAMDC Adapting to new service types VODesktop UI Registry parser AstroRuntime SLAP strategy SSAP strategy TAP strategy CEA strategy VAMDC strategy VAMDC strategy

35. Rixon: VObs interfaces reused in VAMDC Summary • VAMDC needs adaptable, future-proof software on the cheap • We think the VObs model will work • We will copy the VObs ideas • We will use their service protocols to start • We will use the ready-made software • We will exploit the flexibility of the system progressively to fit it to VAMDC.