The AstroGrid-D Information Service Stellaris

1. The AstroGrid-D Information Service Stellaris A central grid component to store, manage and transform metadata - and connect to the VO!

2. Overview 1. Motivation, or: How and where to connect Grid and VO 2. Stellaris – an Introduction 3. Stellaris Example: Robotic Telescopes 4. VO Applications Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

3. 1. Motivation What part of the Grid-VO connection are we talking about? Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

4. Focus • Many different ways to combine VO and a Grid • Some are more likely than others • This talk will not deal with: ”Use Aladin to retrieve and visualise the intermediate results of my grid job pipeline” (data sharing and VO protocols) • It can touch: “On the grid I want to produce (or retrieve) data and publish it via a VO compatible web interface/service, when the user demands so.” (Grid augmented VO publication) • Where can we place the connection elements conceptionally? Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

5. some other p http Web Frontend (VO compatible) other VO Service Metadata Database Interpreter Application Grid Compute Resources Grid Storage Resources Grid-VO Connection Principles The Internet Stellaris Semantic Web Services Grid Interface Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

6. Typical VO DB Requirements • A central database that stores and manages scientific metadata • Interface to query and the metadata and, possibly, transform it • Complies with VO standards • Can connect to a web interface or semantic web services (or maybe included in parts) • (Scalable and able to handle large or distributed sets of data) • Can be easily accessed from grid resources, knows grid technology Common solution: SomeSQL and Tomcat Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

7. 2. Introduction Stellaris in a Nutshell Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

8. Basic Features • Stellaris is a database application • It consists of a (http-) interface to create, retrieve, update, delete and query metadata. • It uses Virtuoso, Sesame and python RDFlib. • Data is represented in RDF format • Queries are performed in SPARQL • It offers a separate command line client • Fine grained access control through grid security mechanisms and additional means (X.509) • Prototype (V0.2.0), developed by Mikael Högqvist Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

9. The Semantic Web RDF • “Resource description framework”, a W3C recommendation (=standard). • RDF is a “semistructured” description of data (examples below) • Stellaris uses the RDF/XML syntax • Stellaris can use a general xslt to transform XML into RDF (developed by Frank Breitling) • Numerous tools and interfaces available from the semantic web community SPARQL • “SPARQL Protocol and RDF Query Language”, used to search within the RDF tuples. • Results returned in XML or JSON Details are beyond the scope of this talk… Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

10. Interoperability • Since Stellaris works on a sufficiently abstract layer (i. e. is not bound to specific grid standards) it can serve as an exchange point between different middlewares. • It is flexible enough to adopt existing and future schemes. • Independend of AstroGrid-D, Stellaris development continues also as a D-Grid project (D-Grid middlewares are GT4, Unicore and gLite) • There is a similar development in EGEE (GDSE, …). Exchange? Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

11. 3. Stellaris Example Robotic Telescopes as information providers to Stellaris Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

12. Robotic Telescopes • For this use case, Stellaris adopts a custom XML schema, the Robotic Telescope Markup Language (RTML) • Static metadata: Location, filters, instrumentation • Dynamic metadata: Observation, Weather, Data • Stores original XML and transforms it to RDF/XML Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

13. RDF @prefix file: <http://www.gac-grid.de/schema/files#> . @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . @prefix rtml: <http://www.rtml.org/v3.1a#> <http://storage.gac-grid.de/test/eaglenebula.fits> rdf:type <file:DataObject>; file:owner “Strassmeier, K.”; file:location <http://stella.aip.de/data/20080410_07.fits>; file:filesize "259342"; rtml:Telescope <rtml://de.aip.Robotel/STELLA> . Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

14. SPARQL PREFIX file: <http://www.gac-grid.de/schema/files#> . PREFIX rtml: <http://www.rtml.org/v3.1a#> . SELECT ?location ?owner ?telescope FROM <http://stella.aip.de/context/ robotic_telescopes> FROM NAMED <http://stellaris.astrogrid.net/context/files> WHERE { <http://storage.gac-grid.org/test/eaglenebula.fits> file:location ?location; file:owner ?owner; rtml:telescope ?telescope . } Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

15. Static metadata display Telescope Map Demonstration of Stellaris visualising static metadata using the Google maps web service. Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

16. Dynamic metadata display Timeline Demonstration of Stellaris visualising dynamic metadata using the Simile timeline web service. Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

17. 4. VO Applications How we plan to connect the VO to this Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

18. VO implementation • So far, Stellaris is used for monitoring of resource, activity and application metadata. • The next step is to store scientific metadata. • A data detection service will extract metadata and upload it to Stellaris where it will be transformed to RDF (ontology issues?) • The service should be automatic or at least very simple for the user (sufficient metadata assumed…) Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

19. Web front • Once uploaded to Stellaris, data produced on the Grid can be located and monitored (complex queries possible) • A interface will be implemented to translate VO compatible URL-queries to Stellaris (SOAP? APIs?) • Metadata is to be integrated into the existing visualisation methods. • Distinguish between private or public data (include virtual organisations). • Start from a specific use case, research how much can be abstracted to general use. • Other VO protocol applications: VOspace? Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

20. Why Stellaris? / Summary • Stellaris is a powerful tool to deal with metadata in the grid context • Stellaris offers semantic web features (unlike relational databases). • Offers the necessary abstraction layer, but also flexibility. • Stellaris can become a central service for both Grid and VO information. Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008

21. Contributions and Links • Stellaris, Visualisations: Mikael Högqvist (AstroGrid-D / ZIB) • Robotic Telescopes, Visualisations: Frank Breitling (AstroGrid-D / AIP) • General Design: Harry Enke (AstroGrid-D / AIP) GAVO: http://www.g-vo.org AstroGrid-D: http://www.gac-grid.org Running Version of Stellaris: http://is.astrogrid-d.org:25000 Stellaris development: http://stellaris.zib.de Iliya Nickelt - Grid and the Virtual Observatory EuroVO DCA Workshop 2008