The German Astrophysical Virtual Observatory (GAVO)

1. The German Astrophysical Virtual Observatory (GAVO) Knowledge Networking for Astronomy in Germany and abroad Gerard Lemson1,2, Wolfgang Voges1, Joachim Wambsganss2 GAVO team 1 Max-Planck-Institut für extraterrestrische Physik, Garching 2 Astronomisches Rechen Institut/Zentrum für Astronomy der Universität Heidelberg

2. Overview • Knowledge networking for astronomy: the Virtual Observatory • Standardisation: the International Virtual Observatory Alliance (IVOA) • The Virtual Observatory in Germany: GAVO • Theory in the VObs

3. The status quo • Astronomy produce a huge range of valuable data products, • Of sometimes astronomical sizes (soon Petabytes/yr). • From ground-based observatories and satellites • From all-sky surveys, and from large variety of individual targeted observations. • In all wavelengths. • Don’t forget computer simulations • Requires large variety of disciplines/specialisations to produce and analyse. • Data products are relatively quickly made public (~1 yr). • Many already available as online archives, more or less standardised, homogeneous. • In combination promises interesting new science.

4. Some astronomical data products Images Spectra Catalogues Simulations • John Hibbard http://www.cv.nrao.edu/~jhibbard/n4038/n4038.html • NASA/CXC/SAO/G. Fabbiano et al. • Di Matteo, Springel and Hernquist, 2005

6. But • Where is the data? • What do these archives contain? • How can they be accessed? • How do we analyse these (very) large datasets? • How can we combine them? • ... • The Virtual Observatory (VObs) is the proposed answer to these questions.

7. Requires archiving curation online availability description access tools filtering tools analysis tools standardisation Promises improved communication reuse of results validation comparison combination federation collaboration new science The VObs ... → O(16) national VObs projects, organized in the IVOA

8. IVOA

9. Mission statement Facilitate the international coordination and collaboration necessary for the development and deployment of the tools, systems and organizational structures necessary to enable the international utilization of astronomical archives as an integrated and interoperating virtual observatory.

10. IVOA organisation • Since 2002 • Activities divided in working - and interest groups, “tiger teams” • Meet twice a year in interoperability meetings (two weeks from now, Beijing) • Active mailing lists, wiki pages • Standardised standardisation process

11. IVOA activities • Working groups create standards for • publication and discovery (Resource Registry) • (meta-)data description (DM, Semantics) • selection and remote filtering (DAL, VOQL) • formats for transmitted data (VOTable) • (web) services, distributed workflows (GWS) • application interoperability (Applications) • event notification (VOEvent) • interest groups represent special interests • Data curation • Grid • Theory

12. Some results • Resource registry data models + various implementations • VOTable format (XML schema) + many support tools • Data models for • space-time coordinate systems • characterisation of observations • spectra • Simple data access protocols for • source catalogues • 2D images • 1D spectra • In development: • Astronomical data query language (ADQL) • Astronomical web service standards (UWS) • Application messaging • Simulation data models + access protocols

13. The German Astrophysical Virtual Observatory http://www.g-vo.org

14. BMBF funded Partners Astrophysikalisches Institut Potsdam (AIP) Astronomisches Rechen Institut/Zentrum für Astronomy der Universität Heidelberg (ARI) Max-Planck-Institut für extraterrestrische Physik (MPE), Garching Hamburger Sternwarte Asociated partner: Max-Planck-Institut fur Astrophysik (MPA), Garching Activities R&D Prototyping Special attention: Archive publication: ROSAT, RAVE Data mining: cross-matching, classification Grid computing: simulations, distributed cluster finder Theory: virtual telescopes, archiving simulations, IVOA GAVO I: 2002-2005

15. GAVO II: 2006-2008 • BMBF funded • Partners • AIP, MPE, MPA, • Technische Universität Munchen-Informatik • Universität Tübingen • Focus: move to scientifically useful services • Projects • Millennium database (see later) • IVOA representation (theory, VOQL) • Standard services (SIA, SSA,SCS) • Custom services • VObs expertise center at ARI ...

16. VObs expertise center @ ARI • IVOA compatible metadata repository for community • Implementation data access query protocols • Storage (smaller) data sets, especially science ready data • Tools • Outreach/PR • Help-desk

17. Theory in the VObs: some observations • Simulations not as simple as observations • less homogeneous • complex observables • no standardisation on data formats • archiving ad hoc, for local use • Current IVOA standards somewhat irrelevant • no common sky • no common objects • requires data models for content, physics, code • Moore’s law for N-body) simulations • Very large simulations possible • NB: also makes useful lifetime relatively short

18. “Moore’s law” for N-body simulations Courtesy Simon White

19. Virgo collaboration’sMillennium database • Largest cosmological simulation to date • 10 billion particles evolving under gravity • 500 Mpc (~2Gly) box • 64 snapshots • 350000 CPU hours • O(30Tb) raw data • Derived data • density fields • clusters, merger trees • galaxies, merger trees • realistic, “observed” galaxy catalogues Courtesy Volker Springel

20. Time evolution: merger trees

21. Real and Mock catalogues Courtesy Volker Springel

22. Database + web server • Derived data products only • SQLServer database • Apache web server • portal: http://www.mpa-garching.mpg.de/millennium/ • public DB access: http://www.g-vo.org/Millennium • private access: http://www.g-vo.org/MyMillennium • Access methods • browser producing various formats, plotting capabilities • stream based wget + IDL, R, etc allows • finite query time (30sec-7min) • Features • efficient tree storage+access • spatial indexing • MyDB

25. Usage statistics • Up since Aug 2006 • Community notified via preprint server http://xxx.lanl.gov/abs/astro-ph/0608019 • 130 registered scientific users • >1.4 million individual SQL queries • > 4 billion returned rows (since March 8 2007)

26. Summary • VObs is natural extension of astronomy’s history of data archiving, standardisation, online and open access. • IVOA active, questions are complex technically, but politics are sometimes even harder. • GAVO relatively small, but has found some niches, particularly theory. • To be successful requires use by non-VO scientists, requires proper PR.

27. Thank you. Further thanks to: Volker Springel, Simon White, Gabriella DeLucia, Jeremy Blaizot, Manfred Kitzbichler (MPA, Garching), Carlos Frenk, John Helly , Richard Bower (ICC, Durham, UK), Alex Szalay (JHU, Baltimore) Opening picture courtesy of NASA Goddard Space Flight Center.