an International Virtual Observatory data exchange format

an International Virtual Observatorydata exchange format Roy Williams François Ochsenbein Clive Davenhall Daniel Durand Pierre Fernique David Giaretta Robert Hanisch Tom McGlynn Alex Szalay Andreas Wicenec VOTable

XML: Structured Information <From>Antonio Stadivarius</From> <To>Domenico Scarlatti</To> <Date> <Day>13</Day> <Month>4</Month> <Year>1723</Year> </Date> <Body> Io bisogno una appartamento acoglienti a Cremona … </Body> Separation of structure from presentation 4/13/23 April 13, 1723 17.iv.1723 The computer can read the document: “Find all memos from April 1723”

VOTable • Full metadata representation • Hierarchy of RESOURCEs • containing PARAMs and TABLEs • UCD (unified content descriptor) • a has unit meter • a has UCDORBIT_SIZE_SMAJ (Semi-major axis of the orbit ) • Can reference remote and/or binary streams • Table can be • Pure XML • "Simple Binary" • FITS Binary Table

VOTable Parentage • Astrores • XML format for tables • Developed at CDS Strasbourg • Presented at ADASS 1999 • Vizier implementation • XSIL • XML format for Tables and Arrays • Developed at LIGO Caltech 2000 • Extensible through Type-Class dynamic loading • Java parsing, browsing, editing • Matlab interface

<DATA> <FITS> <STREAMhref="ftp://server.com/mydata.fits" expires="2002-02-22"actuate="onRequest"/> </FITS> </DATA> Sample VOTable <?xml version="1.0"?> <!DOCTYPE VOTABLE SYSTEM "http://us-vo.org/xml/VOTable.dtd"> <VOTABLE version="1.0"> <DEFINITIONS> <COOSYS ID="myJ2000" equinox="2000." epoch="2000." system="eq_FK5"/> </DEFINITIONS> <RESOURCE> <PARAM name="Observer" datatype="char" arraysize="*" value="William Herschel"> <DESCRIPTION>This parameter is designed to store the observer's name </DESCRIPTION> </PARAM> <TABLE name="Stars"> <DESCRIPTION>Some bright stars</DESCRIPTION> <FIELD name="Star-Name" ucd="ID_MAIN" datatype="char" arraysize="10"/> <FIELD name="RA" ucd="POS_EQ_RA" ref="myJ2000" unit="deg" datatype="float" precision="F3" width="7"/> <FIELD name="Dec" ucd="POS_EQ_DEC" ref="myJ2000" unit="deg" datatype="float" precision="F3" width="7"/> <FIELD name="Counts" ucd="NUMBER" datatype="int" arraysize="2x3x*"/> <DATA> <TABLEDATA> <TR> <TD>Procyon</TD><TD>114.827</TD><TD>5.227</TD> <TD>4 5 3 4 3 2 1 2 3 3 5 6</TD> </TR> <TR> <TD>Vega</TD><TD>279.234</TD> <TD>38.782</TD><TD>8 7 8 6 8 6</TD> </TR> </TABLEDATA> </DATA> </TABLE> </RESOURCE> </VOTABLE>

etc Table Cell boolean bit unsignedByte short int long char unicodeChar float double floatComplex doubleComplex scalar Primitives arrays variable length arrays etc

VOTable is Flexy • eg Table of images • UCD="JPEG_IMAGE" datatype="unsignedByte" arraysize="*" • eg Table of URL links • UCD=“DATA_LINK"datatype="char" arraysize="*"

VOTable Schema (xsd)

Table Data Model • Metadata • Class definition for Row • FIELD • data type • semantic type • Data • Each Row is a list of Cells • Each Cell is an array of Primitives • may be variable length

Table Data Layout • All metadata first • small, complex, XML • Class definition for table record • + params, description, etc etc • Then data • (may be) large, remote • XML | binary | FITS • Instantiations of table record • All records MUST have same format

Param Data Model • Param is “Table with one cell” • Like a FIELD value • But with a “value” attribute

Primitives • All have fixed binary length • Same as FITS primitives • Except Unicode

Multidimensional Array Cell • A table cell can have lots of Primitives • Example: WCS parameters are arrays • <FIELD name=“CRVAL” datatype=“double” arraysize=“2”/> • Example: up to 10 images, each 64x64 • <FIELD name="thumbs" datatype="unsignedByte" arraysize="64x64x10*"/>

Hierarchy • A VOTable contains RESOURCES • RESOURCE can contain: • TABLE • RESOURCE • etc etc • Usage example • Many observations in the file, • each is a RESOURCE • Each observation is • Parameters • Calibration table • Raw data table

Hierarchy • New feature: GROUP <TABLE name=“Nutation and Aberration”> <GROUP name=“Nutation”> <FIELD name=“Longitude”/> <FIELD name=“Obliquity”/> </GROUP> <GROUP name=“Aberration”> <GROUP name=“Equinox 1950.0”> <FIELD name=“C”/> <FIELD name=“D”/> </GROUP> <GROUP name=“Equinox 1955.0”> <FIELD name=“C”/> <FIELD name=“D”/> </GROUP> </GROUP> </TABLE>

Unified Content Descriptors • UCD is a “semantic type” PHOT.INT-MAG.B Integrated total blue magnitude ORBIT.ECCENTRICITY Orbital eccentricity STAT.MEDIAN Statistics Median Value INST.QE Detector's Quantum Efficiency • Can be resolved by web service • to description, examples, etc • Base + Specifiers • eg error in default right ascension • POS.EQ.RA, MAIN, ERROR

VOTable Friends Some “self-describing” file formats

XML Parsing SAX: Event-Based Handlers for StartElement, Text, EndElement, etc. Found elementBookCatalogue Found elementBook Found ElementTitle Found TextThe Cambridge Star Atlas Found End ElementTitle ….

Parsing DOM: Document Object Model Returns a tree-like Document object with data attached BookCatalogue Book Book Title Title Author Cambridge Star Atlas ISBN Parallel Computing Works! Wil Tirion

Binding to make a Parser From the Schema an API and library is generated JAXB Breeze Castor This is JAVOT (Caltech) for(int i=0; i<table.getFieldCount(); i++){ Field field = (Field)table.getFieldAt(i); String u = field.getUcd(); if(u != null && u.equals("POS_EQ_RA_MAIN")) System.out.println("Field " + i + " is for RA"); }

VOTable Software Treeview from UK-VO

VOTable Software VOPlot from India-VO

VOTable Software VOTool from US-VO

VOTable software Mirage from Bell Labs

an International Virtual Observatory data exchange format