1 / 10

DAL Clients: Scripting Data Access with Python

T HE US N ATIONAL V IRTUAL O BSERVATORY. DAL Clients: Scripting Data Access with Python. Ray Plante. The Data Access Layer: from data discovery to data retrieval. Standard interfaces for finding and retrieving particular types of data Simple Cone Search (object catalog data)

ianthe
Download Presentation

DAL Clients: Scripting Data Access with Python

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. THE US NATIONAL VIRTUAL OBSERVATORY DAL Clients:Scripting Data Access with Python Ray Plante NVO Summer School 2008 – Santa Fe

  2. The Data Access Layer: from data discovery to data retrieval • Standard interfaces for finding and retrieving particular types of data • Simple Cone Search (object catalog data) • Simple Image Access Protocol (SIAP) • Simple Spectral Access Protocol (SSAP) • Table Access Protocol (TAP)* • Simple Line Access Protocol (SLAP)* *not covered here • Common mechanisms accessing data • Search query returns matching items • “Simple”: Query expressed as a URL (a base URL + name=value arguments) • Result returned as a VOTable • SIAP/SSAP: record includes URL for downloading data item • Bridge from discovery to retrieval • Differences reflect maturing interface designs • “2nd generation” will have more unified behavior NVO Summer School 2008 – Santa Fe

  3. Simple Cone Search (SCS) • Retrieve a table of objects located within a given distance from a sky position http://www.ivoa.net/Documents/latest/ConeSearch.html • Ultra-simple interface: http://<base_url>RA=<deg>&DEC=<deg>&SR=<deg> • RA – right ascension of central search position in degrees • DEC – declination of central search position in degrees • SR – search radius in degrees • Optional argument: VERB=<num> • 1=minimum # of columns, 3=maximum # Examples: • Just want the table metadata? • Use SR=0 • Service will return VOTable header with no rows http://archive.stsci.edu/befs/search.php?RA=75.5&DEC=-5.4&SR=1.0 http://www.nofs.navy.mil/cgi-bin/vo_cone.cgi?CAT=USNO-B1&RA=75.5&DEC=-5.4&SR=1.0 NVO Summer School 2008 – Santa Fe

  4. Simple Cone Search (SCS) • VOTable results • Only one RESOURCE element with one TABLE • Each record represents an object found within the cone • Must contain columns with the following UCDs*: • ID_MAIN – a name for the object • RA_POS_EQ_MAIN – the object’s J2000 RA • DEC_POS_EQ_MAIN – the object’s J2000 Dec *Note: these are Version 1.0 UCDs NVO Summer School 2008 – Santa Fe

  5. Simple Image Access (SIA) • Retrieve a table of available images that intersect a “rectangular” region of sky http://www.ivoa.net/Documents/latest/SIA.html • Pretty simple interface: http://<base_url>POS=<deg>,<deg>&SIZE=<deg>[,<deg>][…] • POS – R.A. and dec. in degrees, separated by a comma • SIZE – width & height in degrees • One value: “square” region • INTERSECT – restrict how image intersects the region • COVERS, ENCLOSED, CENTER, or OVERLAPS • FORMAT – restrict the types of images returned • Mime types: e.g. image/fits or image/jpeg • ALL – all available types • GRAPHIC – any computer graphics format (JPEG, GIF, PNG,…) • Image generation parameters • Anything else: service-specific parameters • Just want metadata? • Use FORMAT=METADATA • Service will return VOTable header with no rows NVO Summer School 2008 – Santa Fe

  6. Simple Image Access (SIA) • VOTable results • RESOURCE element with attribute type="results" • Contains exactly one TABLE element • Each record represents an available image • Must contain columns with the following UCDs*: • VOX:Image_Title – a concise description of the image • RA_POS_EQ_MAIN – the RA of the image center • DEC_POS_EQ_MAIN – the Dec of the image center • VOX:Image_Naxes – the # of axes in image • VOX:Image_Naxis – the # pixels along each axis (comma-separated) • VOX:Image_Scale – the pixel size along each axis, in degrees • VOX:Image_Format – the MIME type of the image • E.g. image/fits, image/gif • VOX:Image_AccessReference – the URL for retrieving the image • Other columns are recommended • World Coordinate System (WCS) metadata • Spectral information NVO Summer School 2008 – Santa Fe

  7. Simple Spectral Access • Retrieve a table of available spectra that intersect a “rectangular” region of sky http://www.ivoa.net/Documents/latest/SSA.html • Fairly simple interface: http://<base_url>REQUEST=queryData&POS=<deg>,<deg>&SIZE=<deg>[…] • REQUEST – what to do with inputs • Other operations coming: e.g. getData, stageData, getAvailability, … • POS – R.A. and dec. in degrees, separated by a comma • SIZE – width & height in degrees • FORMAT – restrict to spectra encoded particular format • BAND – restrict to wavelength range • Range format: 2.7E-7/0.13 (optical to radio) • TIME – restrict to observation time range • Range format: 1998-05-21/1999 (May 21, 1998 to 1999) • Other metadata parameters • Anything else: service-specific parameters • Just want metadata? • Use FORMAT=METADATA • Future: will be replaced by REQUEST=getCapabilities NVO Summer School 2008 – Santa Fe

  8. Simple Spectral Access (SSA) • VOTable results • RESOURCE element with attribute type="results" • Contains exactly one TABLE element • Each record represents an available image • Semantic meaning of a column identified by its UTYPE • More specific in meaning than UCD • Must contain columns with the following UTYPEs: • Dataset.Length – # of values in spectrum • DataID.Title – dataset name • Curation.Publisher – name of the spectrum’s publisher • Char.SpatialAxis.Coverage.Location.Value – observed position for the spectrum (RA & Dec.) • Char.SpatialAxis.Coverage.Bounds.Extent – angular diameter of spectrum’s apeture • Char.TimeAxis.Coverage.Location.Value – midpoint of observation time • Char.SpectralAxis.Coverage.Location.Value – wavelength spectrum midpoint • Char.SpectralAxis.Coverage.Bounds.Extent – wavelength bandwidth of spectrum • UTYPEs are available for many columns of spectrum metadata NVO Summer School 2008 – Santa Fe

  9. Caution: pre-v1.0 SSA • SSA is relatively new • Fairly few, fully compliant services • Majority of currently registered SSAs pre-date the final v1.0 version of standard • Registered as version 0.5 compliant • See interfaceVersion column in Registry Results browser NVO Summer School 2008 – Santa Fe

  10. What is “simple” anyway? • With a “simple” DAL service, • You can express the query as a single URL • Can be easily submitted via a web browser • Easy to call via a scripting language • Results are returned as a VOTable • Can use any XML or VOTable viewer/parser to open it up NVO Summer School 2008 – Santa Fe

More Related