Desktop Integration with VO VOClient, DALClient NVO Summer School, Aspen Sept 2006

1. US National Virtual Observatory Desktop Integration with VO VOClient, DALClient NVO Summer School, Aspen Sept 2006 Doug Tody (NRAO), Mike Fitzpatrick (NOAO) NVO Summer School, September 2006

2. Application Development in the VO • Types of VO-Aware Applications • Webapps (e.g., datascope) • interface is a web browser • Application services (wesix, wcsfixer, etc.) • interface is a web service • Application environments (IRAF, IDL, Python, PHP, etc.) • interface is a full user applications environment • Grid workflows (e.g., astrogrid) • interface is a user-generated workflow script (batch job) • At the back-end, all of these can share the same client-side interface to the VO. NVO Summer School, September 2006

3. VO Client-Side Development: Requirements • Support for multiple languages/environments • Every language or environment has its application domain • Let the user decide which is best for their application • High level functionality • Applications generally prefer a high level environment • Integrate VO with existing powerful data analysis tools • Isolate application from details of VO infrastructure • VO interface is relatively low level • Functionality can be complex • multiple versions of interfaces • multiple protocols, grid and web service technologies • storage managers (VOSpace), asychronous data transfers, etc. • Java is the standard language for the VO infrastructure • but is not favored for most science applications development NVO Summer School, September 2006

4. Option 1: Custom Integration for each Language • Approach • Thin layer implemented natively in each target environment • Advantages • Simplest approach so long the as middleware is thin • Easier to customize interface for each target language • Avoids need for multi-language integration • Issues • Doesn't scale well as middleware becomes more complex • have to reimplement functionality for every target environment • ok, until it gets beyond a certain size • Interface and capability matrix will tend to be different for each target environment. NVO Summer School, September 2006

5. Option 2: Common Facility with Multiple Bindings • Approach • A common client-side middleware facility with a thin binding for each target language or environment • MySQL (and many other such facilities) use this architecture • Advantages • Portable • Uniform interface and capability matrix for all target environments • Scales well • Only one implementation required for all target languages • Also scales well in the sense of parallel computing • New capabilities appear immediately in all target languages • Can use Java • Most VO and web/grid system-level software is in Java • Issues • More complex as architecture is multi-language and distributed • Distributed computing performance can be a concern if not careful NVO Summer School, September 2006

6. The VOClient Middleware • Objectives • Provide a high level interface to the VO for user development • Support all major languages and environments • Provide reference code within NVO for the DAL services • Origin • Subsystem in the Opticon/NVO scalable framework architecture • stands on its own due to open systems approach • History • began in 2005 with the dalclient Java library • expanded this year to add multi-language support NVO Summer School, September 2006

7. Python IRAFIDL etc. Java voclientd dalclient registry Architecture • Applications layer • Python, Java, IRAF, IDL, PHP, CASA, etc. • Message bus • RPC and async progress events • VOClient daemon (voclientd) • dalclient Java library • Registry interface • Sesame name resolver • VOSpace, security, async operations • Data access objects • table, image, spectrum, etc. NVO Summer School, September 2006

8. VOClient Functionality • Capabilities • Sesame name resolver • convert astronomical object names to coordinates • Registry query • required for data collection and service discovery • Client side of DAL interfaces • cone search (catalogs), SIAP (images) • soon: SSAP (spectra) and SLAP (line lists) • Basic interface • Multi-threaded, multiple clients and connections • Stateful client-server connection • connection-oriented, like a DBMS or other object interface • state is maintained where the data is managed (voclientd) • concurrent access to a shared object is possible • DAL service interface is RPC to dalclient running in voclientd NVO Summer School, September 2006

9. Sesame Name Resolver • Start voclientd if not already running • Open a connection voc_initVOClient (config_opts) • Name resolver API sr = voc_nameResolver (target) pos_str = voc_resolverPos (sr) radeg = voc_resolverRA (sr) decdeg = voc_resolverDEC (sr) • Close connection voc_closeVOClient (shutdown_flag) NVO Summer School, September 2006

10. DALClient Interface • DAL class hierarchy • Generic dataset (any dataset) • Cone catalogs • SIA images • SSA spectra (soon) • SLAP spectral line lists (soon) • SNAP theoretical models (future) (etc.) • Basic API • Similar for each class of data • Form query, execute query, walk result set (Similar to a JDBC database interface) • Multi-level outputs • stream, votable, data model NVO Summer School, September 2006

11. DAL Query (Base class) dal = new DALConnection () dal = new DALConnection (service-url) dal.addService (service-url) dal.getServiceCount () dal.getServiceURL (index) query = dal.getDALQuery () query.addParameter (name, value) query.getQueryString (serviceIndex) query.executeCSV () query.executeCSV (fname) vot = query.executeVOTable () is = query.executeRAW () NVO Summer School, September 2006

12. CONE Query (extends DAL Query) cone = new ConeConnection () cone = new ConeConnection (service-url) query = cone.getConeQuery () query = cone.getConeQuery (ra, dec, sr) query.addParameter (name, value) qr = query.execute () query.executeCSV () query.executeCSV (fname) vot = query.executeVOTable () is = query.executeRAW () NVO Summer School, September 2006

13. SIAP Query (extends DAL Query) siap = new SiapConnection () cone = new SiapConnection (service-url) query = siap.getSiapQuery () query = siap.getSiapQuery (ra, dec, size) query = siap.getSiapQuery (ra, dec, size, format) query = siap.getSiapQuery (ra, dec, ra_size, dec_size) query = siap.getSiapQuery (ra, dec, ra_size, dec_size, format) query.addParameter (name, value) qr = query.execute () query.executeCSV () query.executeCSV (fname) vot = query.executeVOTable () is = query.executeRAW () NVO Summer School, September 2006

14. Query Response (data model view) • Format • Result set: a sequence of records • Interface qr.getRecordCount () rec = qr.getRecord (i) rec.setAttribute ("<attrname>", "<value>") v = rec.getAttribute ("<attrname>") v.intValue () v.floatValue () (etc.) rec.getDataset ("<path>") path = rec.getDataset () NVO Summer School, September 2006

15. Registry Interface • Approach • Very similar to DAL query • Form query, execute query, step through result set • Interface res = voc_regSearch (sql, keywords, orValues) res =voc_regSearchByService (svc, term, orValues) query = voc_regQuery (term, orValues) voc_regAddSearchTerm (query, term, orValue) res = voc_regExecute (query) count = voc_resGetCount (res) str = voc_resGetStr (res, attribute, index) dval = voc_resGetFloat (res, attribute, index) NVO Summer School, September 2006

16. Current status • Language Support • C/C++, Java, IRAF, FORTRAN (so far) • Partial support for Python, PHP, etc. • Messaging • Message bus currently uses a simple brute-force approach • VOClient Daemon • Connection context currently limited to single service • Usage • Java and C interfaces are usable • IRAF interface is a good example of where we are headed with this NVO Summer School, September 2006

17. NVO Summer School, September 2006