1 / 19

Other Services in Hyrax

Other Services in Hyrax. Other Services in Hyrax. HTML: Web-page form interface ASCII: Data responses Info: Metadata as formatted HTML Catalogs: Directory and THREDDS responses GridFTP: Direct access to the BES. HTML Form Interface. The Form interface provides a uniform way to access data

hor
Download Presentation

Other Services in Hyrax

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Other Services in Hyrax

  2. Other Services in Hyrax • HTML: Web-page form interface • ASCII: Data responses • Info: Metadata as formatted HTML • Catalogs: Directory and THREDDS responses • GridFTP: Direct access to the BES

  3. HTML Form Interface • The Form interface provides a uniform way to access data • It’s quite primitive because it must work for all data - even data with no semantic metadata • When a constraint is formed with this interface, the subsetting must be done using only syntactic information like the physical size of an array. • This interface is part of every Hyrax and most other DAP servers, although there are some slight variations among different implementations • It provides a way to view the structure of the data and form a constraint so that a complete URL can be cut and pasted into another application

  4. ASCII • ASCII is the lowest common denominator for data since many programs can read it • Hyrax has a handler that transforms the DAP data response (binary data with structural metadata) into a CSV ASCII response. • You can use a spreadsheet to read data • Try it!

  5. Find some data in the form interface Build a constraint with the form (remember this is ASCII we’re talking about, so data volume is a consideration!) and then hit ‘Get ASCII’

  6. FireFox puts the new data into a tab, other browsers open a separate window…

  7. Try it with a spreadsheet; type (or paste) the URL into the ‘Location’ box. Then hit ‘OK.’ This example is using Kspreadsheet from the Virtual machine; Excel works too.

  8. Data come back in the spreadsheet cells. You may be prompted to supply information Depending on the spreadshhet

  9. Info • The DDS (structural) and DAS (semantic) metadata responses can be combined and returned as an HTML page • Add the extension ‘.info’ to the DAP URL • Try it…

  10. Catalogs • OPeNDAP directory is a kind of catalog • Conceptually based on the HTML directory pages • Provides a click-to-navigate interface • Different servers implement it differently and this has created problems with uniformity for clients • THREDDS provides a framework for cataloging data sources • Provides a way to list several different access protocols • Listings iin XML

  11. Catalogs, cont. • To get the OPeNDAP directory listing from an old server, end the URL in a slash (/) • To get this from Hyrax, use either a slash or ‘contents.html’ • To get a THREDDS catalog from any DAP server, use ‘catalog.xml’ for the XML listing or ‘catalog.html’ for the HTML version • The HTML THREDDS catalog supports a click-to-navigate interface like the old OPeNDAP directory • Try it

  12. THREDDS examples • Switch Hyrax to the THREDDS HTML view: Choose the HTML view

  13. The THREDDS HTML view • The top-level THREDDS catalog on our test server defines a single data root directory (SVN Test Data Archive) • This illustrates how THREDDS can be used to control the view of data presented by the server • Use ‘catalog.xml’ in place of ‘catalog.html’ to get the catalog data in an XML document.

  14. Traverse the links to find data

  15. THREDDS data set page • THREDDS catalogs can list more than one access mechanism - here we see on the DAP, but WCS, WMS, et c., are other possibilities

  16. GridFTP • Hyrax does not actually support GridFTP but the BES does. • The BES was originally developed as a DAP-capable server for the Earth Systems Grid project (now ESG II) • The BES can be run standalone in appropriate environments and the GridFTP protocol can be used to connect to it. • Using GridFTP, DAP requests can be made and DAP responses transferred to ESG clients.

  17. Numerical Grid Computations within BES • CSAC - Community Specro-polarimetric Analysis Center • Two part problem; • Compute the so called Stokes vector for a given vector parameter space P using a model F(x,y,λ; P) • Correct P to improve the fit of the theoretical signal to the observed signal using non-linear optimization • The purpose is to determine a vector parameter space for which the model produces the best fit in the Least Square sense. • The complete problem is solved with a Numerical Library called MERLIN • Very computational Intensive, once took up to 6 hours now takes 16 minutes.

  18. High Performance Solution • Create multiple versions of the computing node waiting for data to crunch • Create a central place (controller) to browse spectra- polarimetry data inventories and launch inversions upon user’s request. • Let the controller decide how to distribute the problem. • Requires multiple identical processes waiting to be called for duty.

  19. Distributing the Work topology GRID BES Server 1 Client BES Server 2 APACHE TOMCAT MODULE . . . . . . Linux Dualcore Xeon 64-bit processor with 10 Terabyte RAID BES Server 10 Merlin Inversion Engines 20 64-bit AMD Opteron processors running on Solaris 10. Can also plug in MacPro EightCore machines

More Related