1 / 42

Clients

Clients. Accessing Data with DAP. Web Browser Already discussed… Graphical clients ncBrowse, ODC, Ferret, GrADS Command-line clients getdap (UNIX, win32), loaddap (Matlab, IDL), nco (UNIX, win32) Custom clients C++, C, Java, Python netCDF. Using a Graphical Client.

jonathanmoore
Download Presentation

Clients

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Clients

  2. Accessing Data with DAP • Web Browser • Already discussed… • Graphical clients • ncBrowse, ODC, Ferret, GrADS • Command-line clients • getdap (UNIX, win32), loaddap (Matlab, IDL), nco (UNIX, win32) • Custom clients • C++, C, Java, Python • netCDF

  3. Using a Graphical Client • Example: The OPeNDAP Data Connector • Combines data location with retrieval and display • Shows the built URL, including constraint expression • Can be transferred to another application

  4. Start the ODC

  5. The ODC opens to the search pane Five different panes Choices within a pane

  6. Use the dataset list to find the TPAC climatologies Choose the Antarctic Cooperative Research Centre TPAC/CISRO Climatologies …then hit ‘To Retrieve’ to move the selection to the next pane

  7. The Retrieve pane Double click ‘levitus_annual_97.nc’ To see the contents of the file in The area on the right

  8. The ODC shows the URL as it builds it. Click the checkbox for SALT and O2. For both, set the range of z_index to ‘0 to 0’. Make sure to hit tab/return in The boxes. …then hit ‘Output to’ to move to the View pane

  9. There are a number of ways to view The data. Here the plotter has been Chosen (the default). Hit ‘Plot to’ to generate a plot using the Default settings.

  10. When the plot is made, the interface Switches to the ‘Preview’ tab Switch back to the ‘Variables’ tab to Plot O2

  11. Choose ‘O2’ from the menu, then hit ‘Plot to.’

  12. Now that the data have been read and Cached, you can switch back and Forth between variables quickly without Any additional data transfers When ready, go back to the ‘Retrieve’ Pane.

  13. Choose ‘TEMP’ Set the constraint …then plot

  14. ODC Summary • The ODC provides a way to search for, access and plot data • Acts as a ‘URL builder;’ the URLs can be pasted into other applications • We didn’t need to know anything about DAP, its Request or Response objects or how a URL is used to request data • The data set list often contains stale entries • Also supports using the GCMD for data location - more on this when we cover searching

  15. Using a Command-line Client • Matlab - demonstration • NCO - a powerful tool developed and maintained by another group

  16. Matlab • Demonstration of custom-built Graphical interfaces for Matlab • Matlab scripting is used to build the interfaces and provide some dataset-specific processing • A Matlab command extension is used to read the data (written in C/C++). • Two things are required in addition to Matlab: The DAP command extension (‘loaddap’) and the graphical interface software.

  17. Running the Matlab Demonstration • Start Matlab • Download the command extension • Download the interface software • In Matlab change directory to the ‘ml-ocean-testbed’directory. • Type ‘OCEAN_TOOLBOX’ • The interface will start…

  18. The Ocean Toolbox

  19. Open a dataset I choose the Pathfinder dataset

  20. Fill in the information SST & Quality fields Load data into the Matlab workspace

  21. Get the data Load data into the Matlab workspace

  22. Plot/Display the data

  23. Using the loaddap command extension directly • Start Matlab • Add the directory with the extension to the Matlab command path • Verify the command extension is working • Feed it a URL • Plot the data

  24. Pass a URL, constrain the response To the ‘u’ and ‘v’ vectors only Plot those vectors; See Figure 1

  25. Matlab Summary • Command line client is the tool used to move the data • Easily used in Matlab scripts to hide the details and make custom interfaces • To the the command extension directory, user must know: • Data location (URL) • Internal structure of the data set (syntactic metadata - DDS/DDX) • How to write a constraint expression

  26. NetCDF Operators (NCO) • Unix command line client • Unlike the previous two clients, NCO uses the netCDF client library to read from a DAP server • A client library is a collection of functions which hide the mechanics of (most of) the interaction with a server so the client can go about its business • The NCO client is, in fact, just the NCO package linked to our (OPeNDAP’s) version of the netCDF library (aka. the netCDF client library)

  27. Build the NCO Software • Change directory to /root/src/nco-3.9.2 • root@slax:~# cd /root/src/nco-3.9.2 • Run configure to build the Makefile, then build and install the software • root@slax:~# ./configure • root@slax:~# make • root@slax:~# make install

  28. Use NCO to Convert the FNOC1 vectors into a speed • NCAP: NCO Arithmetic Processor • ncap2 -s ‘speed=sqrt(u^2,v^2)’ http://localhost:8080/opendap/data/nc/fnoc1.nc spd.nc • The URL is the input ‘file’ and wndspd.nc is the output • Use ncdump to look at the result file • ncdump -h spd.nc • ncdump -v speed spd.nc

  29. View the Result: ncBrowse • We can use ncBrowse to look at the local neetCDFZ we just built • ncBrowse can also look at the DAP server directly • Built using the DAP-enabled Java netCDF library (a client library where access to DAP servers hides behind the netCDF API)

  30. Start ncBrowse Double click on ‘speed’ - the new data we made with the previous NCO example

  31. Fix up the latitude and longitude axes, the ‘Graph Variable.’

  32. We have to be somewhat savvy about the units - check back and look at the attributes…

  33. Custom clients • What options exist to build clients • C++ using libdap • C using Ocapi • C,Fortran using the netcdf client library • Python using PyDAP • Java using Java-OPeNDAP • Matlab & IDL using the respective versions of loaddap

  34. Clients Summary • Custom clients offer an opportunity to develop for a specific audience or a particular problem/project. • Example: ComMIT Tsunami inundation model client developed by NOAA/PMEL and BOM • General purpose clients like loaddap can read any kind of data while clients built using the netCDF client library are limited to the semantics of netCDF • Example: Record access is slow because each access is separate network request

  35. Clients Summary, cont. • ODC: A client built specifically to provide a ‘browse’ capability for any data source • Uses Java-OPeNDAP • Loaddap: a client built to read any data into an analysis application • Can be used as a building block for more sophisticated applications • Use libdap (C++, Matlab) or Ocapi (C, IDL) • netCDF client library: A client-building tool • convert ‘legacy’ code • provide a simple way to write new applications • C++, C, Fortran

  36. Finding Data • Ways to find data: • The OPeNDAP Data Set List • GCMD • TPAC • Google • THREDDS • We maintain a page with links to dataset searching sites: • http://www.opendap.org/data/index.html

  37. Common Features • All of these data location features except Google depend on active community involvement in building catalogs of data • The solutions can be described as static documents or crawlers • Google and TPAC are crawlers • Crawlers can discover datasets without human intervention • They can make mistakes that seem silly • The The Dataset List, GCMD and THREDDS are static documents or collections of static documents • Static lists can be tailored by hand • They can go out of date quickly

  38. Differentiating Features • Google & TPAC: • Google is just crawling HTML. If a server is not linked to a HTML page, it won’t be found. • TPAC is preset with server locations and picks up changes at those sites

  39. Differentiating Features, cont. • The Static Lists: • The Dataset List has a very low metadata requirement • Not maintained as actively as either GCMD or THREDDS catalogs • GCMD: • The GCMD has a fairly high entry level threshold • Professional staff maintain the GCMD as their sole job • THREDDS • THREDDS catalogs are, or can be, located at the data - locality distributes maintenance • Quality varies from site to site

  40. Finding Data Summary • Locating data seems like it would be the place to start building a system, but it’s far more varied than the one-size-fits-all approach most tried in the 1990’s • Crawlers and hierarchical lists show the most promise but maintained centralized lists are also useful

More Related