1 / 27

EPIC Tools for in-situ data collections

EPIC Tools for in-situ data collections. Donald W. Denbo, NOAA/PMEL Willa H. Zhu, NOAA/PMEL/JISAO John Osborne, OceanAtlas Software Christopher Moore, NOAA/PMEL/JISAO. NOAA Coastal Storms Initiative Meeting May 6-8, 2003. EPIC Components. Data Archive and Retrieval

tucker
Download Presentation

EPIC Tools for in-situ data collections

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EPIC Toolsfor in-situ data collections Donald W. Denbo, NOAA/PMEL Willa H. Zhu, NOAA/PMEL/JISAO John Osborne, OceanAtlas Software Christopher Moore, NOAA/PMEL/JISAO NOAA Coastal Storms Initiative MeetingMay 6-8, 2003

  2. EPIC Components • Data Archive and Retrieval • Multiple file formats supported with “epslib” • Metadata database supports subsetting of large data collections • “Classic EPIC” • Unix programs for desktop display, analysis • Gateways to Matlab, PPLUS, Ferret, IDL, etc. • EPIC Web Browser • Search, graph, list, download from the Web • Toolkit for building custom web applications • Java applications • Java OceanAtlas - Sophisticated browse and data exploration • Climate Data Portal - distributed data access (CORBA/OPeNDAP) • OceanShare - Network data collaboration tool • ncBrowse - Generalized browse tool for data/attributes (local files, network files, OPeNDAP files)

  3. EPIC Archive and Retreivalfor Classic EPIC, Web, and Java Applications • Data collections may include thousands or even millions of profiles or time series • Typically the researcher will use EPIC tools to: • Select data by lat, lon, time, keyword, etc. • Refine the initial data selection • Display and/or analyze the selected data

  4. EPIC Data Management Strategy DataFile Collections Data SearchLat, long, timekeyword, etc Meta Data Alaska DataSearchandSelection Data LoaderUtility Pacific MySQLdatabase NODC Selecteddatasets creates pointers to data files EPIC data usage “Classic EPIC”programs Matlab FerretIDLPPLUS EPIC Web Java OceanAtlas Climate Data PortalOceanSharencBrowse

  5. EPIC Data File Input/Output • Data file format • Multiple data file formats are supported • New formats are readily added • Applications software • Independent of file format • API for most programming languages • Gateways • to Matlab and other graphics/analysis packages EPS Library Applications programs API File Independent Layer FDL Existing Planned

  6. Classic EPICSuite of Unix programs for the desktop • Over 100 EPIC programs are available for: • Data selection, metadata and file editing... • Graphics • Time series, multivariable plots, section plots... • Parameter calculation • Dynamic height, geostrophic velocity... • Statistical, spectral, tidal harmonic analyses... • EPIC’s modularity allows a researcher to easily add new analysis techniques or use a favorite graphics or analysis program.

  7. EPIC Web Browser

  8. EPIC Web BrowserData selection, graphics, download

  9. Plots User customized plots for profiles and time series (T, S, Sigma-t, Ox) vs Pressure/Depth Temperature vs Salinity One or two user-selected Properties vs Pressure/Depth Contoured section plots List/download data, metadata New plots and functions readily included EPIC Web Browser Capabilities

  10. EPIC tools for the Web for building customized web access to data Example applications • EPIC Web Browser (general) • TAO (El Nino buoys) • Unaami (Arctic time series) • OSCAR (Satellite ocean currents) • Multibeam data (hydrography) • http://www.epic.noaa.gov EPIC Web Toolkit • Interface design • Backend graphics • Map data selector

  11. EPIC Java Tools and ApplicationsDistributed data accessand Advanced data browsing

  12. Java Tools for locating and selecting from distributed data collections EPIC Java Applications

  13. Challenge • Locating and selecting individual observations from large, distributed collections of in situ data observations • Examples: Individual oceanic or atmospheric profiles, time series, buoy tracks, etc. EPIC Java Applications

  14. Some solutions • Tools and Applications • LDAP Directory Services • NdEdit for subsetting large in-situ collections • Interactive Java graphics • Intensive exploration of multiple data types • OPeNDAP (DODS) clients and servers for in-situ data • Access from Desktop and Web • Assembled from OpenSource components • http://www.epic.noaa.gov/epic/software/ep_java.htm EPIC Java Applications

  15. LDAP Directory ServiceFinding distributed data collections • Search on geographic region, observed quantities or keyword • Automatically populated from data servers • Metadata instantly known to LDAP directories • Access control for experimental data products • Open Source • Standards-based • Platform independent • Scalable, extensible • Supports desktop or web client EPIC Java Applications

  16. NdEdit Explore Data Locations and Subset Data Collections • Interactive, linked windows • View any 2D sliice • xy, xt, xz, yt, yz, yt • Investigate space-time distribution • Select data • in any View Select time rangeSee datain range X-Y map view Y-Ttime range X-Y map view Y-Ttime range EPIC Java Applications

  17. NdEdit Section Tool - select stations along a depth contour EPIC Java Applications

  18. NdEdit Selected stations are in a narrow time band EPIC Java Applications

  19. NdEdit Polygon tool – select an oblique section line See time range of stations EPIC Java Applications

  20. NdEdit • NdEdit is an interactive Java tool • NdEdit is being re-engineered for the Web EPIC Java Applications

  21. SGT: A Scientific Graphics Toolkit • Java toolkit for scientific graphics • On-line documentation, examples, tutorials, streaming video • JavaBeans in next release • Features • Contour, line and vector plots, • Axes user settable or automatically scaled • multiple Time Axis formats • mouse-movable, customizable Labels, line keys, icons • Legend generation automatic • Widely used, freely available • scientific and commercial applications use SGT • 14,782 downloads by 5,151 unique sites from 70 countries • http://www.epic.noaa.gov/java/sgt/ EPIC Java Applications

  22. Java Applications for the Desktop and the Network EPIC Java Applications

  23. ncBrowseJava tool for data/metadata browse • NetCDF and full OPeNDAP support • Desktop Java application • Axis mapping, parametric plots, flexible mapping of independent variables to axes • Line, vector and contour plots • Animations, 3D graphics via VisAD • Popular, freely available • 8,187 downloads by 3,415 unique sites in 59 countries • OPeNDAP (DODS) client, Unidata netCDF client, WOCE data CD, etc. • http://www.epic.noaa.gov/java/ncBrowse/ EPIC Java Applications

  24. Java OceanAtlasfor browsing ocean data • Intensive Java desktop browsing of ocean data • Property-property plots with up to 7 x axes, waterfall diagrams, Contour plots/residual contour plots, Map plots with optional "bubble" isosurface plots, Linked browsing of all plot types, Built-in and custom calculations, NdEdit for data location/selection • Reads WOCE, EPIC, OPeNDAP netCDF data, some ARGO formats • Extended for wide range of data types • Profile, time series, drifting buoy, gridded analyses, model outputs • Widely used, freely available • http://www.epic.noaa.gov/epic/software/JavaJOA.htm • http://odf.ucsd.edu/joa/jsindex.html EPIC Java Applications

  25. Climate Data PortalDistributed ocean observations • Major in situ data collections: • TAO El Nino buoy data (realtime/PMEL), Global sea level data (realtime/U of Hi), GTSPP (NODC), PMEL hydrographic data and time series collection, WOCE ADCP (U of Hi), IPRC/APDRC in situ data, US Coastal Ocean Time Series database (NODC) • OPeNDAP: in-situ data server and client • LDAP directory services • NdEdit for data selection/subsetting • Data Servers, Clients (Java and Web) – focus on in situ data • http://www.epic.noaa.gov/cdp/cdpjava.htm EPIC Java Applications

  26. OceanShareNetwork collaboration tool • Collaboration features • Local, remote data file access and Java graphics • Standard collaboration tool features (annotation, whiteboard, etc) • Secure LDAP Document Repository • Proposals, papers in progress, data, images • Access control by document or by folder • Accessible from web and desktop • OPeNDAP compatible soon • http://www.epic.noaa.gov/collab/ EPIC Java Applications

  27. EPIC Java Tools and Applications • Modular Web and Open Source components • Java interactivity • Low cost development/maintenance • OpenSource, standards-based • Scalable, extensible • OPeNDAP supported or planned • http://www.epic.noaa.gov • http://www.epic.noaa.gov/epic/software/ep_java.htm

More Related