1 / 13

New Features in ERDDAP v2.0 (coming soon)

Discover the upcoming new features in ERDDAP v2.0 by Bob Simons at NOAA. From user filters to output file types, and cloud-related data ingest improvements, ERDDAP is evolving for better user experience and administration.

rayjames
Download Presentation

New Features in ERDDAP v2.0 (coming soon)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New Features in ERDDAP v2.0 (coming soon) Bob Simons DOC / NOAA / NMFS / SWFSC / ERD Monterey, CA bob.simons@noaa.gov

  2. Improvements in Every Part of ERDDAP For Users Filters Converters Output File Types Other For Administrators Dataset Types GenerateDatasetsXml Other/Cloud-Related Data Ingest System

  3. For Users:Filters New syntax option for grouping: orderByMax("stationID,time/1day") New orderByMean("stationID,time/1day")*Coding for both by Rob Fuller (Ireland's Marine Institute)

  4. For Users:Converters New Converter to Update URLs (e.g., http: -> https:)GenerateDatasetXml uses this. Time Converter can now convert any(?) time string into an ISO 8601 formatted time string.E.g., "Jan 3, 1984 3:24pm" -> "1984-01-03T15:24:00Z"The data file readers can use this to standardize collections of files with different formats. Units Converter can now "Standardize Udunits"E.g., "deg_C/meter" and "degrees_C meter-1" both become "degree_C m-1".The data file readers can use this to standardize collections of files with different, but compatible, units.

  5. For Users:New Output File Types .dataTable: a JSON variant for Google Visualization* Code by Roland Schweitzer (PMEL) .jsonlCSV1: a JSON CSV variant with column names

  6. For Users:Other Log Axes on GraphsE.g., for acoustic power spectrum graphs. New HTTP Error Numbers and New Error MessagesError { code=404; message="Not found: Your query produced no matching results. (Because ...)";} OrcID Authentication: OAuth2, like Google's OAuth2(not quite finished)

  7. For Administrators:New Dataset Types and Features New EDDTableFromJsonlCSV New EDDTableFromInvalidCRAFiles EDDTableFromMultidimFiles <treatDimensionsAs> EDDGridFrom...Files can extract info from fileNameE.g., axis0 value EDDTableFrom...Files <standardizeWhat>scale_factor, add_offset, missing_value, units, etc.

  8. For Administrators:GenerateDatasetsXml New: EDDType=ncdumpoption (What's in this file?) Improved guessing of cdm_data_type Improved guessing of subsetVariables Improved conversion of other metadata to CF/ACDD Improved conversion of time formats to ISO 8601 New: Correct out-of-date URLs Improved error corrections (e.g., misspellings) Lots of little improvements

  9. For Administrators:Other/Cloud-Related <cacheFromUrl> <cacheMaxGB> Accept externally compressed files (e.g., .gz).Save money by saving disk space! <nThreads> (Axiom wanted) Extract info (e.g., time) from filenames (not read all files) Above features useful on their own, or for cloud (AWS)

  10. For Administrators:Data Ingest via EDDTableFromHttpGet .insert?var1=value1&var2=value2&var3=value3.insert?var1=value1&var2=[v2a,v2b,v2c]&var3=[v3a,v3b,v3c] .delete?var1=value1&var2=value2 Data recorded in log files of changes Infinite Versioning(unique!) Fast, Efficient, Ready large datasets *This is a simplified description.

  11. For Everyone: Numerous little new features Numerous little improvements Numerous bug fixes (nothing major)

  12. Read about ERDDAP and try it outhttp://coastwatch.pfeg.noaa.gov/erddap/Download and install ERDDAPhttp://coastwatch.pfeg.noaa.gov/erddap/download/setup.htmlThank you!Questions? Comments? Suggestions? bob.simons@noaa.gov

  13. Most Underutilized Featuresin ERDDAP All example URL's below are to be appended tohttps://coastwatch.pfeg.noaa.gov/erddap/download/ You can customize the Look And Feel of ERDDAP web pages:setup.html#customize <subsetVariables> identifies vars with a limited number of values:setupDatasetsXml.html#subsetVariables <defaultGraphQuery> specifies the default MakeAGraph query:setupDatasetsXml.html#defaultGraphQuery You can promote global and variable attributes to be variables:setupDatasetsXml.html#globalSourceNames Easiest way to make ERDDAP even faster: SSD for data (1TB for $200)

More Related