1 / 34

ncl.ucar/Training/Workshops/lectures.shtm

Slides based on lectures from the NCL workshop. http://www.ncl.ucar.edu/Training/Workshops/lectures.shtm. Also see the NCL homepage. http://www.ncl.ucar.edu/. This ppt can be downloaded off the web at the below address. http://www.atmos.albany.edu/student/janiga/temp/incoming_grad_lecture.ppt.

holli
Download Presentation

ncl.ucar/Training/Workshops/lectures.shtm

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Slides based on lectures from the NCL workshop http://www.ncl.ucar.edu/Training/Workshops/lectures.shtm Also see the NCL homepage http://www.ncl.ucar.edu/ This ppt can be downloaded off the web at the below address http://www.atmos.albany.edu/student/janiga/temp/incoming_grad_lecture.ppt

  2. ncl-talk@ucar.edu For additional help… http://www.ncl.ucar.edu/Support/ncl_talk.shtml [haley@ucar.edu , shea@ucar.edu]

  3. Introduction to netCDF The NCL variable model is based on netCDF

  4. netCDF files self describing • (ideally)all info contained within the file • no external information needed to determine file contents portable[machine independent] • Supported by many software tools / languages • NCL, IDL, Matlab, ferret, GrADS, F, C, C++,Java • command line file operators: NCO, CDO • perform various tasks (i.e. concatenation) very efficiently • http://nco.sourceforge.net • http://www.mpimet.mpg.de/fileadmin/software/cdo • ncview • quick visualization using gui if in COARDS format: • IDV, NCVIEW, Panopoly, etc. Similar to HDF (NASA) and GRIB (NCEP, ECMWF, etc) formats; all these are read by NCL using the same one line.

  5. Examining a netCDF file • ncdump file_name • dumps the entire contents of a file; prints every value • ncdump -h file_name • Dumps header information [most commonly used] • NCL equivalent: ncl_filedump file_name • ncdump -v U file_name • NCL equivalent: ncl_filedump –v U file_name • ncl_filedumpfile_name [ more general ] • netCDF3/4, GRIB-1,GRIB-2, HDF, HDF-EOS [HDF5] • ncviewfile_name[visualize file contents] Now for an example….

  6. Parts of netCDF file VARIABLES: Names , Types, Attributes, Coordinate Variables variables: floatlat(lat) lat:long_name = "latitude" lat:units = "degrees_north" floatlon(lon) lon:long_name = "longitude" lon:units = "degrees_east" inttime(time) time:long_name = "time" time:units = "Month of Year" doubleT(time, lat, lon) T:long_name = “Temperature” T:units = “C" T:missing_value = 1.e+20f T:_FillValue = 1.e+20f DIMENSION SIZES dimensions: lat = 64 lon = 128 time = 12 FILE ATTRIBUTES global attributes: title = “Temp: 1999” source = “NCAR” conventions = "None”

  7. visual: simple 2D netCDF Variable coordinate variables (rectilinear grid) • attributes @: • long_name • units Latitude coordinate variable (1D, &) Longitude coordinate variable (1D, &)

  8. Non-Traditional Grids or Arrays • netCDF and NCL can handle various “other” types of data: • Spectral coefficents (YOTC) • Lambert conformal (WRF) http://www.ncl.ucar.edu/Applications/wrflc.shtml • Radiosondes • Surface station data • Hurricane tracks http://www.ncl.ucar.edu/Applications/unique.shtml • Make your own! • Tons of pre-defined functions for manipulating and converting between different formats as well as plotting. http://www.ncl.ucar.edu/Document/Functions/list_alpha.shtm

  9. X Scalar or Array attributes attributes long_name _FillValue units add_offset scale_factor etc. long_name _FillValue units add_offset scale_factor etc. accessed via @ accessed via & X coord var values time lev lat lon etc. coordinates Scalar or Array time lev lat lon etc. netCDF [NCL] Variable model f = addfile(“foo.nc”,”r”) ; grb/hdf x = f->X NCL reads the scalar/array, attributes, and coordinate variables as an object

  10. Detailed Look netCDF Variable (NCL) ncl<return> ; interactive mode ncl0 >f =addfile("UV300.nc", "r") ; open file ncl1 >u = f->U; import STRUCTURE ncl2 >printVarSummary(u) ; overview of variable Classic netCDF Variable Model Variable: u Type: float Total Size: 65536 bytes 16384 values Number of Dimensions: 3 Dimensions and Sizes: [time | 2] x [lat | 64] x [lon | 128] Coordinates: time: [ 1 .. 7 ] lat: [ -87.8638 .. 87.8638 ] lon: [ 0 .. 357.185] Number of Attributes: 5 _FillValue : 1e36 units : m/s long_name : Zonal Wind Component short_name : U missing_value : 1e36 NCL syntax/funcs query use modify add any aspect of data object

  11. NCAR Command Language • Complete Programming Language • data types • variables • operators • expressions • conditional statements • loops • functions/procedures/graphics • Features • query / manipulate meta data • import data in a variety of formats • array syntax / operations • can use user fortran/C codes and commercial libraries • most functions/procedures ignore missing data

  12. Running NCL • Interactive Mode (Command line) • ncl[options][command-line-arguments]<return> ncl> enter commands ncl> quit <return> • can save interactive commands ncl> record “file_name” ncl> stop record • Batch Mode[ .ncl suffix is optional] • ncl[options][arguments]script.ncl • ncl< script.ncl[also acceptable] • nohup ncl[options][arguments]script.ncl >out.txt & • "&" means put in background • “nohup” means don’t quit when I logoff (ie long scripts).

  13. NCL predefined options • ncl –hnxV [predfined options are preceded by dash] • may be used for interactive or batch mode • informational • ncl –h[display predefined options and usage and exit] • ncl –V[print the NCL version and exit] • action • ncl –x[echo statements as encountered (debug)] • ncl –n[don't enumerate dimensions of values in print()] • multiple predefined options • ncl –nx[ not ncl –n –x ] Variables can be set on the command line prior to a script. However, this tends to not be necessary. Input from a text file or systemfunc() http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclCLO.shtml

  14. Sample Batch Script: sample.ncl load “$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl“ load“$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl“ begin ; optional f1 = addfile("TMP_58-97.nc", "r") ; open netCDF file T = f1->Tmp ; T(time,lev,lat,lon) =>(480,17,73,144) f2=addfile(“P_1958-1997.grb", "r") ; open GRIB file P = f2->Pres ; P(time,lev,lat,lon) f3=addfile(“Q_1958-1997.hdfeos", "r") ; open hdf –eos file Q = f3->Specific_Humidity ; Q(time,lev,lat,lon) pot = T* (1.+0.622*Q) *(1000/P)^0.286; potential temperature (array) print("pot: min="+ min(pot) + " max="+ max(pot) ) wks =gsn_open_wks("ps", “sample") ; open a graphic workstation gsn_define_colormap (wks,"gui_default") ; change from default plot =gsn_csm_contour_map_polar (wks,T(0,5,:,:),False) res = True;change default plot res@cnFillOn = True; use colors res@gsnSpreadColors = True; use entire color map plot = gsn_csm_pres_hgt (wks,T(0,:,{50},:),res) end ; only if begin is present

  15. Language Basics

  16. Outline: NCL Language Basics • special syntax characters • data types • Variables  netCDF/NCL variable model • attributes • _FillValue • named dimensions • coordinate variables • print and printVarSummary • shaping • subscripting

  17. NCL Syntax Characters • ;- comment [can appear anywhere] • @- reference/create attributes • ! - reference/create named dimension • & - reference/create coordinate variable • {…}- coordinate subscripting • $ - enclose strings when (im/ex)port variables via addfile • (/../)- array construct characters • : - array syntax • |- separator for named dimensions • \- continue character [statement to span multiple lines] • ::- syntax for external shared objects (eg, fortran/C) • ->- use to (im/ex)port variables via addfilefunction • “ “- use for a string

  18. Numeric double (64 bit) float (32 bit) long (32 or 64 bit) integer (32 bit) short (16 bit) byte ( 8 bit, 0-255) complex NOT supported non-Numeric string character graphic file logical list Data Types Time coordinate variables frequently double. If IR data (K) was offset by 100 it could be stored in byte instead of double (8 times less disk space) or float (4 times less disk space). Data will often be packed into short or byte this way xFloat = short2flt(xShort) xShort@scale_factor xShort@add_offset

  19. Attributes • info about a variable or file [meta data] • attributes can be any data type except file • scalar, multi dimensional array (string, numeric) • assign/access with@character • T@units = “deg C” • T@long_name = “temperature” • T@whatever = an array of any dimension or type • T@_FillValue = -999. • plot title uses x@long_name by default • attribute functions [isatt, getfilevaratts] • if (T@units.EQ.”K”) then • deletecan eliminate an attribute • delete(T@title)

  20. _FillValue • Unidata and NCLreservedattribute • most NCL functions ignore_FillValue • Note: “missing_value” attribute: no special status to NCL • if “T” has “missing_value” attribute but no “_FillValue ” • NCL creates: T@_FillValue = T@missing_value • use built-in function “ismissing” to check for _FillValue • if (any(ismissing(T))) then • T = where (ismissing(T),0,T) • netCDF Operators[NCO]: _FillValue attribute • ncview: missing_value attribute • best to create netCDF files with both • NCL: don’t use commonly encountered values as a _FillValue!

  21. Arrays • row major ….. like C/C++ • left dimension varies slowest; right varies fastest • dimension numbering left to right [0,1,..] • indicies [subscripts] are zero based [0,N-1] Consider T(:,:,:) left dimension is 0 [ T!0 ] middle dimension is 1 [ T!1 ] right dimension is 2 [ T!2 ]

  22. NCL (netCDF) Dimensions • may be “named” • provides alternative way to reference subscripts • recommendation:always name dimensions • use NCL syntax • assigned with ! character{let T(:,:,:)} • T!0 = "time" ; leftmost [slowest varying] dim • T!1 = "lat“ • T!2 = "lon" ; rightmost [fastest varying] dim • dim names may be renamed, retrieved • T!1 = "LAT" … dName = T!2 • delete can eliminate: delete (T!2) • Named dimensions used to reshape

  23. NCL (netCDF) Coordinate Variables • data coordinates [ eg: time, level, lat, lon ] • strict netCDF definition • 1D numeric array • monotonically in[de]creasing numeric values • can only be assigned to a named dimension • 1D array must be the same size as dimension • should not have _FillValue attribute • coordinate values assigned via &character • T&lat = latitude ; Here latitude is a 1D array • used in graphics, coordinate subscripting • coordinate functions • iscoord , isfilevarcoord • deletecan eliminate coordinate array • delete(T&time)

  24. Create and Assign Coordinate Variables • create 1D array • time = (/1980, 1981, 1982 /) ; integer • lon = (/ -90, ... 90 /) ; float • assign dimension name [same as variable name] • time!0 = “time” • lon!0 = “lon” • assign values to named dimension • time&time = time • lon&lon = lon • let x be 2D: name dimensions • x!0 = “time” … x!1 = “lon” • assign coordinate variables to x • x&time = time … x&lon = lon

  25. Access/Change/Create/Delete Meta Data • @ attributes • u@long_name = "U" • lonName = u@long_name • !named dimensions • u!0 = "TIME" • tName = u!0 • &coordinate variable • u&lat = (/ -90., -85, .... , 85., 90. /) • latitude = u&lat • $ substitute string • x = fin->$variable(n)$(date,{-30:60},:)

  26. Variable Assignment • Variable-to-Variable assignment • consider y = xwhere x is previously defined • if y not defined: • y has same type/shape/values/meta data as x • if y predefined: • y must have same shape and type • or, x must be coerceible to the type of y • y attributes, dimension names and coordinate variables, if they exist, will be over written • Value-only assignment(no meta copy) • U multi-dimensional array with meta data • Uval = (/ U /) or Uval = (/f->U/) • the (/ ... /)operator pair strips meta data

  27. Variable Subscripting (1 of 3) Standard Array Subscripting • ranges: start/end and [optional] stride • indices separated by : • omitting start/end index implies default begin/end Consider T(time,lat,lon) T  entire array [ don't use T(:,:,:) ] T(0,:,::5)  1st time index, all lat, every 5th lon T(0,::-1,:50)  1st time index, reverse lat order, 1st 51 lon T(:1,45,10:20)  1st 2 time indices, 46th index of lat, 10-20 indicies of lon

  28. Variable Subscripting (2 of 3) Coordinate Variable Subscripting • only applies to coordinate variables • same rules apply for ranges, strides, defaults • use curly brackets {…} • standard and coordinate subs can be mixed [if no reorder] T(:,{-30:30},:)  all times/lon, lat -30° to +30° (inclusive) T(0,{-20},{-180:35:3})  1st time, lat nearest - 20°, every 3rd lon between -180° and 35°

  29. Variable Subscripting (3 of 3) Named Dimensions • only used for dimension reordering • indicated by | • dim names must be used for each subscript • named/coordinate subscripting can be mixed Consider T(time,lat,lon) t = T(lat|:, lon|:, time|:)  makes t(lat,lon,time) t = T(time|:,{lon|90:120},{lat|-20:20})  all times, 90-120° lon, -20-20° lat

  30. Standard and Coordinate Subscripting Standard: T(9:13,1:8) Coordinate: T({-10:20},{-170:-110}) Latitude coordinate variable (1D) Combined: T({-10:20}, 1:8) Longitude coordinate variable (1D)

  31. “printing” • printVarSummary • gives gross overview of a variable • print • same info as printVarSummary • prints values • write_matrix • print to standard out or a file • format control of numerical output • can write to file also

  32. print (1 of 2) • Prints out all variable information including • meta data, values • T(lat,lon): print (T) Variable: T Type: float Total Size: 32768 bytes 8192 values Number of Dimensions: 2 Dimensions / Sizes: [lat | 64] x [lon | 128] Coordinates: lat: [-87.86379 .. 87.86379] lon: [ 0. 0 .. 357.1875] Number of Attributes: 2 long_name: Temperature units: C (0,0) -31.7 (0,1) -31.4 (0,2) -32.3 (0,3) -33.4 (0,4) -31.3 etc. [entire T array will be printed]

  33. print (2 of 2) • can be used to print asubsetof array • meta data, values • T(lat,lon): print( T(:,103) ) orprint( T(:,{110}) ) • Variable: T (subsection) • Type: float • Total Size: 256 bytes • 64 values • Number of Dimensions: 1 • Dimensions / Sizes: [lat | 64] • Coordinates: • lat: [-87.86379 .. 87.86379] • Number of Attributes: 3 • long_name: Temperature • units: C • lon: 109.6875[ added ] • (0) -40.7 • (1) -33.0 • -25.1 • -20.0 • (4) -15.3 etc.

  34. Things that can be done (look up or ask) • Write netCDF files (compression, making large files) • Read and write binary and ascii • Functions and procedures in NCL language (over 1000), fortran or C (most are earth science orientated; essentially Matlab or IDV for earth science). • Loops, conditional statements. • OPeNDAP support (query and read files directly from the web without downloading the whole file) • http://www.ncl.ucar.edu/Document/Functions/index.shtml

More Related