1 / 57

MODIS Ocean Filenames, Structures, and useful metadata

MODIS Ocean Filenames, Structures, and useful metadata. Kay Kilpatrick University of Miami/RSMAS. Most FAQ:Why does MODIS oceans have so many files and why is everything so BIG!. 1km global resolution every day Many more products

royal
Download Presentation

MODIS Ocean Filenames, Structures, and useful metadata

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MODIS OceanFilenames, Structures, and useful metadata Kay Kilpatrick University of Miami/RSMAS

  2. Most FAQ:Why does MODIS oceans have so many files and why is everything so BIG! • 1km global resolution every day • Many more products • Meeting the needs of a diverse user community (pleasing all the people all the time) • Pretty pictures/quick looks ; Medium quality, low resolution • Tracking fronts and features ; all pixels, high resolution • High quality climate data; High quality, medium resolution • Modelers; high quality, various projections not maps • Algorithm developers; underlying raw values

  3. Where to get data and more information Information locations: MODIS Oceans home page • http://modis-ocean.gsfc.nasa.gov MODIS Oceans QA Browse 36km Imagery (MQABI) • http://jeager.gsfc.nasa.gov/browsetool/ Select Terra collection 4 Useful links to documentation and related web pages • http://modis-ocean.gsfc.nasa.gov/doclinks.html Data Ordering locations: NASA GES DAAC WHOM (NASA - Goddard DAAC) http://daac.gsfc.nasa.gov/MODIS/ WHOM, datapool, multi parameter subsetting EOS DATA GATEWAY EDG • http://modis.gsfc.nasa.gov/data/ordering.html JPL PODAAC 4km global SST only http://podaac.jpl.nasa.gov/modis

  4. MODIS Ocean Products • MODIS Instruments: • Terra (1030 morning), MO* • Aqua (1330 afternoon) MY* • Resolution: • Spatial: • Level 2 - 1km swath , ~2030km x 1354km; • Level 3 - 4km, 36km, 1 deg [all products are global] • Temporal Resolution: • Level 2 - 5 minute granule; • Level 3 - daily, 8 day week, monthly, yearly • Other data sets: - SST matchups database (kkilpatrick@rsmas.miami.edu) - ocean color diagnostic data sites

  5. MODIS Ocean data products • There are 86 ocean parameters available in over 100 categories of MODIS Ocean data types archived by (and may be obtained from) the NASA Goddard Distributed Active Archive Center. • The three basic groupings of MODIS ocean data parameters are: • ocean color • sea surface temperature • ocean primary production

  6. Parameters 36 Ocean Color parameters 3 Sea Surface Temperature parameters 8 Primary Productivity parameters (including 2 Primary Production indices) 38 Quality Control parameters.

  7. Processing levels • Level 1 - Unprocessed top of the atmosphere radiance/reflectance • At 1-km spatial resolution • 5 minute granule time resolution ; 288 granules/day • Level 2 swath data • At 1-km spatial resolution • 5 minute granule time resolution; color 144/day, SST 288/day • Level 3 global binned or mapped data • spatial resolutions of 4.63km, 36km, or 1 degree • Time resolutions of one day, 8 days, a month or a year. • The binned; ISEAG grid • The SMI mapped; cylindrical Equidistant Projection Centered at 0,0.

  8. Level 4 Productivity • Ocean primary production data is available only as binned or mapped Level 4 (i.e. L4) data. • Ocean Productivity outputs are averaged weekly or yearly. Like the L3 data, the L4 data is organized spatially as either 4km ISEAG gridded bins or as maps using a Cylindrical Equidistant Projection. The mapped data products are available in a choice of 4km , 36km, or 1 degree spatial resolutions. More than one model is used for deriving these data products and some quality statistics are available.

  9. L2 filenames2 night and 6 day ocean files for each 5minutes • M*DCL2 - water leaving radiance products • M*DCL2A - derived products group 1 • M*DCL2B- derived products group 2 • M*DOCQC - ocean color QC parameters • M*D28L2 -SST products • M*D28QC- SST QC parameters • File size ~80-100MB/file • * = “Y” for AQUA or “O” for Terra

  10. Water leaving radiance products • M*DOCL2 water leaving radiance (parameters 1-12) • 7 bands; nLw412,nLw443,nLw488,nLw531,nLw551,nLw667,nLw678 • Tau 865 Aerosol Optical thickness 865 • Epsilon 765/865 • Aerosol model 1 • Aerosol model 2 • Epsilon of clear water

  11. Derived products group 1 • M*DOCL2A- 13 products (parameters 13-25) • CZCS_pigment Chlorophyll-a + pheopigment (fluorometric,empirical) • Chlor_MODIS Chlorophyll-a concentration (HPLC, empirical) • Pigment_c1_total Total pigment concentration (HPLC, empirical) • Chlor_flur_ht Chlorophyll fluorescence line height • Chlor_fl_base Chlorophyll fluorescence baseline • Chlor_fluor_effic Chlorophyll fluorescence efficiency • Susp-solids-conc Total suspended matter concentration in ocean • Cocco_pigment_conc Pigment concentration in coccolithophore blooms • Cocco_conc_detached Detached coccolithophore concentration • Calcite_conc Calcite concentration • K_490 Diffuse attenuation coefficient at 490 nm • Phycoeryth_conc Phycoerythrobilin concentration • Phycou_conc Phycourobilin concentration

  12. Derived products group 2 • M*DOCL2A -11 products (parameters 26-36) • chlor_a2 Chlorophyll-a concentration (SeaWiFS analog) • chlor_a3 Chlorophyll-a concentration (semianalytic) • ipar Instantaneous photosynthetically available radiation • arp Instantaneous absorbed radiation by phytoplankton for fluorescence • absorp_coef_gelb Gelbstoff absorption coefficient at 400 nm • chlor_absorb Phytoplankton absorption coefficient at 675 nm • tot_absorb_412 Total absorption coefficient at 412 nm • tot_absorp_443 Total absorption coefficient at 443 nm • tot_absorb_488 Total absorption coefficient at 488 nm • tot_absorb_531 Total absorption coefficient at 531 nm • tot_absorb_551 Total absorption coefficient at 551 nm

  13. SST L2 products • M*28L2 - 3 products (parameters 36 to 40) • SST_D1 - Sea surface temperature (daytime), 11um • SST_D2 - Do not use Sea surface temperature (daytime), 4 um • SST_N1 - Sea surface temperature (nighttime), 11um • SST_N2 -Sea surface temperature (nighttime), 4um

  14. L2 Ocean color QC file • M*DOCQC - • U_Wind m/s • V_Wind m/s • Pressure mBar • Humidity kg/m2 • Ozone dobson • Latitude degree • Longitude degree • SolarZenith angle • SolarAzimuth angle • SatelliteZenith angle • SatelliteAzimuth angle • nLw670 W/m2/um/sr • Aerosol radiance 765 W/m2/um/sr • Rayleigh radiance 443 W/m2/um/sr • Glint radiance W/m2/um/sr • Whitecap radiance W/m2/um/sr

  15. L2 SST QC file • M*D28QC • D1,N1 Channel 20 brightness temperature degrees C • D2.N2 Channel 22 brightness temperature degrees C • D3,N3 Channel 23 brightness temperature degrees C • D4,N4 Channel 31 brightness temperature degrees C • D5,N5 Channel 32 brightness temperature degrees C • D6,N6 Channel 20 radiance W/m2/um/sr • D7,N7 Channel 22 radiance W/m2/um/sr • D8,N8 Channel 23 radiance W/m2/um/sr • D9,N9 Channel 31 radiance W/m2/um/sr • D10,N10 Channel 32 radiance W/m2/um/sr

  16. L3 Binned global files • Binned files M*DOC{D,W,M,Y}{prod#}.parameter M*D28{D,W,M,Y}{prod#}.parameter • Only available at 4km resolution • intergerized sinusoidal equal area grid (ISEAG) • Only bins with data values are present; • land bins and bins with no data are not in the files • Each files is self contained with sums, weights and quality levels,and all flags • Useful if you need to do statistic or your own averaging/compositing algorithms. • File size ~640MB

  17. L3 mapped files • Various time and space resolutions • Cylindrical equidistant projection • All bins present, including fill values (255) for land and missing data • 8 map files for each product • Mean, Count,Stdev, Common flags,Quality flags,Flag byte 1, Flag byte 2, Flag byte 3 • Files sizes : 4km =67MB, 36km=1MB,1deg= .2MB

  18. File formats • All Ocean products are in EOS HDF format • Values are stored as counts • Scaling information to convert from counts to real numbers is located in the file metadata for each parameter (SDS) • Scale_type • Slope • Intercept

  19. QL=<=3 QL=0 Pixel Quality Flags Each product contains all values at all levels of quality you must filter the data for your application using the quality flag. • Each pixel is associated with a quality level stored as a 2 bit value (0,1,2,3) in the “quality” SDS. • The position of the quality level within the byte is given in the metadata attributes for each parameter (“Quality_Bits”). • Example: MODCL2A: Chlor_MODIS quality is in bit position 11-12, while Chlor_Fluor_ht is in position 19-20. (note: 1-based numbering convention)

  20. Quality level definitions • There are four quality levels • 0=good • 1=questionable/suspect • 2= sun glint or possible cloud contaminated • 3=bad, thick cloud or other failure • These levels are derived by evaluating various combinations of threshold tests common to all products and specific to individual products. Other sets of flags,known as common and product specific flags, are used to store the results of these threshold tests.

  21. MODIS OceanFlags, quality control, and masks Part 2 Kay Kilpatrick University of Miami/RSMAS

  22. Flags • The Ocean products are designed to meet the needs of a variety of users. These flags are key to controlling the quality and coverage of the data for your application. • Quality flags • Common flags • Product specific flags

  23. How are they stored in the files All flags are stored as data type byte Quality SDS -Quality levels are stored at 2 bits Level 3- 1 byte per file only for that product integer value of 0,1,2,3,or 255 Level 2 - maximum of 3 bytes per files, must decode the bits Common flags SDS - each test is 1 bit, Level 2&3 1 byte per file L2_flags SDS - product specific tests each test is 1 bit Level 2&3 max 3 byte per file

  24. Example metadata ncdump -h MODOCL2.A2001191.1605.003.2001312071143 dimensions: Number of records:Swath = 2030 ; Number of samples per record:Swath = 1354 ; variables: short nLw_412(Number of records:Swath, Number of samples per record:Swath) ; nLw_412:Long_name = "Normalized water-leaving radiance at 412 nm" ; nLw_412:Scale_type = "y= Slope * x + Intercept;" ; nLw_412:Slope = 0.001f ; nLw_412:Intercept = -5.f ; nLw_412:Name = "Radiance" ; nLw_412:Units = "W/m^2/um/sr" ; nLw_412:Quality_Bits = "1-2" ;

  25. Common flags • The common flags are tests that are the same for every product The results of these tests are stored in the “common_flag” SDS as type byte. • Bit 1 Pixel not processed • Bit 2 Atmospheric correction failed • Bit 3 Satellite Zenith angle >45 • Bit 4 Solar Zenith angle >70 • Bit 5 Shallow water (<5km from coast or <50m deep,or inland lake) • Bit 6 Sun glint threshold or cloud 678 homogeniety exceed • Bit 7 Invalid or missing ancillary data • Bit 8 Land (include ephemeral water)

  26. Common flag metadata byte common_flags (Number of records:Swath, ……. common_flags:Units = "bits" ; common_flags:f01_name = "UNPROC" ; common_flags:f02_name = "ATMCOR" ; common_flags:f03_name = "SATZ" ; common_flags:f04_name = "SOLZ" ; common_flags:f05_name = "SHALLOW" ; common_flags:f06_name = "GLINT" ; common_flags:f07_name = "SUPPDATA" ; common_flags:f08_name = "LAND" ;

  27. L2_flags radiance byte 1 of 2(nLw_412, nLw_443, nLw_488, nLw_531, nLw_551, nLw_667, nLw_678, Tau_865, Eps_78, aer_model1, aer_model2)

  28. L2_flagsradiance byte 2 of 2(nLw_412, nLw_443, nLw_488, nLw_531, nLw_551, nLw_667, nLw_678, Tau_865, Eps_78, aer_model1, aer_model2)

  29. L2_flagsderived products group 1 byte 1 of 3(eps_clr_water, CZCS_pigment, chlor_MODIS, pigment_c1_total, chlor_fluor_ht, chlor_fluor_base, chlor_fluor_effic, susp_solids_conc, cocco_pigmnt_conc, cocco_conc_detach, clacite_conc, K_490, phycoeryth_conc, phycou_conc)

  30. L2_flags derived products group 1 byte 2 of 3 (eps_clr_water, CZCS_pigment, chlor_MODIS, pigment_c1_total, chlor_fluor_ht, chlor_fluor_base, chlor_fluor_effic, susp_solids_conc, cocco_pigmnt_conc, cocco_conc_detach, clacite_conc, K_490, phycoeryth_conc, phycou_conc)

  31. L2_flags derived products group 1 byte 3 of 3(eps_clr_water, CZCS_pigment, chlor_MODIS, pigment_c1_total, chlor_fluor_ht, chlor_fluor_base, chlor_fluor_effic, susp_solids_conc, cocco_pigmnt_conc, cocco_conc_detach, clacite_conc, K_490, phycoeryth_conc, phycou_conc)

  32. L2_flags derived group 2 byte 1 of 2(chlor_a_2, chlor_a_3, ipar, arp, absopr_coef_gelb, chlor_absorb, tot_absorb_412, tot_absorb_443, tot_absorb_488, tot_absorb_531, tot_absorb_551)

  33. L2_flagsderived group 2 byte 2 of 2(nLw_412, nLw_443, nLw_488, nLw_531, nLw_551, nLw_667, nLw_678, Tau_865, Eps_78, aer_model1, aer_model2)

  34. L2_flags SST byte 1 of 2(SST_D1, SST_N1, SST4_N2)

  35. L2_flagsSST productsbyte 2 of 2(SST_D1, SST_N1, SST4_N2)

  36. Product specific flag metadata L2_flags:Name = "one bit per flag, flag 1 is in the least significant bit" ; L2_flags:Units = "bits" ; L2_flags:f01_name = "blend" ; L2_flags:f02_name = "package" ; L2_flags:f03_name = "IparArp_In" ; L2_flags:f04_name = "hi_wind" ; L2_flags:f05_name = "Cloudy" ; L2_flags:f06_name = "chl_hipackage" ; L2_flags:f07_name = "NDT_package1" ; L2_flags:f08_name = "NDT_package2" ; ……………..

  37. Other useful metadata • Processing control • INPUTPOINTER • CorVer

  38. Metadata: Processingcontrol Provides information on various externally controlled thresholds and settings Processing Control = "REFTYP = 1;AER_MODEL = Quadratic/ Linear 3d; MET1 =……….. ;SUNGLINT1 = 0.005;SOLZEN1 = 70.0;SATZEN1 = 63.0;HIGHTAU1 = 10.00;CLDICE1 = 2.50;COCCOLITH1 = 11.00, 8.10, 1.00, 2.00, 1.00, 1.60, 0.95, 1.50;EPSILON1 = 0.67, 1.50;TURBIDW1 = 3.420;MSKFLG = LAND1;NOAERO = 0;RHO_WN_MAX = 0.010;GlintSC = 2.280 2.280 2.280 2.280 2.280 2.260 2.260 2.270 2.270 2.250 2.250;LgCalc = 0;LgSSTc = 0.000 0.000 0.000 0.000;HighAOI = 0;USEPOL = 1;USEAVG = 1;DOCORR = 1;DOAVGCH1 = 2;LAC ;SGLINTTH = 0.017;POLENAB = 1,1,1,1,1,1,1,1,1;RHO_T_MAX = 5.000;USEF0VAR

  39. Metadata :INPUTPOINTER List all files input to the processing for a given L2 granule Level 3 lists only the first l2 granule that when into the global file OBJECT = INPUTPOINTER\n", " NUM_VAL = 75\n", " VALUE = (\"modis_radcor_v12_56.col.hdf\", \"MOD28L2.A2001073.2000.004.2002183150615.hdf\", \"MOD28QC.A2001073.2000.004.2002183150615.h df\", \"MOD021KM.A2001073.2000.003.2001183210423.hdf\", \"MOD03.A2001073.2000.00 3.2001183190535.hdf\", \"S200107312_NCEP.MET\", \"S200107318_NCEP.MET\", \n", \"S200107400_NCEP.MET\", \"S200107212_EPTOMS.OZONE\", \"S200107rayleigh_modis_667_iqu3.hdf\", \"rayleigh_modis_678_iqu3.hdf\", \ \"new_modis_pol_corr5a.hdf\", \"new_modis_pol_ corr5a.hdf\")\n", …..

  40. Metadata :CORVER Radcor version notes :CorVer = "v12_56: Inter-detector corrs from few grans in each calib Epochs; Bands 8-12 calibrated wrt time trends using modal analysis of Hawaii time series, adjusted absolutely using MOBY data; Cross-scan and mirror-side corrs using the flat-field; rough cal of 6xx bands via a filtered modes (flh consistent); LIMITATIONS: need cross-scan per detector; mirrors-side corrs not stable per Epoch; inter-detector trends in Band 15,16 still reflected in Bands 8-14; 6xx detectors not stable enough" ;

  41. Where to get data and more information Information locations: MODIS Oceans home page • http://modis-ocean.gsfc.nasa.gov MODIS Oceans QA Browse 36km Imagery (MQABI) • http://jeager.gsfc.nasa.gov/browsetool/ Select Terra collection 4 Useful links to documentation and related web pages • http://modis-ocean.gsfc.nasa.gov/doclinks.html Data Ordering locations: NASA GES DAAC WHOM (NASA - Goddard DAAC) http://daac.gsfc.nasa.gov/MODIS/ WHOM, datapool, multi parameter subsetting EOS DATA GATEWAY EDG • http://modis.gsfc.nasa.gov/data/ordering.html JPL PODAAC 4km global SST only http://podaac.jpl.nasa.gov/modis

  42. nLw quality Mandatory Quality for all of Gordon's nLw parameters 0 = good, if common flags are clear, and product-specific L2_flags 1-4 and 9-16 are clear. 1 = questionable, if large satellite or solar zenith angles 2 = cloud or sun glint contaminated 3 = bad other than cloud, if any input radiances are negative or saturated , or Atmospheric Correction failed, aerosol model==16,or Land

  43. eps_clr water Quality Mandatory Quality Carder's eps_clr_water parameter 0 = good, if common flags, input Lw flags and product specific L2_flag 15 is clear. 1 = questionable, if any of: shallow, large zenith angles, bad ancillary data, or product-specific L2_flag 15 is set. 2 = cloud or sun glint contaminated 3 = bad, if any input radiances are negative  or saturated, or Atmospheric Correction failed, or Land

  44. Chlor_Modis Quality Mandatory Quality for Clark Chlor_MODIS 0 = good, if pigc between 0.01 and 100 mg/m**3 1 = questionable, if high satellite zenith angle , shallow water, bad ancillary data, data values <0.01 or >100 mg/m**3 2 = Sun glint contaminated 3 = input radiances are negative or saturated, cloudy, aersol model==16, land or atmospheric correction problem, pig_c <0.0 mg/m**3

  45. chlor_fluor_ht Quality Mandatory Quality for Abbott's chlor_fluor_ht parameter ( quality  level demoted by 1 if high sat or solar zenith angle) 0 = good, if common flags except shallow are clear, input Lw flags are clear, product specific flags clear 1 = questionable, L2_flags 7 (FLH/CHL too large) or L2_flag 9 (FLH too large) 2 = if FLH/Chl way too large, FLH way too large, or modis_chl=-1 3 = bad,  if input radiances are negative or saturated , cloudy/albedo,  atmospheric correction problem, glint, bad ancillary data, or land

  46. Coccolith quality Mandatory Quality for Gordon/Balch cocco_pigmnt_conc, cocco_conc_detach, calcite_conc 0 = good, if common flags are clear, input Lw flags are clear, and product-specific L2_flags clear. 1 = questionable, if any of: shallow, large zenith angles, bad ancillary or product flags 15-17 2 = cloud, if any input radiances are negative and saturated, cloud, or glint 3 = bad, if any input radiances are negative or saturated, or Atmospheric Correction failed, aersol model==16, Land, or L2_flags 22-24 are set

  47. CZCS Quality Mandatory Quality for Clark's CZCS_pigment parameter 0 = good, if pigc between 0.01 and 100 mg/m**3 1 = if any high satellite zenith angle , shallow water, bad ancillary data, or data vales <0.01 or >100 mg/m**3 2 = Sun glint contaminated 3 = bad, if input radiances are negative or saturated , glint, cloudy, aer0sol model==16, land or atmospheric correction problem , pig_c <0.0 mg/m**3

  48. Pigment_c1_total Quality Mandatory Quality for Clark's pigment_c1_total  0 = good, if pigc between 0.01 and 200 mg/m**3 1 = questionable if any high Sateillite zenith angle, shallow water, bad ancillary data, data vales <0.01 or >200 mg/m**3 2 = Sun glint contaminated 3 = bad, if any input radiances are negative or saturated, glint, cloudy, aersol model==16, land or atmospheric correction problem , pig_c <0.0 mg/m**3

  49. Susp_solids_conc Quality Mandatory Quality for Clark's susp_solids_conc parameter 0 = good, if values between  0.03 and 100 g/m**3 1 = questionable, susp_solid  <0.03 or > 100 g/m**3, high satellite or solar zenith angle, shallow water, bad ancillary data 2 = sun glint contaminated 3 = bad, if any input radiances are negative or saturated, or AtmosphericCorrection failed, aersol model==16, or susp_solid less than 0.03 or greater than 100 g/m**3, Land, cloudy

  50. K_490 Quality Mandatory Quality for Clark's K_490 parameter  0 = good, K490 between 0.016 and 20.0 1 = questionable, if any values <0.016 or > 20.0 , high satellite or solar zenith angle, shallow water, or bad ancillary data 2 = glint contaminated 3 = bad, if any input radiances are negative and or saturated, or Atmospheric Correction failed, aerosol model==16, k490 <0.0 , Land, or cloudy

More Related