1 / 10

MODLAND Volumes and Loads Status

MODLAND Volumes and Loads Status. MODIS Land Science Team Workshop July 15, 2003 Robert Wolfe MODIS Land Team Support Group NASA GSFC Code 922, Raytheon ITSS. Land Products @1x – EDC DAAC. 2/96 baseline was 154GB/day 10/99 baseline is 330GB/day (with combined Aqua/Terra products)

azura
Download Presentation

MODLAND Volumes and Loads Status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MODLAND Volumes and Loads Status MODIS Land Science Team Workshop July 15, 2003 Robert Wolfe MODIS Land Team Support GroupNASA GSFC Code 922, Raytheon ITSS

  2. Land Products @1x – EDC DAAC • 2/96 baseline was 154GB/day • 10/99 baseline is 330GB/day (with combined Aqua/Terra products) • Current Collection 4 reprocessing volume is 310GB/day (no combined products) • Land volume will be 240GB/day in Collection 5 reprocessing (reduced L2G volume and other changes) in Fall 2004

  3. Land Products @1X – NSIDC DAAC • 2/96 baseline was 13GB/day • 10/99 baseline is 40GB/day • Current Collection 4 reprocessing volume is 15GB/day • Snow and Sea Ice product volume will be 25GB/day with Collection 5 reprocessing in Fall 2004

  4. Proposed C5 Volume Changes • Objective to estimate: • how to further reduce the size of the L2G products, and • start generating 500m data for a number of products that are currently only produced at 1km • Changes implemented for a Collection Version 5 reprocessing • likely to begin next fall • will mean changes for downstream software • Compression could also help with throughput • will be considered separately • Did not consider: • changing the compositing period for 8-day products • eliminating any products (except when replaced by 500m products) • any Land products archived at NSIDC (only EDC products)

  5. L2G Option A • Reduce the number of L2G products archived from 7 to 2 per tile • incorporate the pointer and geoangle products into the surface reflectance products • Savings from: • removing the "orb_cov" field • eliminating the "line" and "sample" fields • removing overlap between scans • Impact: reduces the L2G products to less than half the current size • From 251 GB/day to 120 GB/day • But … MOD09HK is now a more complicated product because it has both a 1km and 500m grid within the same file

  6. L2G Option B • Turn the L2G product into a "AGG" product • eliminate the need to have both a daily L2G surface reflectance and a daily Aggregated surface reflectance • Same criteria for selecting observations as in the current MODAGG code will be used • maximal angular distribution of the good observations; no more than 4 orbits/day • Non-compact (full) form is used • each layer is stored as a single 2-dimensional array • savings from reducing the need for storing the links between the different resolutions • additional space is needed to go to the non-compact form, and to add the texture field • Impact: the L2G products to about 2/3 of current size • From 251 GB/day to 165 GB/day (vs. 120 GB/day for Option A)

  7. Generating four current 1km products at 500m • Candidate 1km products: BRDF/Albedo, LAI/FPAR, PSN/NPP and Land Cover • Need an intermediate AGG/TEX at 500m or L2G option B • Current candidate 1km product size: 6.5 GB/day • excludes 1km AGG/TEX (current size 51.4 GB/day, reduced volume: 25.7 GB/day) • Estimated new 500m product size: 17.8 GB/day • excludes 500m AGG/TEX (reduced volume: 102.7 GB/day) • Easily fits with L2G Option B or for L2G Option A if AGG/TEX is not archived

  8. Summary

  9. Volume Drivers • Main drivers • Extending Terra and Aqua mission lifetimes three years (from 5 to 8+ years) • Volume growth from ’96 baseline • Archive management issues • Previous collection must be kept for 6 months after current one – tape reuse is more difficult than anticipated • Solution: Technology refresh • Anticipated for about this time • New drives write 3 times as much data on tapes, current tapes are being reused

  10. Strawman MODLAD Volume Recommendations • Reduce size of L2G products using Option A or B and start producing new 500m products (reduces volume by ≥ 27%) • Implement internal HDF compression where feasible • Must consider any impact on end-user analysis tools • Perform technology upgrades to keep tape archive current • Plan for C5 reprocessing at 5.6X (means reprocessing time is about 1 year) – by reducing overall size (including oceans) the bottleneck will be Level 1 processing • Increase size of data-pools and tie data-pools to ordering interface (so data in pool doesn’t need to be staged from tape) • Work with land community to better understand product usefulness

More Related