NASA’s: Science Data Purchase

1. NASA’s: Science Data Purchase IKONOS, ETM, MODIS NDVI: comparison Jeff Morisette, MODLAND, SSAI Positive Systems for Appalachian Transect Rob Sohlberb, MODLAND, UMd Report from Stennis Space Center on SDP validation activities Mary Pagnutti, SSC One Ikonos DEM/Stereo Pair for Barton Bendish site J. Peter Muller, MODLAND, ULC

2. One approach to scaling Comparing ETM+, IKONOS, and MODIS NDVI products Framed in the context of statistical hypothesis testing J. Morisette

3. General validation procedure: correlative analysis(slide from 1999 validation mtg.) Field data “Tasked” acquisitions: Airborne and high res. Satellite Automatic acquisitions: reference data and products to be validated Point Fine resolution Fine resolution Compare: Need to consider all three elements as samples from unknown distributions, use each component to estimate the respective distribution, and compare distributions • points to pixels • parameters and distributions • relationships • surfaces

4. Spectral Bands, Red and NIR AVHRR grass reflectance* MODIS ETM+ IKONOS Difference may be important: Gitelson and Kaufman, 1998; Bo-Cai Gao, 2000; which compared MODIS to AVHRR and found large differences in NDVI *ASD spectrum from grass area near GSFC

5. Study Area: Konza Prairie Data: MODIS daily products: Sept. 11 500m surface reflectance 500m pointer file 1km viewing geometry LDOPE tools to combine (available through EDC EDG) ETM+, Sept. 11 IKONOS, Sept. 15 Aeronet (Meyer) (available through Konza Prairie Core Site web page) Vermote et al.’s Six S code (for ETM+ and IKONOS)

6. IKONOS area on MODIS 500mand ETM+

7. Subset area in Sinusoidal Projection

8. Sampling with MODIS “Tile Mapper” Done for both IKONOS and ETM+

9. Sampled imagery ETM+ IKONOS (30m)

10. Comparison at Multiple-scales ETM+ 14, 16 n=224 Considering all three as variable and subject to errors, consider MODIS pixel relative to the distribution from the higher resolution data MODIS Pixel 1, 1 IKONOS 116, 120 n=13,920

11. IKONOS vs ETM+ Correlation = .5639 Reject hypothesis of zero correlation Using standard Pearson method (p value ~0)

12. IKONOS vs MODIS Correlation = .3114 Reject hypothesis of zero correlation Using standard Pearson method (p value ~0)

13. ETM+ vs MODIS Correlation = .3401 Reject hypothesis of zero correlation Using standard Pearson method (p value ~0)

14. Do the data follow a normal distribution? Null Hypothesis: Normally distributed Test: Kolmogorov-Smirnov Goodness-of-Fit Test: MODIS data: Reject (p = .0079) ETM+ at 500m: Reject (p = .0004) IKONOS at 500m: Reject (p ~ 0) ETM+: Reject (p ~ 0) IKONOS at 30m: Reject (p ~ 0) So, should consider testing correlation with non-parametric methods.

15. Non-parametric correlation Null Hypothesis: Zero Correlation Test: Spearman's rank correlation IKONOS vs ETM+: Reject (rho = .5791, p ~ 0) (corr = .5639) IKONOS vs MODIS: Reject (rho = .3099, p ~ 0) (corr = .3114) ETM+ vs MODIS: Reject (rho = .3362, p ~ 0) (corr = .3401) But we still might want to question the hypothesis being tested.

16. Test for Paired Differences Null Hypothesis: average paired difference is zero Test: T test (assume normality and homogeneity of variance) Test: Wilcoxon Rank Sum Tests IKONOS vs ETM+ IKONOS vs MODIS Reject all three pair-wise combination ETM+ vs MODIS based on either test. So, for these data we are somewhere in the middle: There is positive correlation, but the average difference is not zero

17. Normalized differences to include variability in validation data MODIS – IKONOS(average) = “z score” Std. Dev (IKONOS ave.)

18. Z score analysis IKONOS vs ETM

19. Z score analysis IKONOS vs MODIS

20. Z score analysis ETM vs MODIS

21. Do the z-scores follow a normal distribution? Null Hypothesis: Normally distributed Test: Kolmogorov-Smirnov Goodness-of-Fit Test: Z from IKONOS vs ETM+: Reject (ks = 0.1955, p ~ 0) Z from IKONOS vs MODIS: Reject (ks = 0.0537, p = 0.0142) Z from ETM+ vs MODIS: Reject (ks = 0.0538, p = 0.013) So, should consider testing z-scoresw with at least both parameteric and non-parametric methods

22. Test of z-score “centered” on zero Non-Parametric Null Hypothesis: Median value is zero Test: Wilcoxon Signed Rank Sum Tests Z from IKONOS vs ETM+: Reject (Z =-46.493, p ~ 0) Z from IKONOS vs MODIS: Reject (Z = -9.9305, p ~ 0) Z from ETM+ vs MODIS: Reject (Z = -6.2677, p ~ 0) Parametric Null Hypothesis: Mean value is zero Test: T test Z from IKONOS vs MODIS: Reject (t = -10.143, p ~ 0) Z from ETM+ vs MODIS: Reject (t = -5.4727, p ~ 0)

23. Conclusions • Assumption of normality is not always met • Non-parametric methods are available • Z-score method shows one possible way to scale up; which incorporates variability and considers the validation data with respect to its distribution • There is a fundamental difference between the null hypothesis of the correlation being zero and the difference being zero • There is closer statistical agreement between MODIS and either IKONOS and ETM+ than between IKONOS and ETM+ • There is a difference between statistical and practical difference

24. Comments • ETM+, IKONOS, MODIS and Sun photometer data were easily available • Major difficulty was ISIN projection and georeferencing – coordination of Jacqueline Le Moigne, GSFC might prove helpful. • Results are planned to be communicated in the validation article in the Special Issue of RSE.