Physical retrieval (for MODIS)

1. Physical retrieval (for MODIS) Andy Harris Jonathan Mittaz PrabhatKoner (Chris Merchant, Pierre LeBorgne)

2. Satellite data – pros and cons • Main advantages of satellite data • Frequent and regular global coverage (cloud cover permitting for IR) • ‘Single’ source of data • Many observations • Challenges • Not a direct measure. A retrieval process is required • Single source + many observations means that data must be accurate, or risk swamping the conventional record with erroneous values • Lack of other sources in remote regions to cross-compare

3. MODIS (A)ATSR AVHRR 1975 1980 1985 1990 1995 2000 2005 El Chichon El Niño El Niño Mt Pinatubo Reprocessing of historical data • Unless we have one of these… • …we must reprocess old data to the standard required for climate monitoring Expected SST trend is ~0.2 K/decade Hence requirement is that observing system must be stable to <0.1 K/decade

4. Radiative transfer-based retrievals • The chief advantage of radiative transfer (RT) is that it allows specification of the retrieval algorithm without bias towards the data-rich regions • The in situ data can then act as a random independent sampling of the retrieval conditions • If the observed errors agree with the modeled ones, then high confidence can be placed on the modeled errors in data-sparse regions Is it necessary?Let’s consider the likely errors in the empirical ‘state-of-the-art’ AVHRR Pathfinder SST…

5. Early theory required SST – Ti = kiF(atm) This allowed SST = k2T1 – k1T2 ——————————— (k2 – k1) And hence the “split-window” equation, mystique about channel differences, etc. Only need to assume SST – Ti SST – Tj to get SST = a0 + aiTi Some refinements to account for non-linearity, scan angle: SST=a0+a1T11+SSTbga2(T11–T12)+(SZA-1)a3(T11–T12)

6. Some issues w.r.t. regression-based SST retrievals • Does it matter what form the scan-angle correction takes? • Sec(ZA) term multiplying a channel difference (e.g. NLSST) • Separate SZA-dependent term for each independent variable • How much signal is coming from the surface? • Dependence on water vapor • Geophysical coupling

7. Scan-angle dependence • Simulation using Modtran & 1,358 globally sampled ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5 • In-sample testing of algorithm accuracy

8. Scan-angle dependence • Simulation using Modtran & 1,358 globally sampled ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5 • In-sample testing of algorithm accuracy

9. How much signal? • Simulation using Modtran & 1,358 globally sampled ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5 • The mid-IR ~4 μm channel(s) are powerful…

10. How much signal? • Simulation using Modtran & 1,358 globally sampled ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5 • The mid-IR ~4 μm channel(s) are powerful… • Use all surface-sensitive channels…

11. Simulated Pathfinder Retrieval Errors • • AVHRRPathfinder Oceans is current “state-of-the-art” SST • Distribution & quantity of matchup data change over time • How does this affect SST retrieval performance? • • The effect can be modeled using the following • Pathfinder matchup dataset • Full-resolution ERA-40 atmospheric and surface data • Fast-forward radiative transfer model (CRTM) • • Process is as follows • Pathfinder matchup information is used to select the appropriate atmospheric profiles for each month • SSTs and profiles are used to generate ToA radiances which are then used to generate algorithm coefficients • Retrieval coefficients are then applied to full month of radiances generated from cloud-free ERA-40 data

12. Pathfinder Mean Bias 1985 – 1999

13. Pathfinder Mean Bias 1985 – 1999

14. Changes in sampling 1985 – 1999 Physically-based methodology models retrieval effects explicitly –apply to whole AVHRR record to improve consistency of SST record for climate studies

15. Using Radiative Transfer in SST Retrieval • Since algorithm biases can be modeled, just subtract them • Has been employed operationally (LeBorgne et al., 2011) • Has little impact on S.D., no impact on sensitivity, but does improve bias • Dependent on model input (!) • Accuracy of NWP profile • Not in model state not corrected (e.g. aerosol)

16. Direct removal of local retrieval bias [From presentation by Pierre LeBorgne @GHRSST XIII, Tokyo] • Example - Methodology of Meteo-France:

19. “Physical” Retrieval:Estimate adjustment to “guess” SST • Based on local linearization, i.e. Δy= KΔx • Δy =yo – yg , K, yg from RTM + NWP • x is reduced state vector, at least [SST, TCWV]T • K is Jacobian of partial derivatives of ygw.r.t. x • Ordinary unweighted least squares solution: Δx= (KTK)-1KTΔy [ = GΔy ] • Potential issues: • G may be ill-conditioned (noise amplification) • K may be “incorrect” • Δymay be biased, i.e.Δy ≠ 0when Δx= 0 • e.g. RTM, calibration

20. Incremental Regression • Only use Δy and simply derive a regression coefficient-based operator to retrieve ΔSST (Petrenkoet al.) • Do not retrieve a water vapor adjustment • Side-step issues of calibration, errors in RTM calculation, etc. • Essentially no control over algorithm sensitivity • Noisy input will suppress coefficients • No means of estimating uncertainty • Cf. other physical methods • No means of iterating solution • No adjustment to model state (apart from SST) • Revision of other processing elements requires recalculation of coefficients

21. The “Traditional” ApproachOptimal Estimation • E.g. Merchant et al. for SST • Primary issue is dealing with the ill-conditioned nature of gain matrix G • Use prior estimates of uncertainty in RTM, model state and instrument noise • RTM + satellite errors combined into one covariance matrix Se • Error in model state propagated through covariance for reduced state vector Sa Δx = (KTSe-1K +Sa-1)-1KTSe-1Δy • Error covariance matrices function as a regularization operator on the gain matrix • Reduces the “condition number” • Any regularization will reduce sensitivity to true SST change

22. Extended OESST [From presentation by Chris Merchant @GHRSST XIV, Woods Hole]

23. Extended OESST

24. Extended OESST - Results

25. MODIS - OSTIA, Feb 2012

26. [SST (night) – OSTIA] cf. modeled • Modeled clear-sky NLSST bias is February average for 1985 – 1999 • No aerosols • Several features common to both observed and modeled biases • Boundary currents  over-estimate of gamma • Cold tongue  under-estimate of gamma • High SST but low water vapor • (Decoupling of SST and air temperature) • Modeling shows annual cycle of bias

27. Hence Lat-band coefficients, etc. • Employ a physical retrieval methodology • MTLS

28. Modified Total Least Squares:A Deterministic Regularization • Based on local linearization, i.e. G= (KTK + rI)-1KT • r is dynamically calculated regularization strength r= (2 log(κ)/||Δy||)σ2end • κ is the condition number of K • σ2end is the lowest singular value of [K Δy] • ||Δy|| is L2-norm of Δy • Regularization uses observation-based estimate of noise amplification • Closer to “actual” error for retrieval/observation conditions on a case-by-case basis

29. MTLS applied to MODIS • Validation against iQUAM • MTLS: -0.02±0.36K SST(night): -0.21±0.47 K • Reduction of ~0.3 K of independent error • (SST4: -0.24±0.44 K) • (Difference in bias due to skin effect)

30. Uncertainty Estimation • OE uses “goodness of fit” χ2 = (KΔx – Δy)T[Se(KSaKT + Se-1)Se]-1(KΔx– Δy) • MTLS formulation for total error, e ||e|| = ||(MRM – I)Δx|| + ||G||||(Δy - KΔx)||KT • Δx should be true value, but is substituted by retrieved quantity • MRM is model resolution matrix (a.k.a. averaging kernel)

31. Summary • Inherent limitations in NLSST • MODIS has 16 TIR channels • Currently, only a very few used for SST retrieval • 1st cut MTLS physical retrieval shows promise • Initial result subject to MODIS cloud mask • Extra channels permit more complex retrieval vector • WV scale height, air temperature, aerosol, etc. • Multiple iterations • Smoothed inputs (Merchant) • OE and MTLS allow prospect for direct uncertainty estimation