Inter-comparison and Homogenization of TOLNet Lidar Algorithms

1. TOLNet lidar algorithms inter-comparisonand homogenization Thierry Leblanc, JPL/TMFJohn Sullivan, UMBC/GSFCShi Kuang, UAH Russell DeYoung, LaRC Denis Pliutau, LaRC Chris Senff, NOAA/ESRL

2. Algorithm intercomparisons: Why and how? Why? 1- To ensure consistency of the lidar measurements when two or more instruments are deployed on a field campaign 2- To ensure consistency of the lidar measurements when two or more instruments contribute data to the same network (TOLNet, NDACC, etc.) 3- To ensure consistency of metadata such as vertical resolution and uncertainties How? 1- By documenting thoroughly on the lidar instruments specs, configuration and performance expectations 2- By using “controlled” measurement conditions, i.e., by simulating lidar raw data corresponding to a known atmospheric state (forward model) 3- By retrieving ozone profiles from the simulated raw data, and comparing them to the “true state” (inverse model)

3. Validation of algorithms: operating principle

4. Validation of algorithms: Simulated signals Simulating the raw signals: Done for all instruments Simulations done for 10-minutes Example for UAH: UAH (RO3QET) signals 16-channels Analog and Photon-counting New configuration with no SIN Daytime The “Rolls Royce” of TOLNet 

5. Validation of algorithms: Simulated signals Example for GSFC TROPOZ: GSFC (TROPOZ) signals 8-channels Photon-counting only Daytime

6. Validation of algorithms: Simulated signals Example for LaRC LMOL: LaRC (LMOL) signals 4-channels Analog and Photon-counting Nighttime (daytime also simulated, not shown)

7. Validation of algorithms: Simulated signals Example for JPL-TMF (old config): JPL-Table Mountain (TMT) signals Configuration up to 2012 4 channels + 355 strato. channel Photon-counting only 75-m sampling resolution Nighttime

8. Validation of algorithms: Simulated signals Second example for JPL-TMF: JPL-Table Mountain (TMT) signals New configuration: since 2012 6 channels + 355 strato. channel Photon-counting only 7.5-m sampling resolution Nighttime

9. Validation of algorithms: Separating numerical errors from algorithm errors Detecting small errors inside an algorithmis not easy to achieve numerically: To identify small errors (5% and less) owed to specific signal corrections,we first must minimize vertical filtering As a result we must remove detection noise (Poisson noise), then workwith a 3-point central difference when we apply the DIAL equation However…

10. Validation of algorithms: Separating numerical errors from algorithm errors Detection hardware and softwarerecords long-integers only ! Numerical discretization introducesartificial signal steps if statistical noiseis not present Artificial steps alter the retrieval

11. Validation of algorithms: Separating numerical errors from algorithm errors Numerical errorspropagate to retrieval  Vertical filteringmust be applied

12. Validation of algorithms: Separating numerical errors from algorithm errors Example aftervertical filtering isapplied(compare withprevious slide) Next step is tocombine multipleranges at suitablealtitudes

13. Validation of algorithms: Filtering and combining ranges should do the trick.. Example shown is nowafter vertical filteringand after combiningthe different ranges Looks good right? Not so so….

14. Validation of algorithms: Test 1: Simulated signals contain no correction Same retrieved profile asprevious slide, butnow compared to original“true atmospheric state” UAH retrieval shows+/-3% error intransition layer fromone range to another(red curve) New JPL retrieval “LidAna”also shown for reference(blue) This comparison shown isfor raw signals withno correction of any type

15. Validation of algorithms: Test 2: Saturation correction included The bad news: UAH retrieval still shows+/-3% error intransition layer fromone range to another The good news: No apparent impact ofsaturation correction(max of -1% at 7 km)

16. Validation of algorithms: Test 3: Rayleigh extinction correction included The bad news: UAH retrieval still shows+/-3% error intransition layer fromone range to another The good news: No apparent impact ofRayleigh extinctioncorrection

17. Validation of algorithms: Using simulations for uncertainty estimates Example for UAH RO3QET Back to retrievals usingsignals with Poissonnoise: Multiple ranges combinedshow range dependenceof uncertainty estimates Combined profile: Minimizes uncertainty

18. Validation of algorithms: Using simulations for uncertainty Example for UAH RO3QET Detection noise: Relative magnitude ofuncertainty owed todetection noise andto background noiseextraction dependson range

19. Validation of algorithms: Using simulations for uncertainty Example for UAH RO3QET Saturation correction: Not always decreasingwith altitude !(see ISSI Team Report) Ozone absorptioncross-sections: Here: cross-sectionsassumed uncorrelated

20. Validation of algorithms: Using simulations for uncertainty Example for UAH RO3QET T/p uncertainty:in this case: 0.5 K and0.1 hPa Rayleigh extinction:Uncertainty does notdepend on range if DIALwavelengths are identical(wavelength-dependenceonly)

21. Validation of algorithms: Comparing the potential of the TOLNet lidars 4 TOLNet lidars(TOPAZ not included) Used same “trueatmospheric state” forALL stations(grey curve) Each lidar instrumentyields its own sensitivity  The retrieved profilesare different

22. Validation of algorithms: Comparing the TOLNet lidars potential 4 TOLNet lidars(TOPAZ not included) TROPOZ and RO3QET yieldsmallest uncertaintybelow 4 km JPL/TMF old-configurationyields smallest uncertaintyabove 6.5 km

23. CONCLUSION Where do we go from here ?

24. Where do we go from here ? This iswherewe are now

25. Where do we go from here ? This isour goal THANK YOU

26. BACKUP SLIDES

27. Motivation for consistency The problem: Multi-decadal (?) span of measurements from multiple instruments  can lead to misinterpretation of results from correlative data, especially if precise knowledge of vertical resolution and/or uncertainty is required  may produce variations/inconsistencies in data/metadata content on the long-term The solution: Brainstorm early in the network development stage to: - Standardize how vertical resolution is reported - Standardize how uncertainty is expressed and reported - Define a sound, common framework in which lidar data remain fully traceable - Define a sound, common framework in which lidar instruments should operate Keep in mind: Standardize the definitions, not necessarily the algorithms!

28. Using leverage from NDACC and GRUAN

29. ISSI Team on NDACC Lidar Algorithms ISSI = International Space Science Institute, Bern, Switzerland Project Objective: Assess and standardize the reporting of vertical resolution and the propagation and reporting of uncertainty in the data processing algorithms of the NDACC lidars Contributors: ~10 NDACC lidar groups (USA, Canada, Germany, France, Italy, Netherlands) ~ 5 different communities (lidar, Satellite/GEOMS, EARLINET, Microwave, GRUAN) Timeline/milestones: Dec. 2010: First Meeting, Critical Assessment of existing definitions June 2011: Tools for NDACC-Standardized Vertical Resolution implemented Sept. 2012: Third meeting to finalize NDACC-Standardized definition for Uncertainties”Early 2013: Expected end of Project  Vert. resolution and uncertainty implemented Full details: http://www.issibern.ch/teams/ndacc

30. GRUAN Lidar Programmes Definition The overall infrastructure underlying the lidar measurement and the subsequentproduction of a GRUAN lidar product, from data acquisition to data product management Domains of application - Instrumentation setup, modification and maintenance of the instrumentation - Standard operating procedures for raw data acquisition and lidar calibration - Thorough and homogenized upload of the raw data and meta-data onto the GRUAN lidar data handling center for subsequent centralized processing Mandatory components - A lidar instrument  - Dedicated and motivated staff  - An Individual GRUAN Lidar Instrumentation and Measurement Protocol (IGLIMP) - The LidarRunClient utility for traceable data and meta data recording and upload - A centralized GRUAN Lidar Analysis Software Suite (GLASS) for consistent data processing

31. How does TOLNet compare to NDACC and GRUAN?… and what are the consequences for consistency? TOLNet 5 instruments sites, US-based Target species: Trop. O3 (and aer.) Target altitudes: Troposphere Instruments: Lidar Timescales: Short-term and more Approach: Undefined Data analysis: PI-independent, so far Data archive: Centralized Data format: Standardized Traceability: Not addressed Quality control: Not addressed Database: 0-2 years Overall quality: Unknown  Consistency: Not addressed (yet) 70 sites, international Target species: O3 and related Target altitudes: UT, S, LM Instruments: Lidar and more Timescales: Long-term and more Approach: “Bottom-up” Data analysis: PI-dependent Data archive: Centralized, basic Data format: Semi-standardized Traceability: One meta data file Quality control: Occasionally: Campaigns and algorithms Database: Multi-decadal Overall quality: high to unknown  Consistency: Low 10 sites, international Target species: H2O, temperature, O3 Target altitudes: T, S Instruments: RS , GPS, and more Timescales: Long-term and more Approach: “Top-down” Data analysis: Centr. “GRUAN Product” Data archive: Centralized, expandable Data format: Semi-standardized Traceability: Complex structure Quality control: Routine: At SOP level and at processing Database: A few years for RS and GPS Overall quality: high to unknown  Consistency: Depends on instrumentation

32. 20+ MORE SLIDES TO SUPPORTSTARTING DISCUSSION Including: GRUAN Lidar Programme detailsThe GRUAN LidarRunClient utility to record meta dataNDACC-standardized vertical resolution definitionsNDACC-standardized vertical resolution implementation toolsNDACC-standardized treatment of uncertaintyAlgorithm validation tools Also available on Thierry’s Laptop: GRUAN Lidar Guide Draft (under revision)ISSI-Team Report DraftDemos for algorithm validation tools

33. NDACC-standardized vertical resolution, definition 1 From Digital Filter theory • Use cut-off frequency of digital filter (DF)(Transfer Function (TF) of DF calculatedfrom the filter’s coefficients) • If filter used is differential (DIAL):Cut-off frequency is calculated from the product of the TF by sampling frequency • If data filtering occurs multiple times:TF is calculated for each occurrenceThe product of all TFs is used for final vertical resolution

34. NDACC-standardized vertical resolution, definition 2 From Impulse Response (IR) Use Full-Width at Half Maximum (FWHM) of the Response to a Finite Impulse (Dirac “Delta” function) convolved with the filter’s coefficients. If the filter used is differential (DIAL)The Impulse Delta Function is replaced by theHeaviside Step Function. If data filtering occurs multiple times:The output response of each occurrence is used as input impulse for the next occurrence The final vertical resolution is the FWHM of the last calculated response.

35. NDACC-standardized vertical resolution: How is it implemented?

36. NDACC-standardized vertical resolution: How is it implemented?(cont.) Tools validated in 2011 currently under implementation Currently written in 3 languages:IDL, MATLAB, and FORTRAN PYTHON language planned if need be Simple, ready-to-use subroutines: NDACC PIs do NOT need to modify their data processing software, only need to include the subroutines and re-compile. Tools’ validation completed in 2011:Usedsimulated lidar signals and backward/forward model runs to ensure the tools’ outputs were correct and consistent Similar tools are planned (2014) for the standardized expression and reporting of uncertainties

37. NDACC-standardized treatment of uncertainty: Terminology Agreed Definition and Terminology: 1. The NDACC-lidar-standardized uncertainty is the combined standard uncertainty u as defined by the BIPM technical documents JCGM-100 (2008) and JCGM-200 (2008) 2. u must be obtained from known, traceable standard uncertainty components ux associated with multiple sources of uncertainty x 3. If no traceable standard uncertainty is available for x, ux must be taken as the standard deviation sx of the normally distributed probability function describing x 4. The contribution of multiple standard uncertainty components ux to the combined standard uncertainty u must follow the law of propagation of variance, i.e.: If y is defined as a function of the independent variables x1, x2,.., xn : And if un = u(xn) is the standard uncertainty for the source xn,then the combined uncertainty for y is:

38. NDACC-standardized treatment of uncertainty: Physical constants Reference values and their uncertainty From the updated list of the CODATA Working Group by the BIPM Accessible at: http://www.bipm.org/extra/codata/ NDACC/ISSI “zero-uncertainty” for standardized physical constants A physical constant is considered as a number with “zero-uncertainty”, i.e.,its value, as reported by the CODATA Working Group of the BIPM,` shall be roundedat such decimal level that addition or subtraction of its uncertainty (as reported by theCODATA Working Group of the BIPM) does not incur any change in its rounded value. Example The molar gas constant R value, as reported by BIPM/CODATA, is: R = 8.314 4621(75) Jmol-1kg-1 with a relative uncertainty of dR /R = 9.1 10-7 We therefore have: R-dR = 8.314 4613 Jmol-1kg-1 and R+dR = 8.314 4631 Jmol-1kg-1 The NDACC/ISSI standardized constant for R will therefore be: R = 8.314 46 Jmol-1kg-1 With the above NDACC/ISSI “zero-uncertainty” approach, there is nouncertainty propagation associated with fundamental physical constant

39. NDACC-standardized treatment of uncertainty: Propagation Proposed Approach: 1. Identify and introduce separately all uncertainty sources that are assumed to be uncorrelated 2. Propagate each of these components in parallel , yet estimating correlation degree between altitude bins and between channels, within each component 3. Compute ozone combined standard uncertainty only after reaching very last stage of processing Next 3 slides show where uncorrelated uncertainty components areintroduced into tropospheric ozone DIAL data processing chain

40. Funding Generic DIALflow chart uRAW Uncertainty (detection noise) Uncertaintysourceswill appear one afteranother ut e.g., SOUT=SIN/(1-t SIN) uSAT e.g., N(z)=b0+b1z ub0 , ub1 uBKG uOVER uMERGE uRAW uSAT uBKG uOVER uMERGE

41. uRAW uSAT uBKG usO3 uOVER uMERGE uDsO3 usRay uNa uDsRay uNa usIG uNIG uDsIG uNIG

42. uAer uRAW uSAT uBKG uOVER uMERGE uDsO3 uDsRay uNa uO3 NO3 uDsIG uNIG uTa uAER upa uTa , upa uTa upa O3 VMR uO3VMR

43. NDACC-standardized treatment of uncertainty: Sources Case 1: Case 2: Case 3:

44. Consistency in Standard Operating Procedures? (SOPs)

45. LidarRunClient Start Page

46. LidarRunClient “Observation” page

47. LidarRunClient “New Measurement” page

48. LidarRunClient “Instrumentation” page

49. LidarRunClient “Channels” page

50. LidarRunClient “Measurement Conditions” page