System Level Approach to Satellite Instrument Calibration

1. System Level Approach to Satellite Instrument Calibration Space Dynamics Laboratory at Utah State University: Joe Tansock, Alan Thurgood, Gail Bingham, Nikita Pougatchev, Randy Jost NIST: Raju Datla Ball Aerospace & Technologies Corp.: Edward Knight

2. Outline • Calibration Philosophy • “Specmanship” • Workshop to Improve Calibration • Calibration Planning • Subsystem/Component measurements • Ground Calibration • On-Orbit Calibration • Internal and external calibration sources • Satellite Instrument Validation and End-to-end Error Model • Summary

3. Calibration Philosophy • Calibration • Provides a thorough understanding of sensor operation and performance • Verifies a sensor’s readiness for flight • Verifies requirements and quantifies radiometric and goniometric performance • Provides the needed tools to convert the sensor output to engineering units that are compatible with measurement objectives • Provides traceability to appropriate standards • Estimates measurement uncertainties

4. Calibration Philosophy – Cal Domains • A complete calibration will address five responsivity domains • Radiometric responsivity • Radiance and irradiance traceable to NIST • Response linearity and uniformity corrections • Nominal/outlying pixel identification • Transfer calibration to internal calibration sources • Spectral responsivity • Sensor-level relative spectral response • Spatial responsivity • Point response function, effective field of view, optical distortion, and scatter • Temporal • Short, medium, and long-term repeatability, frequency response • Polarization • Polarization sensitivity

5. Calibration Philosophy – Cal Domains • The goal of calibration is to characterize each domain independently • Together, these individually characterized domains comprise a complete calibration of a radiometric sensor • Domains cannot always be characterized independently • Complicates and increases calibration effort • Example: Spectral spatial dependence caused by Stierwalt effect • Calibration parameters are grouped into two convenient categories • Calibration equation • Converts sensor output (counts, volts, etc.) to engineering units • Radiometric model • All parameters not included in calibration equation but required to meet calibration requirements

6. Calibration Philosophy – Phases of Cal • A complete and methodical approach to sensor calibration should address the following phases:

7. Establishment of Good Specifications Improves Calibration • Programs often start with a requirement such as • “The instrument shall be radiometrically calibrated to a 3% absolute error, 1.5% band to band error, and a 0.25% intra-band pixel to pixel error” • The designers are then asked for cost, schedule, and risk to meet this requirement, which could vary dramatically • E.g., is “error” a 1-sigma or 3-sigma requirement? • Furthermore, incomplete, changing, or impossible specifications are often the cause of cost and schedule overruns

8. So, What Makes a Good Specification? • A good specification clearly communicates what must be accomplished • To an audience that is reading (vs. oral communication) • No other “clues” to help understanding • To an audience that may not be able to ask questions easily • Example: reading the specification at the end of the program after there’s been personnel turnover • To an audience that may have a different background, training, or understanding of the problem than the author Also see E. Knight, “Lessons Learned in Calibration Specsmanship, CALCON 2005 proceedings.

9. Lessons Learned in Specifications • Lessons • Cover all domains (spectral, spatial, temporal, radiometric, polarization) • Including interactions and “worst case” for requirements • Scrub for ambiguity • Use mathematical equations whenever possible to define requirements • Have at least one idea for implementation in mind when writing the specification • Or upon first round of review/questions • Have at least one idea for verification in mind as well • Conclusion • The chance of an instrument • Being “poorly calibrated” • Overrunning cost and schedule targets can be reduced with improved calibration specsmanship

10. Workshop to Improve Quality of Calibration • EO/IR Calibration & Characterization Workshops held in Feb 2005 and March 2006 at SDL/USU • Envision self governing community based organization with goal of improving calibration for all participating organizations • Workshop Objectives • Explore ways to improve the quality of IR/Visible/UV measurements, community-wide, based on an ISO 17025 standard, as pioneered by the RCS community • Benefits based on experiences of RCS community • Measurably and quantifiably improve the quality of measurements made in the community • Facilitate data comparison between sensors, systems, facilities, programs and customers • Increase in customer confidence in measurement results due to improved: accuracy, uncertainty, repeatability, comparability, consistent documentation

11. Workshop to Improve Quality of Calibration • Universal Agreement • There is an unmet need that can not be addressed by any one organization • Intermediate results will continue to be presented at annual CALCON (Calibration Conference) • For more information • CD available containing the presentations and recommendations of the 2005 and 2006 workshops. • http://www.sdl.usu.edu/conferences/eo-ir/ • Based on attendee feedback provided at the 2006 workshop, we have started the planning process for the next workshop, to be held Spring 2007, at NIST, in Gaithersburg, MD

12. Calibration Planning • Calibration planning • Start as soon as possible (I.e. requirements definition, concept design, sensor design, etc.) • Influence sensor design to allow for efficient and complete calibration • Encourages optimum sensor design and calibration approach to achieve performance requirements • Planning phase can help shake out problems • Schedule and cost risk can be minimized by understanding what is required to perform a successful calibration • Calibration equipment needs should be identified early to allow time to build and test any required new equipment

13. Identify instrument requirements that drive calibration Identify calibration measurement parameters and group into: Calibration equation Radiometric model Flow calibration measurement parameters to trade study Schedule Sensor design feedback GSE hardware & software Measurement uncertainty Risk Perform trade study to determine best calibration approach Mission Requirements Instrument Requirements Calibration Measurement Parameters • Calibration Equation • Radiometric Model Sensor Design Cost & Schedule Calibration Planning GSE Hardware & Software Measurement Uncertainty Risk Calibration Planning

14. Subsystem/Component Measurements • Subsystem and/or component level measurements • Help verify, understand, and predict performance • Collect Parameters for the Radiometric Model that can't be measured well at the system level • Minimize schedule risk during system assembly • Identifies problems at lowest level of assembly • Minimizes schedule impact by minimizing disassembly effort to fix a problem • System/Sensor level model development andmeasurements • Allow for the development of Measurement Equation and Performance prediction • Allow for end-to-end measurements • Account for interactions between subsystems and components that are difficult to predict

15. Subsystem/Component Measurements • Merging component-level measurements to predict sensor level calibration parameters may bring to light systematicsystem-level uncertainties A,B • Comparison of System-level estimate using component measurements with end-to-end measurement of SABER relative spectral responsivity (RSR) • 9 of 10 channels < 5% difference • 1 channel 24% difference (reason unknown) • Helps to resolve and correct for component degradation and sensor performance after launch A.)Component Level Prediction versus System Level Measurement of SABER Relative Spectral response, Scott Hansen, et.al., Conference on Characterization and Radiometric Calibration for Remote Sensing, 1999 B.) System Level Vs. Piece Parts Calibration: NIST Traceability – When Do You Have It and What Does It Mean? Steven Lorentz, L-1 Standards and Technology, Inc, Joseph Rice, NIST, CALCON, 2003

16. Engineering Ground Calibration • Engineering calibration • Performed before ground calibration • Perform abbreviated set of all calibration measurements • Verifies GSE operation, test configurations, and test procedures • Checks out the sensor • Produces preliminary data to evaluate sensor performance • Feedbacks info to flight unit, calibration equipment, procedures, etc. • Engineering calibration data analysis • Evaluates sensor performance, test procedures, calibration hardware performance and test procedures • Based on results of engineering calibration, appropriate updates can be made to prepare for ground calibration data collection

17. Ground Calibration • Provides complete calibration needed to meet related requirements • Is performed under conditions that simulate operational conditions for intended application/measurement • Careful in-lab calibration minimizes problems that arise after launch • Minimizes risk of not discovering a problem prior to launch • Promotes mission success during on-orbit operations • For many sensor applications • Detailed calibration is most efficiently performed during ground calibration • On-orbit calibration will not provide sufficient number of sources at needed flux levels • Operational time required for on-orbit calibration is minimized • Best to perform ground calibration at highest level of assembly possible • Sensor-level at a minimum is recommended

18. Extending Calibration to Operational Environment • Calibration continues after ground calibration Sensor Design/Fabrication Ground Calibration On-Orbit/Field Operations Internal Calibration Source Response Trending On-Orbit/Field Calibration/Verification • Internal Calibration Source Response Trending • Trend sensor response to quantify relative response changes over time • Source types • Blackbodies, glow bars, diffusers, lamps, etc. • Ensure source is stable and repeatable for sensor operational life

19. Internal Calibration Sources • Challenges • Ensure calibration source is stable and repeatable for sensor operational life • Ideally, calibration source should use same optical path as external measurements • Detailed trade to determine best approach is needed for each specific application • Considerations => source type, flux level,configuration,power, space, and weight limitations, etc. • Sources of variability • Temperature stability and/or temperature measurement • Emissivity changes • Thermal variations (external and internal) • Separate drift in observed response between calibration source and sensor response • Control and/or monitor electronics • IR internal calibration source developments are required to achieve stringent stability requirements of many climate change measurements

20. On-Orbit Calibration Verifies Cal and Quantifies Uncertainty • Track, trend, and update calibration throughout a sensor’s operational life • In addition to internal calibration sources make use of external calibration sources • External On-orbit sources • Standard IR stars • Stars aBoo, aLyra, aTau, aCMa, bGem, bPeg • Celestial objects • Moon • Planets provide bright variable sources • Asteroids, etc. • Sometimes you have to be creative: • Off-axis scatter characterization using the moon • Other techniques • View large area source located on surface of earth (often termed vicarious calibration) • Cross-calibration between sensors • Use of atmospheric lines • Etc.

21. Satellite Instrument Validation • The purpose of validation is to assess actual accuracy and precision of Satellite Instruments by comparison with validating measurements • Apparent differences in results between validating and measurement system • Satellite and validation data are not co-located in time and space • Satellite and validating system have different vertical and horizontal resolution • Satellite and validating system have finite accuracy and repeatability • Physical measurement differences (I.e. spectral, sensor, platform, etc.) • Validation Assessment Model makes comparisons more accurate by understanding and accounting for theses differences • Make results comparable • Validation Assessment Model can be used as a tool to better understand the tradeoff between validation approaches

22. ˆ ˆ y x val val End-to-End Error Model Overall Concept Smoothing Parameter Noise Retrieval Parameter Error - δb Noise – ε Instrument True Profile xsat Radiance ysat SDR ˆ y EDR ˆ x Performance Assessment Validation Assessment ModelReconcile differences to make results comparable ATMOSPHERE True Profile xval xval yval Validation System Radiosondes, Aircraft Measurement Systems, Cross-Calibration, etc.

23. Summary • Calibration Philosophy • What does calibration provide • Calibration domains • Phases of calibration • Planning through operational environment • Importance and benefit of good specsmanship • Facilitate clear communication and minimizes risk of failure • Workshop to improve quality of calibration • Community wide participation working to improve calibration • Calibration Planning • Address all phases of calibration as early as possible • Specification and design phases

24. Summary (cont) • Calibration Measurements • Subsystem/Component Measurements • Minimizes schedule risk and facilitates development of instrument model and measurement equation • Engineering Calibration and Calibration • Methodical and careful approach leads to efficient and thorough calibration • Extending Calibration to Operational Environment • Internal calibration sources (I.e. in-flight internal sources) • Challenges and need for improvement • External on-orbit sources • External sources and need for improvement • Satellite Instrument Validation • Overall concept and the need for validation assessment model to account for differences in space, time, resolution, etc.