Multispectral Format from Perspective of Remote Sensing

1. Multispectral Formatfrom Perspective ofRemote Sensing Rulon E. Simmons rulon.simmons@kodak.com (585) 253-5382

2. Outline • Color Issues • Steps to color processing • Color Accuracy in Remote Sensing • Handling color display of non-visible bands • Format issues • Similarities and differences with other multispectral applications • XML for Metadata • Compression

3. Color Issues Contributing Persons: Rulon Simmons Tim Elder Scott Bennett Michael Vaughn

4. Pre-Processing Color Correction Output Preparation Rendering Display Color Processing Approach* RGB raw DRA Channel Balance XYZ  LAB TTC MTFC LAB  XYZ’ RGB’  XYZ XYZ’  RGB” Gamma *Note: This is a standard approach used with digital photography.

5. Calibration Point Black Point Pre-Processing

6. Pre-Processing (Calibration from Digital Counts to Reflectance) • Empirical Line Method* • Using one or more calibration patches • Using best guess of reflectance of one or more objects • Atmospheric modeling (e.g., MODTRAN) *Note: The ELM method of calibration is applicable to any discipline, not just Remote Sensing.

7. Color Correction(Causes of Color Inaccuracies in Remote Sensing) • Multispectral band passes in the visible region not equivalent to HVS response functions • IKONOS, QuickBird • Skylight illumination • Common phenomenon: Materials in shadow appear bluish • Dynamic Range Adjustment (DRA) • Varying approaches produce different color presentations

8. E E = Eye Sensitivity I = IKONOS Sensitivity E E I I I I Color Correction(Problem 1: Different Spectral Sensitivities)

9. Color Correction(Ikonos Vs. HVS Color Example) IKONOS Image Simulated HVS Image

10. Color Correction (GretagMacbeth Color Checker: Truth & IKONOS Simulation)

11. Color Correction(Eckerd’s Blue Roof Turns Purple) HVS IKONOS QuickBird

12. Color Correction(Image Example and Simulation of IKONOS Color) Ground Truth: Digital Camera Simulated IKONOS Color

13. White reference chip in direct sunlight Used to calibrate the ASD spectrometer White reference chip in shadow Illuminated by light scattered by the atmosphere Color Correction(Problem 2: Shadow Illumination)

14. Tone Transfer Curve Luminance (out) Luminance (in) Rendering • Convert from XYZ to LAB (to get to a color appearance space) • Apply a Tone Transfer Curve (TTC) to the “L” channel to stretch midtones while compressing highlights and shadows • Apply Modulation Transfer Function Compensation to the “L” channel to sharpen image • Convert back to XYZ

15. Output Preparation Phosphor Gamut will allow reproduction of most colors seen by IKONOS, but many colors seen by the eye cannot be reproduced by IKONOS. SO HOW CAN YOU DO COLOR CORRECTION?

16. Color Correction Approach • There is no perfect solution, however, all of the problems are addressed by creating a transform that minimizes the Delta E between a set of color patches as seen by the eye and by the sensor • Use real patches were possible • Use synthesized data based on system specs

17. Color Correction(Beyond Visible) • Near IR and other non-visual spectral regions can be handled in the same way as shown on previous slide. (Note: UV is rarely used in Remote Sensing because it is heavily absorbed by the atmosphere.) • The transformation seeks to minimize the Delta E between an acquired or synthetically generated image of a set reference panels and some aim colors • For IKONOS, a “false-color IR” image is produced by mapping the IR channel to the red display, red to the green display, and green to the blue display

18. Color Correction (for “hyper”-visible imagery) • Currently in Remote Sensing there is no standard method of mapping color for systems that have more than three spectral channels covering the visible range of the spectrum • Current visualization packages just display three bands (either user selected or chosen by the program) • A useful option would be to smartly combine the bands to make an image that is as close as possible to what the human visual system would see

19. Format Issues Contributing Persons: Rulon Simmons Bernard Brower

20. Similarities and Differences between Remote Sensing and other Applications of Multispectral Imaging

21. Reconciling Differences • Consider requirements for all disciplines to be a subset of requirements for Remote Sensing • Note 1: This approach does not require that all Remote Sensing issues be addressed in the first release of the standard as long as it is extensible at a later date • Note 2: A basic set of Remote Sensing requirements may not be too difficult to accommodate • Build upon a standard that will be acceptable to all parties in the future • Standardize an approach to recording metadata that can be used within any file format

22. Standard Approach to Metadata (Why Use XML?) • Standard method of coding metadata • Can be used regardless of format • Is human readable as well as machine readable

23. XML for Pre-Processing <PRE-PROCESSING> <DRA> <PARAM NAME=“GAIN” VALUE=“1.5/> <PARAM NAME=“OFFSET” VALUE=“10”/> </DRA> </PRE-PROCESSING>

24. XML for Size <SIZE> <PARAM NAME=“NBANDS” VALUE=“4”/> <PARAM NAME=“NROWS” VALUE=“1000”/> <PARAM NAME=“NCOLUMNS” VALUE=“1000”/> <PARAM NAME=“NBITS” VALUE=“12”/> <PARAM NAME=“SEQUENCE” VALUE=“BIP”/> </SIZE>

25. XML for Location <LOCATION> <PARAM NAME=“COORD PROJECTION” VALUE=“UTM”/> <PARAM NAME=“LAT” VALUE=“XX.XX”/> <PARAM NAME=“LONG” VALUE=“YY.YY”/> </LOCATION>

26. XML for Color Correction <COLOR MATRIX> <PARAM NAME=“RGB TO XYZ” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/> </COLOR MATRIX>

27. XML for Rendering <RENDERING> <PARAM NAME=“XYZ TO LAB” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/> <PARAM NAME=“TTC” VALUE=“X1, Y1, X2, Y2, X3, Y3, …”/> <PARAM NAME=“MTFC” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/> <PARAM NAME=“LAB TO XYZ” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/> </RENDERING>

28. XML for Display(In general would not be included in metadata.) <DISPLAY> <COLOR MATRIX <PARAM NAME=“XYZ TO RGB” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/> <PARAM NAME=“LAB TO RGB” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/> </COLOR MATRIX> <PARAM NAME=“GAMMA” VALUE=“2”/> </DISPLAY>

29. Compression • Compression is required for some, but not all multispectral applications • JPEG 2000 is an emerging international standard that is gaining wide acceptance (can be used in lossy or lossless mode) • JPEG 2000 can be used to stream very large data sets in real time to small computing devices such as PDAs (using the feature that the whole image does not have to be decompressed before it can be sent) • JPEG 2000 can be accommodated within the current NITF/BIIF standard, making it attractive to many currently doing Remote Sensing

30. Conclusions / Recommendations • Use a four-step approach to color management currently used with digital photography • Non-visible channels can be displayed in predictable ways using standard color science • “Hyper”-visible channels can be combined to give a better representation of true color (from HVS perspective) • Multispectral format requirements for Remote Sensing are more similar to than different from other applications • Select a standard that will be acceptable to all disciplines, taking into account widely-used standard formats and one that will likely be used in the future. • Use XML as a standard for recording metadata