Digital RAW Photography – Introduction September 2006

1. Digital RAW Photography – IntroductionSeptember 2006 • What is RAW and should I care?  • REVIEW - How most still camera sensors capture a photo. • (Sigma’s Foveon sensor works differently, but we’ll ignore it in this discussion)

2. REVIEW • The sensor is capable of only seeing gray values. It’s an analog device that records light (photons) as an electrical charge. The charge from each picture element (pixel) is amplified and converted to a 10, 12 or 14 bit digital signal via the ADC (analog to digital converter) with up to 1,024, 4,096 or 16,384 levels, or shades of gray.

3. REVIEW • The RAW fileis a dump of the gray scale data that the CCD captures. It is a relatively compact file with far more color information than the JPG/TIFF files produced inside the camera. We will see why this is so a bit later • Cameras that actually write the raw format to a data file use 12 or 14 bit ADC. Cameras that only record TIFF and/or JPG files use a 10 or 12 bit ADC.

4. REVIEW • How does the Raw data translate into an RGB image? • On top of the CCD's array of pixels (also called photo sites) is fixed a colored filter with millions of little colored cells, each aligned with a single CCD pixel. The filters are arranged in alternating rows, every other row with the filter colors Red and Green, and the remaining rows as Blue and Green. Green appears twice as often because our eyes are most sensitive to differences in green for the detail in images. This is the Bayer mosaic or array.

5. REVIEW • Still we don't have a RGB image at this point. To get this we need to interpolate the RGB values from the information we have in the neighboring pixels. This interpolation, or de-mosaic process creates a file with RGB values for each pixel. The de-mosaic logic ranges from complex to extremely complex and constitutes the lion’s share of the computation work done by the Raw converter

6. Digital Camera Imaging Process

7. Raw Converter • De-mosaicing, is the key role a raw converter plays, but it’s not the only one. Raw conversion requires the following additional steps: • Colorimetric interpretation • White Balance • Gamma Correction • Anti-aliasing & Sharpening • Noise Reduction,

8. Colorimetric interpretation • Many different filter sets are in use with digital cameras. So the raw converter has to assign the correct, specific color meanings to the red, green, and blue pixels, usually in a colorimetrically defined color space such as CIE XYZ, which is based directly on human color perception. • Filter set information must be available from the sensor manufacturer to “support” a camera

9. White balance • The white balance setting on the camera has no effect whatsoever on the captured pixels when you shoot raw—it’s simply recorded as a metadata tag in the raw file. Some raw converters can read this tag and apply it as the default white balance (which the user can then override if desired), while others may ignore it completely and analyze the image to determine white balance. Some allow “pointing” at a neutral feature in the photo, and / or specifying color temperature.

10. Gamma correction. ·  Digital raw captures have linear gamma (gamma 1.0), a very different tonal response from that of either film or the human eye. So the raw converter applies gamma correction to redistribute the tonal information so that it corresponds more closely to the way our eyes see light and shade. (More discussion later)

11. Anti-aliasing and Sharpening • Problems can arise with very small details in an image. If the detail is only captured on a red-sensing pixel or a blue-sensing pixel, the raw converter may have a hard time figuring out what color that pixel should really be. Simple de-mosaicing methods struggle to maintain edge detail so raw converters employ edge-detection and anti-aliasing to compensate + sharpening to restore edge acutance.

12. Noise reduction • High ISO settings increase noise when the analog signal is amplified before being processed by the ADC. Noise reduction may be applied before de-mosaic (color noise suppression) and / or after to the pixel luminance. Some converters also suppress “hot pixels” that show up in long exposures. • It’s worth noting that ISO setting occurs before the ADC step, so it can’t be changed by the Raw converter.

13. Converter Summary • All raw converters perform these tasks, but they may use very different algorithms to do so, which is why the same image may look quite different when processed through different raw converters (or cameras). • 3rd party converters have a UI (user interface) that allows a degree of control for some or all the functions. • Many converters add controls for saturation, contrast, brightness and more.

14. All digital cameras have a raw converter Vendors make a subjective determination of what the best “look” is, and then adjust their converter to produce that result. (In-camera process variation is limited to the camera’s preset options). Also to save processing time, given the limitations in the camera’s small CPU, the converter uses only 8 bits of the available 10 or more bits of pixel information. The unused bits are discarded. When the RBG file is encoded and written, either as a TIFF or JPG, the original raw file is discarded.

15. In-camera Image Quality • Despite the limitations of the in-camera converter, the photo quality can be excellent. However,the quality may not hold up if it’s extensively edited, so getting exposure and white balance “right” are critical.

16. Raw Inconveniences • Raw files can’t be directly viewed (The EXIF file contains a low res thumbnail for camera LCD display) • Rawshots are SLOW - A 12-bit raw file contains 5.3X more data vs. an 8 bit TIFF; 20X more than a high quality JPG, so writing time to memory suffers. • Investment of your time - Converters will do a “default” conversion, with results not much (or no) better than in-camera (but 12-bit file is retained). • Raw file formats are proprietary - Not all converters can convert every format. Adobe’s DNG open format hasn’t been accepted by the camera manufacturers

17. Inconveniences Cont’d • 12-bit color information is only available with RAW files • Full power of a computer to perform more complex conversion algorithms utilizing up to 16 bit files yields enormous control over image tonality • Post-capture processing flexibility - many more options and tools to adjust contrast, saturation, white balance, sharpening, and exposure.

18. So why shoot Raw? • 12-bit color information is only available with Raw files • Full power of a computer to use more complex algorithms utilizing up to 16 bit files yields enormous control over image tonality • Post-capture processing flexibility - many more options and more precise tools to adjust contrast, saturation, white balance, sharpening, and exposure • The Raw file can be processed again and again as better converters becomes available

19. Our eyes and film photography • Our eye’s response to light is fundamentally different compared to the digital camera’s sensor • It’s non-linear –½ the light looks about 30% dimmer, 10X the light looks about 2X as bright. The human visual system easily comprehends 10,000:1 illumination range. • Coincidentally, silver halide photography responds in a manner very similar to the human eye. We measure exposure in zones or f-stops; i.e. doubling (or halving) exposure yields an exposure shift of one zone or f-stop.

20. Digital Camera Sensor • A linear device -1/2 the light is 1/2 as bright, 1/10 the light is 1/10 as bright • A linear device is said to have a “Gamma” = 1 • An  RBG image, at Gamma =1, will look very dark (The sensor will only capture a brightness range about 100:1 to 250:1, depending on the sensor noise & pixel size, but the scene illumination may vary by 10,000:1 or more)

21. Gamma Correction • Gamma correction applied by the converter redistributes the pixel luminance values so that limited brightness range captured by the sensor is “mapped” to match our eye’s sensitivity. Gamma = 2.2 is a good match to distribute relative brightness in a print or in a video display.

22. Gamma =1 vs. Gamma = 2.2

23. Tonality & Dynamic Rangethe 12-bit advantage

24. Conclusions from Table • Levels are lost when converting from a 12-bit RAW format (4096 levels total) to an 8-bit file (8-bit B&W or 24-bit color). A print has an effective contrast range of about 100:1 (6.6 f-stops), so image quality in an 8 bit converted file will be adequate after gamma transform if the exposure is correct and little editing is required. • This is achievable in studio environments, but may be a problem when using "natural" (i.e., uncontrolled) light. The result will be “blocked out” shadows and/or “blown” highlights, which have no tonal detail. 12-bit files have plenty of levels - 4096 total. (Files are processed as 16-bit in the computer, but 4 bits are unused)

25. Conclusions Cont’d • You can edit to your heart's content without fear of banding or other artifacts arising from limitations of 8-bit files. More important, the extra levels allow exposure adjustment “after the fact”. • White Balance adjustment is more accurate because the extra levels allow smaller, more accurate shifts in the RBG channels.

26. What is NOT evident! • Gamma transformed files are Non-linear • Adjusting any of the converter functions discussed earlier should be done before changing gamma to avoid unexpected RBG color shifts and/or color artifacts.  • Not all converters operate in this manner, nor does the software documentation disclose it. The best converters make adjustments in Gamma=1 space, but display the results in Gamma=2.2 space after adjusting. 

27. NOT evident - Cont’d • Photo Editors operate on Gamma transformed files, so even though similar actions are available in the editor and the raw converter, the results may not be the same, especially for colorimetric adjustments (levels, saturation, white balance). Geometric operations (resize, crop, perspective, etc) are generally not affected by gamma.

28. Guidelines for Raw workflow • Save the original raw file. It’s a digital negative that can be converted again as new and more sophisticated raw converters are developed. • Use a converter that doesn’t change the raw file. Instead, it creates a “transform instruction set” that outputs a standard format RBG. The converter software retains the instruction set, so it can be used again.

29. Guidelines Cont’d • Use the converter for all colorimetric adjustments. If local adjustments are needed, create additional converted files with appropriate adjustments, then use selections to build a composite via layers in the photo editor. • Limit sharpening to the minimum required to compensate for anti-alias filter effects during the de-mosaic process. • Selective sharpening can’t be done in the raw converter anyway and output specific sharpening (web or print) should be the last step in the photo editor.

30. Guidelines Cont’d • Apply noise reduction cautiously- it comes at the cost of detail, which can’t be recovered later.  • Avoid cropping in the raw converter to reduce the file size – Many of the geometric adjustments available in the photo editor are lens-centric. Perform them first, then crop. • Some photo editors allow editing in 16-bit files. However, if rule #1 is followed, there is little advantage unless very extensive geometric editing is required, where accumulated 8 bit quantizing errors might be visible. In the end, an 8-bit file is all that can be viewed or printed with today’s technology anyway.

31. Raw Conversion Software • New 3rd party software seems to appear monthly and revisions to existing are nearly as frequent. In addition every camera manufacturer who offers a raw format also offers a converter software package(s), both free and/or purchased. At last count there are 40+ converters available on the web. However, many of them use the same raw converter engine, DC RAW, supplied by David Coffin http://www.cybercom.net/~dcoffin/