Color

1. Color

2. Incident Light Body Reflected Light(object colour) Material surface Absorption Scattering Colorant Object color formation • The colorC of an object is determined by its reflectance r(l) and the visible wavelenghtsl of the • light it is exposed with, and angle of incidence • Objects change their color due to different factors: changes in illumination intensity, changes of the • color of the illumination source and conditions of interaction between light and object • Two basic components of r(l) are related to material body and material surfacereflection terms Surface Reflected Light(light source colour) Material body

3. Color image formation • Color image formation is determined by the relative radiant power distribution of the incident light, • the reflection of the materials and the characteristics of the observer Observer (Camera) () () () Incident light Reflected light surface albedo scene & viewpoint invariant body reflectance object material dependent Radiant power distr.scene dependent object surface normal object shape variant illumination direction scene dependent viewer’s direction viewpoint variant sensor sensitivity devicedependent

4. Observer/sensor sensitivity f () • Reflected light spectrum i.e. every colorcan be represented by a 3 element vector, with values which are the proportions of each of the primary colorsred (R), green (G) and blue (B) used to produce it. These are the tristimulusvalues • Having the light spectrum and the spectral reflectance curve of the object, the appearance of the object depends on the spectral sensitivity of the observer • Considering Tristimulus: RGB values of camera = Colour * Tristimulus. EyeResponse (conesonly) CameraResponse

5. Radiant power distribution of the incident light () • If the spectrum of the light source changes then the colour of the reflected light also changes.

6. RGB cameras • Digital images are nowadaysobtained from photographicdigitalcamerasthat use CMOS or CCD sensors to acquirethree color signals in the red (R)green (G) and blue (B) wavelenghts. Thesecamerasoften operate with a variation of the RGB space in a Bayer filterarrangement: greenisgiventwiceasmany detectors asred and blue (ratio 1:2:1) in order to achievehigherluminancethanchrominanceresolution. • The sensorhas a grid of red, green, and blue detectors arranged so that the first rowis RGRGRGRG, the nextis GBGBGBGB, and thatsequenceisrepeated in subsequentrows. For everychannel, missingpixels are obtained by interpolation to build up the complete image. • A pixel onlyrecordsone color out of three and cannotdetermine the color of the reflected light. An algorithmisneeded to estimate for each pixel the color levels for all color components, ratherthan a single component.

7. Demosaicingalgorithms • To obtain a full color image demosaicingalgorithms are usedthat interpolate a set of complete green, red, blue valuesateachpoint. Thisisdone in-camera producing a JPEG image. • Demosaicing can be done in different ways: • with bilinearinterpolation the redvalue of a non-red pixel iscomputedas the average of the two or fouradjacentredpixels, and similarly for blue and green • With the variableNumber of Gradientsinterpolation, computinggradientsnear the pixel of interest and usingthe lowergradientsrepresentingsmoother and more similarparts of the image to make an estimate • …. original red green blue reconstructedafterdemosaicing Bayer filtersamples

8. Truecolor • Truecoloris a method of representing and storinggraphical image information. Truecolordefines 256 (28) shades of red, green, and blue for each pixel of the digitalpicture, whichresults in 2563 or 16,777,216 (approximately 16.7 million) color variations for each pixel. • Manylow- to medium-end consumer digitalcameras and scannersconvert the camera RGB measurementsinto a standard RGB color spacereferred to assRGB. • Color spaces and color standards are discussed in the following

9. Color spaces • A color space is athree-dimensionaldefinition of a colorsystem. The attributes of the colorsystem are mapped ontothe coordinate axes of the color space. Different colorspaces exist: • Some colorspaces are perceptually linear, i.e. a change in stimulus will produce the same change in perception wherever it is applied. Other colour spaces, are not linear. • Some colorspaces are intuitive to use, i.e. it is easy for the user creating desired colours from space navigation. Other spaces require to manage parameters with abstract relationships to the perceived colour. • Some colorspaces are tied to specific equipments while others are equally valid on whatever device they are used. • Because of the need to have a color representation that is independent from the visualization system the CIE reference color space was defined. All colorspaces are subsets of theCIE color space.

10. CIE color matchingexperiment • The first color matchingexperimentsweredevised in late 1920s to characterize the relationshipbetween the physicalspectra and the perceived color, measuring the mixtures of differentspectraldistributionsthat are required for human observers to match colors • The experimentswereconducted by using a circular split screen 2° in size (the angularsize of theconedistribution in the human fovea). On one side of the field a test color wasprojected and on the other side, an observer-adjustable color wasprojected, thatwas a mixtureof threemonochromatic(single-wavelength) primarycolors, each with fixedchromaticity, but with adjustablebrightness • Notall test colors could be matchedusingthistechnique. Whenthiswas the case, a variableamountof one of the primarieswasallowed to add to the test color. The amount of the primaryaddedto the test color wasconsidered to be a negative value

11. CIE RGB colorspace • In 1931 CIE standardized the RGB color matchingfunctionsobtainedusingthreemonochromaticprimaries at wavelengthsof700 nm (red), 546.1 nm (green) and 435.8 nm (blue). The color matchingfunctions are the amounts of primariesneeded to match the monochromatic test primary at the wavelenghtshown on the horizontal scale. Almostanyspectralcomposition can be achieved by a suitablychosenlinear mix of thesethreemonochromaticprimaries • Ratherthanspecifying the brightness of eachprimary, the curveswerenormalized (scaled) to haveconstant area under them. The tabulatednumericalvalues of thesefunctions are knownas the CIE standard observer (CIE 1931 2° standard observer).Theyroughlycorrespond to coloursensations of red, green and blue. • The CIE RGB color spaceisoneofmany RGB color spaces, distinguishedby a particular set ofmonochromaticprimarycolors. The RGB tristimulus values for a color with a spectral power distribution I(λ) are given by integration of the scaled RGB color matching functions.

12. CIE XYZ color space • To avoid the presence of negative values of the CIE RGB matchingfunctions, the Commissiondevelopedthe CIE XYZ color spacethatwould relate to the CIE RGB color space by a linear transformationsuchthat the new color matchingfunctionswereto be everywheregreaterthan or equal to zero. The XYZ valuesfor a color with a spectralpowerdistributionE(λ) are given by integration of the scaled new color matchingfunctions • The transformationwasdeliberatelydesignedsuchthat: • Y wasa measure of the brightness or luminanceof a color, • Zwasquasi-equal tobluestimulation, • X isa linear combinationof coneresponsecurveschosen to be nonnegative (roughlyred). • The CIE XYZ color spaceservesas a basis from whichother color spaces are defined. 0.4887180 0.3106803 0.2006017 = 0.1762044 0.8129847 0.0108109 0.0000000 0.0102048 0.9897952

13. CIE xyY color space • By normalizing XYZ (i.e. dividing by X+Y+Z ) derivedvalues are obtainedreferred to asx,y,z. • In thatx+y+z= 1 the chromaticityof a color can be specified by twoparametersx y, of the threenormalizedvalues. • The x ydiagramobtained by intersecting the XYZ space with plane X+Y+Z = 1 and projecting • thisintersection on the xyplaneisreferred to asCIE Chromaticitydiagram and the xyvalues • are referredtoaschromaticityvalues. Theyrepresentall the colorsthat are visibleby the • humaneyewithconstantintensityequalto1. The degreeofluminance can beespressed in • percentagereferring to the Y coordinate of XYZ. • The x y Y spaceiswidelyused in practice to represent colors.

14. RGB colorspace • RGB coordinates can be directlyderived from XYZ using a linear transformation. Thisisdefined by fixing the coordinates of the white color and the threeprimaries. • The RGB spaceisusuallyrepresented by a cube using non-negative valueswithin a 0–1 range, assigningblack to the origin at the vertex (0, 0, 0), and with increasingintensityvaluesrunningalong the threeaxes up to white at the vertex (1, 1, 1), diagonally opposite black. • An RGB tripletrepresents the point of the given color within the cube. Computations of the color similarity of twogiven RGB colors isobtained by simplycalculating the distance betweenthem.

15. RGB color spaces • RGB color spaceis hardware dependent. Thereforeseveral RGB color spacesexist. The sRGBcolor space(createdcooperatively by HP and Microsoft in 1996) isthe mostwidelyused in practice • The sRGB color spaceusesthe sameprimariesas the ITU-R BT.709 primaries, standardized for studio monitors and HDTV. Itis the reference standard used for monitors, printers and on the Internet. LCDs, digitalcameras, printers, and scannersallfollow the sRGBstandard • For thisreason, one can generally assume, in the absence of anyother information, thatany 8-bit-per-channel image file or any 8-bit-per-channel image API or deviceinterface can be treatedasbeing in the sRGB color space. • . Color Space Gamut White Point Primaries • sRGB, HDTV (ITU-R BT.709), CRT D65 0.64 0.33 0.30 0.60 0.15 0.06 • scRGBUnlimited (signed) D65 0.64 0.33 0.30 0.60 0.15 0.06 • ROMM RGB Wide D50 0.7347 0.2653 0.1596 0.8404 0.0366 0.0001 • Adobe RGB 98 CRT D65 0.64 0.33 0.21 0.71 0.15 0.06 • Apple RGB CRT D65 0.625 0.34 0.28 0.595 0.155 0.07 • CIE (1931) Wide E 0.7347 0.2653 0.2738 0.7174 0.1666 0.0089 • ………. xRyRxGyGxByB

16. sRGB color space • The transformationbetween XYZ and sRGB and viceversa isobtainedapplying a linear • transformationfollowedby a secondtransformationasbelow: • where: Rlinear, Glinear and Blinearforin-gamutcolors are definedtobe in the range [0,1] • ClinearisRlinear, Glinear, or Blinear, andCsrgbisRsrgb,Gsrgbor Bsrgb. a= 0.055 Standard daylightilluminant White surfaceilluminatedbyaveragemiddaysun in western Europe/northernEurope sRGBgamut

17. rgbcolorspace • Thergbcolor space(RGB normalized) aims to separate the chromatic components from the • brightness components. It is used to eliminate the influence of varying illumination. The red, • green and blue channel can be transformed to their normalized counterpart r, g, baccording to: • One of these normalised channels is redundant since r + g + b= 1. Therefore the normalised RGB • space is sufficiently represented by two chromatic components (e.g. r, g) and a brightness • component

18. Human perception combines R, G and B response of the eye in opponent colors.Opponent colorscan be hence expressed in RGB space. Opponent colorspace + Achromatic Red cone + + + Rods + Red-Green + Green cone - + Yellow - l Blue-Yellow Blue cone + l l

19. HSB – HLS - HSV colorspaces • HSI (Hue, Saturation, Intensity), HLS (Hue, Saturation, Luminance), HSV (Hue, Saturation, Value)….. • allspecify colors usingthreevalues: hue (the color dominantwavelenght), saturation(how • much the color spectraldistribution – colorfulness - isaround a certainwavelenght) and luminance • (the amountof gray ) closely to human perception Colorfulnessis the differencebetween a colour and gray

20. HSVcolor space • HSV can be representedbothas a cylinder or a conespacedepending on whether the termS • refers to: • - saturation: colorfulnesswrtitsownbrightness) • - chroma: colorfulnesswrt the brightness of another • colourwhichappearswhite under similarviewingconditions • Valuescan bederivedby the RGB values: • Max = max (R, G, B); Min = min (R, G, B); • V - Value = max (R, G, B); • S - Chroma • if (Max = 0) then Chroma= 0; • else Chroma = (Max - Min) / Max; • H- Hue • if (Max = Min) then Hue is undefined (achromatic color);otherwise: if (Max = R & G > B) Hue = 60 * (G - B) / (Max-Min) • else • if (Max = R & G < B) Hue = 360 + 60 * (G - B) / (Max - Min) else • if (G = Max) Hue = 60 * (2.0 + (B - R) / (Max - Min)) else Hue = 60 * (4.0 + (R - G) / (Max - Min)) Green (120o) Yellow (60o) Cyan (180o) Red (0o) Blue (240o) Magenta (300o) Value White Hue Chroma Black

21. Distances in color space • Hardware oriented colorspacessuchas RGB, HSV, HSI… must be quantized in order to be processed. Uniformquantization of thesespacesresultsintoperceptuallyredundantbins and perceptualholes. A distancefunctionsuchas the Euclideandoesnotprovidesatisfactoryresults. • A difference between green and greenish-yellow is relatively large, whereas the distance distinguishing blue and red is quite small. • Ifinfinitesimaldistancesbetweentwocolorsasperceivedbyhumanswereconstant, the color spacewouldbeEuclidean and the distancebetweentwocolorswouldbeproportionalto the lenghtoftheirconnectingline. • Instead in the chromaticitydiagramxy, colorscorrespondingtopointsthathave the samedistancefrom a certainpoint are notperceivedassimilarcolorsbyhumans. • Mac Adams ellipsesaccount forthisphenomenon. Ellipses are suchthatcolors inside them are notdistinguishablefrom the color in the center.

22. Perceptualcolorspaces • CIE solved this problem in 1976 with the development of the L*a*b* perceptual color space. Other perceptualspaces are L*u*v* and L*c*h*. All are based on transformationsthatapproximate the XYZ RiemannspaceintoanEuclideanspace.Thesespaces are lessdistortedthan CIE XYZ spacealthoughthey are notcompletely free ofdistortion (MacAdam’sellipsesbecomenearlycircularhere). • In theL*a*b* and L*u*v*color spaces distances can becomputedasEuclideandistances: D(c1,c2) = [ (L*1 - L*2)2 + (a*1 - a*2)2 + (b*1 – b*2)2 ] 1/2 D(c1,c2) = [ (L*1 - L*2)2 + (u*1 - u*2)2 + (v*1 – v*2)2 ] 1/2 • Mathematicalapproximationsintroduced cause deviationsfromthisproperty in certainpartsof the spaces: • In L*u*v* Red is more representedthan Green and Blue. • In L*a*b* thereis a greatersensibilityto Green thanto Red and Blue. Blueishowever more representedthan in L*u*v* . L*a*b* L*u*v*

23. L*u*v* and L*a*b* color spaces 903,3 Y/YnifY/Yn < 0,008856 116 (Y/Yn)1/3otherwise a* = 500 [f(X/Xn)–f(Y/Yn) ] b* = 200 [f(Y/Yn)–f(Z/Zn) ] f(t) = (t)1/3 for t GT 0.008856 7.787 t + 16/116 otherwise XnYnZnare the XYZ valuesof the referencewhite L*= • 903,3 Y/Ynif Y/Yn < 0,008856 • 116 (Y/Yn)1/3 otherwise • u* = 13L* (u’ – u’n) • v* = 13L* (v’ – v’n) • u’ = 4X / (X+15Y+3Z) • v’ = 9Y / (X+15Y+3Z) • Ynunvnare the valuesof the referencewhite L*=

24. Color spaceinvariancetolight-object interaction • Interactions between light and object may produce highlighting and shadowing. Highlighting and shadowing depend on geometry, body and surface reflectance and illumination conditions. They can highly impair image matching. • The appropriateselection of color spaces can filter out or account of these effects to provide a transformed and more useful image. Shadowing Highlighting

25. rgb color spaceis photometric (shadowing/ highlighting) invariant • Considering the body reflection term: • Similarly for the surface reflection term. • Hue of HSI color space is photometric (shadowing/ highlighting) invariant

26. l1l2l3colorspace • l1l2l3color space is obtained from RGB manipulation and is invariant to highlighting effects of light interaction particularly for shiny objects 3D plot ofl1l2l3image Original RGB image l1l2l3image 3D plot of RGB image

27. c1c2c3colorspace • c1c2c3color space is obtained from RGB manipulation and is invariant to shadowing effects of light interaction particularly for matte objects 3D plot of c1c2c3 image 3D plot of RGB image

28. Classification of object edges • Local structures of objects like edges, corners and junctions depend on geometry, surface reflectance and illumination conditions. • Properties of color spaces can be used for their classification Color edges Shadow and geometry edges Highlight edges Geometry edge

29. Color edge maxima by type shadows and geometry highlights colour edges

30. RGB l1l2l3 c1c2 c3 tc1c2c3 highlight Color for shiny and matte objects tl1l2l3 Edge classification by colorspaces Color for shiny and matte objects (invariant to shadowing) Color for matte and shiny objects (Invariant to highlighting) • Using bothc1c2c3 and l1l2l3 color spaces: Color for matte and shiny objects (also highlights for shiny objects) c1c2c3 shadow or geometry l1l2l3

31. Summary of colorspace properties RGB (Red Green Blue) • additive color system based on tri-chromatic theory • easy to implement but non-linear with visual perception • device dependent with semi-intuitive specification of colours • very common, used in virtually every computer system, television, video etc HSL (Hue Saturation and Lightness), HSI (Intensity), … • intuitive specification of color • Obtained as linear transform from RGB and therefore device dependent and non-linear • Appropriate onlyfor moderate luminancelevels. Real world environments are notsuitablyrepresented in thisspace HSV HSL HSI (Hue component) • Invariant under the orientation of the object to the illumination and camera direction Opponent color axes • Advantage of isolating the brightness information on the third axis • Invariant to changes in illumination intensity and shadows c1c2c3color space • Invariant to shadowing effects (for matte objects) I1I2I3color space • Invariant to highlighting effects

32. CIE L*u*v* • based directly on CIE XYZ, attempts to linearize the perceptibility of unit vector color differences.Non-linear transformation but the conversions are reversible • The non-linear relationship for L*u*v* is intended to mimic the logarithmic response of the eye • CIE L*a*b* • based directly on CIE XYZ, attempts to linearise the perceptibility of unit vector color differences. Non-linear transformation but the conversions are reversible • The non-linear relationships for L*a* and b*are intended to mimic the logarithmic response of the eye • Suitable for real world scene color representation