Color, Shading, Light

Color, Shading, Light Ian C. Scott-Fleming

Topics to Cover • Paper: Computational Model for Color Naming • Some background a quick look at: • Color, solar spectrum, eye as a detector • Color spaces • Summary of paper

Light, Spectrum, Eye Interesting note: birds have 4 color receptors – they have one in the UV. Which suggests that other than when displaying for humans, there’s no particular reason to limit computer vision to 3 channels… Note: The visible colorspace contains colors not in the solar spectrum: R+G = Yellow (~575 nm), but R+B=Magenta, which is not the color at 555 nm (Yellow-Green). We perceive ‘chords’ as a single color.

Brightness and Color • Difference between an Eye and a Camera: • Eye adjusts pupil as light level changes • Yellow stays yellow for eye • Cameras (most) do not have an adjustable pupil. • As Yellow gets brighter, Red & Green saturate at (255), and color will shift towards (pale) blue • Camera Detectors not as linear near saturation or near zero, so colors can shift with light levels

Shading with Surface Angle • Surface Brightness is proportional to cosine of angle between surface normal and incident light • A Ball looks spherical even with 1 eye closed, because we infer shape from shading • We see shading of Soccer Ball, but perceive it as all 1 color (orange) and spherical • Computer sees flat (2-D) image of Soccer Ball as range of colors.

Color Spaces • We want a color space where color is orthogonal to brightness, so we can (more easily) detect color independent of lighting level • Some candidates: • YCbCr (YPbPr) • YUV • HSL (HSV) • LAB

RGB • +: matches eye model nicely • +: Easy to implement in hardware • Put 3 photodetectors close together and deposit different bandpass filters • Digitize each pixel and send as color triplet • -: Color and brightness are coupled in each channel

YPbPr, YCbCr, YUV • Y: Brightness • Pb: difference between B & weighted sum of R,G • Pr: diff between R & B,G • Easy to implement: • Software: simple matrix multiply, or: • Hardware: a few op amps between readout & digitizer YPbPr ======================================================== Y' = + 0.299 * R' + 0.587 * G' + 0.114 * B' Pb = - 0.168736 * R' - 0.331264 * G' + 0.5 * B' Pr = + 0.5 * R' - 0.418688 * G' - 0.081312 * B' ........................................................ R', G', B' in [0; 1] Y' in [0; 1] Pb in [-0.5; 0.5] Pr in [-0.5; 0.5] YCbCr identical to YPbPr except scaled 0-255 instead of 0-1 YUV like YPbPr, but U,V scaled slightly differently (used for analog TV signal transmission)

LAB • Intended to encapsulate human perception of color differences: • Euclidean distance between colors in LAB proportional (sort-of) to perceived difference between colors • Calculations complex: • Matrix multiply followed by non-integer exponentiation

HSL, HSV • Hue (color) • Represents angle on a color wheel • Saturation – measure of pureness of color • No saturation: pure color • Some saturation: whitish (pastels) • fully saturated: black/gray/white (no color) • Value (or Lightness) – brightness (grayscale) • +: Better Matches how we perceive color (light green, dark blue) • -: Difficult to implement: requires series of if/then/else statements • -: Color not completely decoupled from brightness

For HSV: H is same, V = max, S = (max-min)/max

Comparison of HSL (left) & HSV (right)

Paper: Color Naming • Develop a human-color-perception naming scheme that can be implemented on a computer, and some metrics for measuring distances in color space • Demonstrate how it can be used (in conjunction with other standard CV tools) to segment an image and describe the colors in it

3 Parts to the Paper • Develop a Language for color Naming • in CS sense of language: • Syntax, semantics, vocabulary • Generate a Metric for mapping an arbitrary color to a ‘nearest’ color prototype (aka focus) • Locate nearest prototype • Measure difference from prototype • Demonstrate Segmentation and extracting color composition

Develop Language • Conducted some experiments with humans to see how consistent naming is, how universal color names are • Created a well-defined syntax with several levels of descriptiveness

Multiple Levels for different descriptive requirements: • Fundamental Level: a generic hue • Course level: add a luminance descriptor • Medium level: add saturation • Minute level: Complete color syntax ( including halfway & quarterway hues?) • Appropriate level used depended upon situation. • The car is green • The sky is light blue while the lake is dark blue • minute level only when needing to be very specific

Color Metric Figure 1 & Table 1 assert that LAB distance (DLAB)not a good distance measure. Author reports that ThetaHLS and DHLS correlate better with perceived distance. Also, with sparse set of color names nearest prototype color not necessarily appropriate. Example: Nearest Prototype in LAB space maps some grays to “Whitish Pink”, etc. Author develops a metric which weighs ThetaHLSmore heavily than LAB distance does in calculating distance

Modified Distance Measure • Geometric rationale for measure: • Find a point, Copt , along prototype color’s (C0) vector from origin, which best represents color Cx. • Adjust Copt so that it’s distance from projection of Cx incorporates penalty for size of thetaHSL Note: Author doesn’t provide clear justification for why this is a better measure. Simply presents statistic that it matches human judgment of color differences better than DLAB. Nor does she explain in what sense Coptis “optimal”.

Example Application: Color-Segmenting an image and obtaining Color Labels • First, a “Textbook case” of how to rigorously segment a color image • Steps correspond in some sense to human perception process to adapt to light levels and assign colors • Adjust for camera nonlinearities (gamma correction, “Color Constancy Issues”) • Most images include gamma encoding so they will display properly on a CRT screen. • Median-Filter image to remove noise • Remove zero-offset (black bias) and do contrast stretch for maximum dynamic range to adapt to lighting • Adaptive filtering to smooth uniform and textured areas • Not clear whether she returns to original pixel values for filtering, or works with reduced-color image at this point. • Segment image using Mean Shift • references Comanicu & Meers paper (1st reading assmt)

Finally, use Naming • After segmenting, then for each segmentation color, apply naming algorithm to find nearest prototype color and assign its name to that region of segmented image • Personal Note: I found the segmenting steps a little distracting here – they take focus away from use of color naming. Presumably the description of how she does the segmenting was to be rigorous. • The naming could be applied to any segmented image, regardless of how the segmentation was done.

Questions • (AC) What exactly is being shown in table 1? • Distance measures for five colors of figure 1 • Dlab, Dhsl, thetaLABand thetaHLS • Author is trying to show that Dlab is not best measure. Would help if we had color copy of paper… • What is a color foci? • One of the 267 prototype colors of the NBS color dictionary • In the section about color naming results, they speak about coarse, medium and minute, why do they have three different levels • Context-specific level to apply naming language • Pictures typically described at coarse or medium • Green ball or light blue car • Specifics of an object, or differentiating objects, may require minute (full) language

What is fig.2 trying to say? • Illustrates steps and terms used in section B. • What is meaning of equation number 5 in page 696? • Attempt to describe mathematically how to choose the blackest and whitest pixel for the contrast stretch/lighting correction. • What is hypothesis of graded (fuzzy ) membership ? • (?) analog to author’s levels (fundamental, coarse, medium, minute) • What is color vocabulary? what is the difference between color naming and color vocabulary ? • What is Munsell color order system and what made it widely used in spite of its lack of color vocabulary? • First (1905) color space separating Hue (color), value (brightness) and chroma (~saturation) • First attempt to define color measure with a meaningful difference measure. • Lack of vocabulary: limited number of colors. • Still in use because of entrenched applications (Brewers use it to define beer color, USGS uses it to define soil colors, etc.)

I do not really understand why it is so important to name colors, instead of just calling them a number. • Automatic labeling of colors in images for DB matching, human understanding, etc.. See examples in section VI, Results, Discussion, Conclusions • How are prototypes (foci) decided when the metric computes the distance between input color and prototypes? (Section III, page4) • Prototypes are a given (input). Author worked from NBS dictionary of 267 colors, but notes that other color dictionaries could be used. • Could you please explain the algorithm in Designing the Color-Naming Metric on page 5?

Color, Shading, Light