Color, Shading, Light Ian C. Scott-Fleming
Topics to Cover • Paper: Computational Model for Color Naming • Some background a quick look at: • Color, solar spectrum, eye as a detector • Color spaces • Summary of paper
Light, Spectrum, Eye Interesting note: birds have 4 color receptors – they have one in the UV. Which suggests that other than when displaying for humans, there’s no particular reason to limit computer vision to 3 channels… Note: The visible colorspace contains colors not in the solar spectrum: R+G = Yellow (~575 nm), but R+B=Magenta, which is not the color at 555 nm (Yellow-Green). We perceive ‘chords’ as a single color.
Brightness and Color • Difference between an Eye and a Camera: • Eye adjusts pupil as light level changes • Yellow stays yellow for eye • Cameras (most) do not have an adjustable pupil. • As Yellow gets brighter, Red & Green saturate at (255), and color will shift towards (pale) blue • Camera Detectors not as linear near saturation or near zero, so colors can shift with light levels
Shading with Surface Angle • Surface Brightness is proportional to cosine of angle between surface normal and incident light • A Ball looks spherical even with 1 eye closed, because we infer shape from shading • We see shading of Soccer Ball, but perceive it as all 1 color (orange) and spherical • Computer sees flat (2-D) image of Soccer Ball as range of colors.
Color Spaces • We want a color space where color is orthogonal to brightness, so we can (more easily) detect color independent of lighting level • Some candidates: • YCbCr (YPbPr) • YUV • HSL (HSV) • LAB
RGB • +: matches eye model nicely • +: Easy to implement in hardware • Put 3 photodetectors close together and deposit different bandpass filters • Digitize each pixel and send as color triplet • -: Color and brightness are coupled in each channel
YPbPr, YCbCr, YUV • Y: Brightness • Pb: difference between B & weighted sum of R,G • Pr: diff between R & B,G • Easy to implement: • Software: simple matrix multiply, or: • Hardware: a few op amps between readout & digitizer YPbPr ======================================================== Y' = + 0.299 * R' + 0.587 * G' + 0.114 * B' Pb = - 0.168736 * R' - 0.331264 * G' + 0.5 * B' Pr = + 0.5 * R' - 0.418688 * G' - 0.081312 * B' ........................................................ R', G', B' in [0; 1] Y' in [0; 1] Pb in [-0.5; 0.5] Pr in [-0.5; 0.5] YCbCr identical to YPbPr except scaled 0-255 instead of 0-1 YUV like YPbPr, but U,V scaled slightly differently (used for analog TV signal transmission)
LAB • Intended to encapsulate human perception of color differences: • Euclidean distance between colors in LAB proportional (sort-of) to perceived difference between colors • Calculations complex: • Matrix multiply followed by non-integer exponentiation
HSL, HSV • Hue (color) • Represents angle on a color wheel • Saturation – measure of pureness of color • No saturation: pure color • Some saturation: whitish (pastels) • fully saturated: black/gray/white (no color) • Value (or Lightness) – brightness (grayscale) • +: Better Matches how we perceive color (light green, dark blue) • -: Difficult to implement: requires series of if/then/else statements • -: Color not completely decoupled from brightness
Comparison of HSL (left) & HSV (right)
Paper: Color Naming • Develop a human-color-perception naming scheme that can be implemented on a computer, and some metrics for measuring distances in color space • Demonstrate how it can be used (in conjunction with other standard CV tools) to segment an image and describe the colors in it
3 Parts to the Paper • Develop a Language for color Naming • in CS sense of language: • Syntax, semantics, vocabulary • Generate a Metric for mapping an arbitrary color to a ‘nearest’ color prototype (aka focus) • Locate nearest prototype • Measure difference from prototype • Demonstrate Segmentation and extracting color composition
Develop Language • Conducted some experiments with humans to see how consistent naming is, how universal color names are • Created a well-defined syntax with several levels of descriptiveness
Syntax: • Proposed language is basically a recursive systematization of HSL color space: • <color name> • <color name>:<chromatic name> | <achromatic name> • <chromatic name>: <lightness><saturation><hue> | <saturation><lightness><hue> • <achromatic name>:<lightness><achromatic term> • Red, Green, Orange, Brown, Cyan, etc. • <lightness>: blackish | very dark | dark | medium | light | very light | whitish • Dark blue, light green, etc. • <saturation>:grayish | moderate| medium | strong | vivid • Strong blue, vivid yellow, etc. • <hue>: <generic hue> | <halfway hue> | <quarterway hue> • <generic hue>: red | orange | brown | yellow | green | blue | purple | pink | beige | olive • <halfway hue>: <generic hue> - <generic hue> • <quarterway hue>: <ish form><generic hue> • <ish form>: reddish | brownish | yellowish | greenish | bluish | purplish | pinkish • <achromatic term>: <generic achromatic term> | <ish form><generic achromatic term> • <generic achromatic term>: gray | black | white
Multiple Levels for different descriptive requirements: • Fundamental Level: a generic hue • Course level: add a luminance descriptor • Medium level: add saturation • Minute level: Complete color syntax ( including halfway & quarterway hues?) • Appropriate level used depended upon situation. • The car is green • The sky is light blue while the lake is dark blue • minute level only when needing to be very specific
Color Metric Figure 1 & Table 1 assert that LAB distance (DLAB)not a good distance measure. Author reports that ThetaHLS and DHLS correlate better with perceived distance. Also, with sparse set of color names nearest prototype color not necessarily appropriate. Example: Nearest Prototype in LAB space maps some grays to “Whitish Pink”, etc. Author develops a metric which weighs ThetaHLSmore heavily than LAB distance does in calculating distance
Modified Distance Measure • Geometric rationale for measure: • Find a point, Copt , along prototype color’s (C0) vector from origin, which best represents color Cx. • Adjust Copt so that it’s distance from projection of Cx incorporates penalty for size of thetaHSL Note: Author doesn’t provide clear justification for why this is a better measure. Simply presents statistic that it matches human judgment of color differences better than DLAB. Nor does she explain in what sense Coptis “optimal”.
Example Application: Color-Segmenting an image and obtaining Color Labels • First, a “Textbook case” of how to rigorously segment a color image • Steps correspond in some sense to human perception process to adapt to light levels and assign colors • Adjust for camera nonlinearities (gamma correction, “Color Constancy Issues”) • Most images include gamma encoding so they will display properly on a CRT screen. • Median-Filter image to remove noise • Remove zero-offset (black bias) and do contrast stretch for maximum dynamic range to adapt to lighting • Adaptive filtering to smooth uniform and textured areas • Not clear whether she returns to original pixel values for filtering, or works with reduced-color image at this point. • Segment image using Mean Shift • references Comanicu & Meers paper (1st reading assmt)
Finally, use Naming • After segmenting, then for each segmentation color, apply naming algorithm to find nearest prototype color and assign its name to that region of segmented image • Personal Note: I found the segmenting steps a little distracting here – they take focus away from use of color naming. Presumably the description of how she does the segmenting was to be rigorous. • The naming could be applied to any segmented image, regardless of how the segmentation was done.
Questions • (AC) What exactly is being shown in table 1? • Distance measures for five colors of figure 1 • Dlab, Dhsl, thetaLABand thetaHLS • Author is trying to show that Dlab is not best measure. Would help if we had color copy of paper… • What is a color foci? • One of the 267 prototype colors of the NBS color dictionary • In the section about color naming results, they speak about coarse, medium and minute, why do they have three different levels • Context-specific level to apply naming language • Pictures typically described at coarse or medium • Green ball or light blue car • Specifics of an object, or differentiating objects, may require minute (full) language
What is fig.2 trying to say? • Illustrates steps and terms used in section B. • What is meaning of equation number 5 in page 696? • Attempt to describe mathematically how to choose the blackest and whitest pixel for the contrast stretch/lighting correction. • What is hypothesis of graded (fuzzy ) membership ? • (?) analog to author’s levels (fundamental, coarse, medium, minute) • What is color vocabulary? what is the difference between color naming and color vocabulary ? • What is Munsell color order system and what made it widely used in spite of its lack of color vocabulary? • First (1905) color space separating Hue (color), value (brightness) and chroma (~saturation) • First attempt to define color measure with a meaningful difference measure. • Lack of vocabulary: limited number of colors. • Still in use because of entrenched applications (Brewers use it to define beer color, USGS uses it to define soil colors, etc.)
I do not really understand why it is so important to name colors, instead of just calling them a number. • Automatic labeling of colors in images for DB matching, human understanding, etc.. See examples in section VI, Results, Discussion, Conclusions • How are prototypes (foci) decided when the metric computes the distance between input color and prototypes? (Section III, page4) • Prototypes are a given (input). Author worked from NBS dictionary of 267 colors, but notes that other color dictionaries could be used. • Could you please explain the algorithm in Designing the Color-Naming Metric on page 5?