C o lo ur an algorithmic approach

PhD Research Topic Colouran algorithmic approach Thomas Bangert thomas.bangert@qmul.ac.uk http://www.eecs.qmul.ac.uk/~tb300/pub/PhD/ColourVision2.pptx

understanding how natural visual systems process information Visual system: • about 30% of cortex • most studied part of brain • best understood part of brain This research is abut the information produced by the early visual system Information which goes from front of brain to higher levels at rear of brain

Image sensors • Binary sensor arraymonochromatic ‘external retina’ • Luminance sensor arraydichromatic colour • Multi-Spectral sensor arraytetrachromatic colour Sensors: what is measured and what information is sent?

What is Colour? Visible Spectrum the stimulus Visual system must measure and represent light within this zone. We start with Luminance – how bright?(we measure how much light) What information does colour add?How do we code this information. Any ideas?

Lets hypothesise … When an astronomer looks at a star, how does he code the information his sensors produce? It was noticed that parts of spectrum were missing.

Looking our own star – the sun • x

Each atomic element absorbs at specific frequencies …

We can Code for these elements … We can imagine how coding spectral element lines could be used for visual perception … by a creature very different to us… a creature which hunts by ‘tasting’ the light we reflect… seeing the stuff we are made of Colour in this case means atomic structure and chemistry…

Sensor we build cannot do spectral analysis Crude system to reproduce colour 3 colour values, usually 8-bit:RGB What does RGB mean?  lights for colour reproduction

The Standard Observer CIE1931 xy chromaticity diagram primaries at: 435.8nm, 546.1nm, 700nm XYZ – a 3 sensors model of human vision 1 central luminance sensor: Yand colour information are 2 difference measurements ... from YThe Math: … z is redundant

Understanding CIE chromaticity Best understood as a failed colour circle White in center Saturated / monochromatic colours on the periphery Everything in between is a mix of white and the colour

But does it blend? Does it match? The problem of ‘negative primaries’ Monochromatic Colours

? The Human Visual System (HVS) does things differently!

Human Visual System (HVS) Coding Colour

The Sensor 2 systems: day-sensor & night-sensor To simplify: we ignore night sensor system Cone Sensors very similar to RGB sensors we design for cameras

sensor array arrangement is random note:very few blue sensors, none in the centre

sensor pre-processing circuitry

First Question: What information is sent from sensor array to visual system? Very clear division between sensor & pre-processing (Front of Brain) andvisual system (Back of Brain) connected with very limited communication link

starting with the sensor:Human Sensor Responseto non-chromatic light stimuli

HVS Luminance Sensor Idealized A linear response in relation to wavelength. Under ideal conditions can be used to measure wavelength.

Spatially Opponent HVS:Luminance is always measured by taking the difference between two sensor values.Produces: contrast value Which is done twice, to get a signed contrast value

Colour Sensorresponse to monochromatic light Human Bird 4 sensors Equidistant on spectrum

if we make a simplifying assumption:our light is monochromatic! 1 . 0 a shift of Δ from a known reference point 0 . 8 G R 0 . 6 0 . 4 0 . 2 0 . 0 λ-Δ λ λ+Δ Wavelength Then:

the ideal light stimulus Monochromatic Light Allows frequency to be measured in relation to reference.

Problem:natural light is not ideal • Light stimulus might not activate reference sensor fully. • Light stimulus might not be fully monochromatic. ie. there might be white mixed in

Solution: Then reference sensor can be normalized Which is subtracted. A 3rd sensor is used to measure equiluminance.

a 4 sensor design 2 opponent pairs • only 1 of each pair can be active • min sensor is equiluminance

What is Colour? What is the information? • Luminance • Equi-Luminance • Colour Colour channels are: RG Byellow 4 primaries. Purpose of Colour is to code wavelength! Information = Luminance + Wavelength

Any Stimuli can be reduced to: Equi-Luminance Location on Spectrum Luminance Complex Spectrum is reduced to very simple equivalent

Colour often involves further high level processing …Examples of real world colour: Colours are often computed, not measured!

… an extreme example What is the colour?

http://www.eecs.qmul.ac.uk/~tb300/pub/PhD/ColourVision2.pptxReferenceshttp://www.eecs.qmul.ac.uk/~tb300/pub/PhD/ColourVision2.pptxReferences Questions? Poynton, C. A. (1995). “Poynton’s Color FAQ”, electronic preprint.http://www.poynton.com/notes/colour_and_gamma/ColorFAQ.html Bangert, Thomas (2008). “TriangleVision: A Toy Visual System”, ICANN 2008. Goldsmith, Timothy H. (July 2006). “What birds see”. Scientific American: 69–75. Neitz, Jay; Neitz, Maureen. (August 2008). “Colour Vision: The Wonder of Hue”. Current Biology 18(16): R700-r702.

C o lo ur an algorithmic approach