- 64 Views
- Uploaded on
- Presentation posted in: General

Removing Shadows From Images

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Removing Shadows From Images

G. D. Finlayson1, S.D. Hordley1& M.S. Drew2

1School of Information Systems, University of East Anglia, UK

2School of Computer Science, Simon Fraser University, Canada

Introduction

Shadow Free Grey-scale images

- Illuminant Invariance at a pixel

Shadow Free Colour Images

- Removing shadow edges using illumination invariance

- Re-integrating edge maps

Results and Future Work

We would like to go from a colour image with shadows, to the same colour image, but without the shadows.

For Computer Vision

- improved object tracking, segmentation etc.

For Image Enhancement

- creating a more pleasing image

For Scene Re-lighting

- to change for example, the lighting direction

Region Lit by Sunlight and Sky-light

Region Lit by Sky-light only

A shadow is a local change in illumination intensity and (often) illumination colour.

So, if we can factor out the illumination locally (at a pixel) it should follow that we remove the shadows.

So, can we factor out illumination locally? That is, can we derive an illumination-invariant colour representation at a single image pixel?

Yes, provided that our camera and illumination satisfies certain restrictions ….

Conditions for Illumination Invariance

(1) If sensors can be represented as delta functions (they respond only at a single wavelength)

(2) and illumination is restricted to the Planckian locus

(3) then we can find a 1-D co-ordinate, a function of image chromaticities, which is invariant to illuminant colour and intensity

(4) this gives us a grey-scale representation of our original image, but without the shadows (it takes us a third of the way to the goal of this talk!)

Camera responses depend on 3 factors: light (E), surface (S), and sensor (R, G, B)

B(l)

G(l)

R(l)

Sensitivity

l

=

Delta functions “select” single wavelengths:

Most typical illuminants lie on, or close to, the Planckian locus (the red line in the figure)

So, let’s represent illuminants by their equivalent Planckian black-body illuminants ...

Here I controls the overall intensity of light, T is the temperature, and c1, c2 are constants

But, for typical illuminants, c2>>lT.

So, Planck’s eqn.

is approximated as:

2500 Kelvin

5500 Kelvin

10000 Kelvin

For, delta function sensors and Planckian Illumination we have:

Surface

Light

Or, taking the log of both sides ...

Where subscript s denotes dependence on reflectance and k,a,b

and c are constants. T is temperature.

Constant independent

of sensor

Variable dependent

only on reflectance

Variable dependent

on illuminant

First, let’s calculate log-opponent chromaticities:

Then, with some algebra, we have:

That is: there exists a weighted difference of log-opponent chromaticities that depends only on surface reflectance

B

P

R

W

G

Y

Narrow-band (delta-function sensitivities)

Log-opponent chromaticities for 6 surfaces under 9 lights

Log-opponent chromaticities for 6 surfaces under 9 lights

Rotate chromaticities

This axis is invariant to illuminant colour

Normalized sensitivities of a SONY DXC-930 video camera

Log-opponent chromaticities for 6 surfaces under 9 different lights

Log-opponent chromaticities for 6 surfaces under 9 different lights

Rotate chromaticities

The invariant axis is now only approximately illuminant invariant (but hopefully good enough)

With certain restrictions, from a 3-band colour image we can derive a 1-d grey-scale image which is:

- illuminant invariant

- and so, shadow free

To complete our goal we would like to go back to a 3-band colour image, without shadows

We will look next at how the invariant representation can help us to do this ...

Consider an edge map of the colour image ...

And an edge map of the 1-d invariant image ...

These are approximately the same, except that the invariant edge map has no shadow edges

From these two edge maps we can remove shadow edges thus:

Edges = Iorig & Iinv

(Valid edges are in the original image, and in the invariant image)

So, now we have the edge map of the image we would like to obtain (edge map of the original image with shadows edges set to zero)

So, can we go from this edge information back to the image we want? (can we re-integrate the edge information?).

Of course, re-integrating a single edge map will give us a grey-scale image:

Red

So, we must apply any procedure to each band of the colour image separately:

Green

Blue

Re-integrated

Original

Colour Channels

Edge Maps of Channels

Shadow Edges Removed

The re-integration problem has been studied by a number of researchers:

- Horn

- Blake et al

- Weiss ICCV ‘01 (Least-Squares)

- ...

- Land et al (Retinex)

The aim is typically to derive a reflectance image from an image in which illumination and reflectance are confounded.

Weiss used a sequence of time varying images of a fixed scene to determine the reflectance edges of the scene

His method works by determining, from the image sequence, edges which correspond to a change in reflectance(Weiss’ definition of a reflectance edge is an edge which persists throughout the sequence)

Given reflectance edges, Weiss re-integrates the information to derive a reflectance image

In our case, we can borrow Weiss’ re-integration procedure to recover our shadow-free image.

Let Ij(x,y) represent the log of a single band of a colour image

We first calculate:

y is the derivative operator in the y direction

x is the derivative operator in the x direction

T is the operator that sets shadow edges to zero

This summarises the process of detecting and removing shadow edges

To recover the shadow free, image we want to invert this Equation

To do this, we first form the Poisson Equation

We solve this (subject to Neumann boundary conditions) as follows:

We solve by applying the inverse Laplacian

Note: the inverse operator has no Threshold

Applying this process to each of the three channels recovers a log image without shadows.

1. Iorig = Original colour image, Iinv = Invariant image

2. For j=1,2,3 Ijorig = jth band of Iorig

3. Remove Shadow Edges: Edges = Ijorig & Iinv

4. Differentiate the thresholded edge map

5. Re-integrate the image

6. Goto 3

The re-integration step is unique up to an additive constant (a multiplicative constant in linear image space

Fixing this constant amounts to applying a correction for illumination colour to the image. Thus we choose suitable constants to correct for the prevailing scene illuminant

In practice, the method relies upon having an effective thresholding step T, that is, on effectively locating the shadow edges.

As we will see, our shadow edge detection is not yet perfect

The Shadow Edge Detection consists of the following steps:

1. Edge detect a smoothed version of the original (by channel) and the invariant images

Canny or SUSAN

2. Threshold to keep strong edges in both images

3. Shadow Edge = Edge in Original & NOT in Invariant

4. Applying a suitable Morphological filter to thicken the edges resulting from step 3.

This typically identifies the shadow edges plus some false edges

OriginalImage

InvariantImage

Detected Shadow Edges

Shadow Removed

OriginalImage

InvariantImage

Detected Shadow Edges

Shadow Removed

OriginalImage

InvariantImage

Detected Shadow Edges

Shadow Removed

OriginalImage

InvariantImage

Detected Shadow Edges

Shadow Removed

We have presented a method for removing shadows from images

The method uses an illuminant invariant 1-d image representation to identify shadow edges

From the shadow free edge map we re-integrate to recover a shadow free colour image

Initial results are encouraging: we are able to remove shadows, even when shadow edge definition is not perfect

We are currently investigating ways to more reliably identify shadow edges ...

… or to derive a re-integration which is more robust to errors (Retinex?)

Currently deriving the illuminant invariant image requires some knowledge of the capture device’s characteristics

- We show in the paper how to determine these characteristics empirically and we are working on making this process more robust

The authors would like to thank Hewlett-Packard Incorporated for their support of this work.