Loading in 5 sec....

Combining Photometric and Geometric ConstraintsPowerPoint Presentation

Combining Photometric and Geometric Constraints

- 64 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Combining Photometric and Geometric Constraints' - hadley-fowler

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Combining Photometric and Geometric Constraints

Yael Moses

IDC, Herzliya

Joint work with Ilan Shimshoni and Michael Lindenbaum, the Technion

Y. Moses

- Recover the 3D shape of a general smooth surface from a set of calibrated images

Y. Moses

Recover the 3D shape of a smooth bilaterally symmetric object from a single image.

Y. Moses

Shape Recovery

- Geometry: Stereo
- Photometry:
- Shape from shading
- Photometric stereo
Main problems:

Calibrations and Correspondence

Y. Moses

3D Shape Recovery

Photometry:

- Shape from shading
- Photometric stereo

Geometry:

- Stereo
- Structure from motion

Y. Moses

Photometric Stereo

- 3D shape recovery:
surface normals from two or more images taken from the same viewpoint

Y. Moses

Photometric Stereo

Main Limitation:

Correspondence is obtained by a fixed viewpoint

- 3D shape recovery (surface normals)
Two or more images taken from the same viewpoint

Y. Moses

Overview

- Combining photometric and geometric stereo:
- Symmetric surface, single image
- Non symmetric: 3 images

- Mono-Geometric stereo
- Mono-Photometric stereo
- Experimental results.

Y. Moses

- Smooth featureless surface
- Taken under different viewpoints
- Illuminated by different light sources

- The Problem:
- Recover the 3D shape from a set of calibrated images

Y. Moses

n

*

- Perspective projection

- Three or more images

- Given correspondence the normals can be computed (e.g., Lambertian, distant point light source …)

*

Y. Moses

Our method

Combines photometric and geometric stereo

We make use of:

- Given Correspondence:
- Can compute a normal
- Can compute the 3D point

Y. Moses

First Order Surface Approximation

Y. Moses

First Order Surface Approximation

Y. Moses

First Order Surface Approximation

Y. Moses

Y. Moses

Y. Moses

Y. Moses

Basic Propagation

Y. Moses

Basic Propagation

Y. Moses

Basic method: First Order

- Given correspondence pi and L
Pand n

- Given P andn
T

- Given P, T andMi
a new correspondence qi

Y. Moses

Extensions

- Using more than three images
- Propagation:
- Using multi-neighbours
- Smart propagation

- Second error approximation
- Error correction:
- Based on local continuity
- Other assumptions on the surface

Y. Moses

Multi-neighbors Propagation

Y. Moses

Smart Propagation

Y. Moses

Y. Moses

Y. Moses

Using more than three images

- Reduce noise of the photometric stereo
- Avoid shadowed pixels
- Detect “bad pixels”
- Noise
- Shadows
- Violation of assumptions on the surface

Y. Moses

Smart Propagation

Y. Moses

Error correction

The compatibility of the local 3D shape can be used to correct errors of:

- Correspondence
- Camera parameters
- Illumination parameters

Y. Moses

Score

- Continuity:
- Shape
- Normals
- Albedo

- The consistency of 3D points locations and the computed normals:
- General case: full triangulation
- Local constraints

Y. Moses

Extensions

- Using more than three images
- Propagation:
- Using multi-neighbours
- Smart propagation

- Second error approximation
- Error correction:
- Based on local continuity
- Other assumptions on the surface

Y. Moses

Error correction + multi-neighbor5 Images

Y. Moses

Detected Correspondence

Y. Moses

Error correction + multi-neighbord

Multi-neighbors

Basic scheme (3 images)

Error correction no multi-neighbors

Y. Moses

Basic scheme

Multi-neighbors Error correction

Sec d

Ground truth

Basic scheme

Multi-neighbors approx.

Error correction

Y. Moses

Combining Photometry and Geometry

Yields a dense correspondence and

dense shape recovery of the object in a single path

Y. Moses

Assumptions

- Bilaterally Symmetric object
- Lambertian surface with constant albedo
- Orthographic projection
- Neither occlusions nor shadows
- Known “epipolar geometry”

Y. Moses

Geometric Stereo

- 2 different images
- Known camera parameters
- Known viewpoints
- Known correspondence

3D shape recovery

Y. Moses

Symmetry and Geometric Stereo

Non frontal view of a symmetric object

Two different images of the same object

Y. Moses

Symmetry and Geometric Stereo

Non frontal view of a symmetric object

Two different images of the same object

Y. Moses

Mono-Geometric Stereo

- 3D reconstruction:
given correspondence and ,

image

x object

known

z

unknown

Y. Moses

Photometric Stereo

- 2 images
- Lambertian reflectance
- Known illuminations
- Known correspondence
(same viewpoint)

3D shape recovery

Y. Moses

Symmetry and Photometric Stereo

Non-frontal illumination of

a symmetric object

Two different images of the same object

Y. Moses

Mono-Photometric Stereo

- 3D reconstruction given correspondence and E (up to a twofold ambiguity):

known

unknown

Y. Moses

Mono-Photometric Stereo

- 3D reconstruction E unknown but correspondence is given
- Frontal viewpoint with
non-frontal illumination.

- Use image first derivatives.

Y. Moses

Mono-Photometric Stereo Using image derivatives

- 3 global unknowns: E
- For each pair:
- 5 unknowns zxzy zxx zxy zyy
- 6 equations

- 3 pairs are sufficient

Y. Moses

Mono-Photometric StereoUnknown Illumination

Y. Moses

Correspondence

- No correspondence => no stereo.
- Hard to define correspondence in images of smooth surfaces.
- Almost any correspondence is legal when:
- Only geometric constraints are considered.
- Only photometric constraints are considered.

Y. Moses

Combining Photometry and Geometry

- Yields a dense correspondence
(dense shape recovery of the object).

- Enables recovering of the global parameters.

Y. Moses

Dense Correspondence using Propagation

Assume correspondence between a pair of points, p0land p0r.

Y. Moses

Computing and

- Object coordinates:
Given computing and is trivial

- Moving from object to image coordinates depends on the viewing parameter

Y. Moses

- Derivatives with respect to the object coordinates:
- Derivatives with respect to the image coordinates:

Y. Moses

General Idea

- Given a corresponding pair and E
n=(zx,zy,-1)T

- Given and n
cxand cy

- Given cxand cy
a new corresponding pair

Y. Moses

Results on Real Images: Given global parameters

Y. Moses

Finding Global Parameters

- Assume E and are unknown.
- Assume a pair of corresponding points is given.
- Two possibilities:
- Search for Eand directly.
- ComputeE andfrom the image second derivatives.

Y. Moses

Integration Constraint:Circular Tour

- All roads lead to Rome …
- Find and verify correct correspondence
- Recover global parameters, E and

Y. Moses

Finding Global Parameters

Consider image second derivatives

- Due to foreshortening effect:
and

- We can relate image and object derivatives by

Y. Moses

Testing E and : Image second derivatives

For each corresponding pair:

and

Plus 4 linear equations in 3 unknown.

Where

Y. Moses

Counting

- 5 unknowns for each pair:
zx zy,zxx zxy zyy

- 4 global unknowns: E,
- For each pair: 6 equations.
- For n pairs: 5n+4 unknowns
6n equations.

4 pairs are sufficient

Y. Moses

Recovering the Global Parameters

Y. Moses

Degenerate Case

- Close to frontal view: problems with geometric-stereo.
reconstruction problem

- Close to frontal illumination: problems with photometric-stereo.
correspondence problem

Y. Moses

Future work

- Perspective photometric stereo
- Use as a first approximation to global optimization methods
- Test on other reflection models
- Recovering of the global parameters:
- Light
- Cameras

- Detect the first pair of correspondence

Y. Moses

Future Work

- Extend to general 3 images under 3 viewpoints and 3 illuminations.
- Extend to non-lambertian surfaces.

Y. Moses

Thanks

Y. Moses

Integration Constraint

Y. Moses

Integration Constraint

Y. Moses

Searching for E

- Illumination must satisfy:
- E is further constrained by the image second derivatives.

Y. Moses

Image second derivatives

For each corresponding pair and E: 4 linear equations in 3 unknown.

Where

Y. Moses

Counting

- 5 unknowns for each pair: zx,zy,zxx,zxy,zyy
- 3 global unknowns: E
- For each pair: 6 equations.
- For n pairs: 5n+3 unknowns
6n equations.

3 pairs are sufficient

Y. Moses

Correspondence

Y. Moses

Variations

- Known/unknown distant light source
- Known/unknown viewpoint
- Symmetric/non-symmetric image
- Frontal/non-frontal viewpoint
- Frontal/non-frontal illumination

Y. Moses

Correspondence

- Epipolar geometry is the only geometric constraint on the correspondence.
- Weak photometric constraint on the correspondence.

Y. Moses

Download Presentation

Connecting to Server..