Download
1 / 53

Some problems... - PowerPoint PPT Presentation


  • 81 Views
  • Uploaded on

Some problems. Lens distortion. Uncalibrated structure and motion recovery assumes pinhole cameras Real cameras have real lenses How can we correct distortion , when original calibration is inaccessible?. Even small amounts of lens distortion can upset uncalibrated structure from motion

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Some problems...' - serena


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Lens distortion
Lens distortion

Uncalibrated structure and motion recovery assumes pinhole cameras

Real cameras have real lenses

How can we correct distortion, when original calibration is inaccessible?


Even small amounts of lens distortion can upset uncalibrated structure from motion

A single distortion parameter is enough for mapping and SFX accuracy

Including the parameter kin the multiview relations changes the 8-point algorithm from

You can solve such “Polynomial Eigenvalue Problems”

This is as stable as computation of the Fundamental matrix, so you can use it all the time.


E structure from motion

ven small amounts of lens distortion can upset uncalibrated structure from motion—


A map building problem
A map-building problem structure from motion

  • Input movie – relatively low distortion

  • Plan view: red is structure, blue is motion

(a) (b)


Effects of distortion
Effects of Distortion structure from motion

  • Input movie – relatively low distortion

  • Recovered plan view, uncorrected distortion

(a) (c)


Does distortion do that

Distortion of image plane is structure from motionconflated with focal length

when the camera rotates

[From: Tordoff & Murray, ICPR 2000]

Does distortion do that?



Distortion correction in natural scenes
Distortion correction in natural scenes structure from motion

[Farid and Popescu, ICCV 2001]

  • In natural images, distortion introduces correlations in frequency domain

  • Choose distortion parameters to minimize correlations in bispectrum

  • Less effective on man-made scenes....


Distortion correction in multiple images
Distortion correction in multiple images structure from motion

Multiple views, static scene

  • Use motion and scene rigidity [Zhang, Stein, Sawhney, McLauchlan, ...]

    Advantages:

  • Applies to man-made or natural scenes

    Disadvantages:

  • Iterative solutions|require initial estimates


A structure from motion

single distortion parameter is accurate enough for map-building and cinema post production—


Modelling lens distortion

x structure from motion:xeroxed noxious experimental artifax

p:perfect pinhole perspective pure

Modelling lens distortion

p

p

x

x

Known

Unknown


Single parameter models
Single-parameter models structure from motion


Single parameter modelling power
Single-parameter modelling power structure from motion

  • Single-parameter model

  • Radial term only

  • Assumes distortion centre is at centre of image

A one-parameter model suffices


A direct solution for structure from motionk


Look at division model again
Look at division model again structure from motion


A quick matlab session
A quick matlab session structure from motion

>> help polyeig

POLYEIG Polynomial eigenvalue problem.

[X,E] = POLYEIG(A0,A1,..,Ap) solves the polynomial eigenvalue problem

of degree p:

(A0 + lambda*A1 + ... + lambda^p*Ap)*x = 0.

The input is [etc etc...]

>>


Algorithm
Algorithm structure from motion


T structure from motion

his is as stable as computation of the fundamental matrix, so you can use it all the time—


Performance synthetic data

Stable structure from motion – small errorbars

Biased – not centred on true value

Performance: Synthetic data

0

-0.1

Computed l

-0.2

-0.3

-0.4

0

0.2

0.4

0.6

0.8

1

Noise s (pixels)


Analogy linear ellipse fitting

Best-fit line structure from motion

Analogy: Linear ellipse fitting

True

Fitted: 10 trials

Data


Performance synthetic data1
Performance: Synthetic data structure from motion


Performance real sequences
Performance: Real sequences structure from motion


250 pairs structure from motion

Low distortion

Linear estimate used to initialize nonlinear

Number of inliers changes by [-25..49]

50

40

30

20

10

0

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3


Conclusions
Conclusions structure from motion


Environment matting

In structure from motion: magnifying glass moving over background

Out: same magnifying glass, new background

Environment matting


Environment matting why

Learn structure from motion

light-transport properties of complex optical elements

Previously

Ray tracing geometric models

Calibrated acquisition

Here

Acquisition in situ

Environment matting: why?


Image formation model

Purely 2D-2D structure from motion

Optical element performs weighted sum of (image of) background at each pixel

suffices for many interesting objects

separate receptive field for each output pixel

Environment matte is collection of all receptive fields—yes, it’s huge.

Image formation model


Image formation model1
Image formation model structure from motion


Step 1 computing background

Input: structure from motion

Mosaic:

Step 1: Computing background

Clean plate:

Point tracks:


Step 2 computing w
Step 2: Computing structure from motionw...

Input:


Computing w x y u v at a single x y
Computing structure from motionw(x,y,u,v) at a single (x,y)


Assume w i independent
Assume structure from motionwi independent


Composite over new background
Composite over new background structure from motion


A more subtle example

Input: Two images structure from motion

Moving camera

Planar background

- Need priors

A more subtle example


Window example
Window example structure from motion


Discussion

Works well for non-translucent elements structure from motion

need to develop for diffuse

Combination assumes independence

ok for large movements: “an edge crosses the pixel”

Need to develop for general backgrounds

Discussion


A clustering problem
A Clustering Problem structure from motion

  • Watch a movie, recover the cast list

    • Run face detector on every frame

    • Cluster faces

  • Problems

    • Face detector unreliable

    • Large lighting changes

    • Changes in expression

    • Clustering is difficult


A sample sequence
A sample sequence structure from motion


Detected faces
Detected faces structure from motion


Face positions
Face positions structure from motion


Lighting correction
Lighting correction structure from motion


Clustering pairwise distances

Raw distance structure from motion

Clustering: pairwise distances


Clustering pairwise distances1

Transform-invariant distance structure from motion

Clustering: pairwise distances


Clusters tangent distance
Clusters: “tangent distance” structure from motion



Conclusions1

Extend to feature selection, texton clustering etc structure from motion

Remove face detector

Conclusions


ad