Some problems
This presentation is the property of its rightful owner.
Sponsored Links
1 / 53

Some problems... PowerPoint PPT Presentation


  • 62 Views
  • Uploaded on
  • Presentation posted in: General

Some problems. Lens distortion. Uncalibrated structure and motion recovery assumes pinhole cameras Real cameras have real lenses How can we correct distortion , when original calibration is inaccessible?. Even small amounts of lens distortion can upset uncalibrated structure from motion

Download Presentation

Some problems...

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Some problems

Some problems...


Lens distortion

Lens distortion

Uncalibrated structure and motion recovery assumes pinhole cameras

Real cameras have real lenses

How can we correct distortion, when original calibration is inaccessible?


Some problems

Even small amounts of lens distortion can upset uncalibrated structure from motion

A single distortion parameter is enough for mapping and SFX accuracy

Including the parameter kin the multiview relations changes the 8-point algorithm from

You can solve such “Polynomial Eigenvalue Problems”

This is as stable as computation of the Fundamental matrix, so you can use it all the time.


Some problems

E

ven small amounts of lens distortion can upset uncalibrated structure from motion—


A map building problem

A map-building problem

  • Input movie – relatively low distortion

  • Plan view: red is structure, blue is motion

(a) (b)


Effects of distortion

Effects of Distortion

  • Input movie – relatively low distortion

  • Recovered plan view, uncorrected distortion

(a) (c)


Does distortion do that

Distortion of image plane is conflated with focal length

when the camera rotates

[From: Tordoff & Murray, ICPR 2000]

Does distortion do that?


Distortion correction in man made scenes

Distortion correction in man-made scenes


Distortion correction in natural scenes

Distortion correction in natural scenes

[Farid and Popescu, ICCV 2001]

  • In natural images, distortion introduces correlations in frequency domain

  • Choose distortion parameters to minimize correlations in bispectrum

  • Less effective on man-made scenes....


Distortion correction in multiple images

Distortion correction in multiple images

Multiple views, static scene

  • Use motion and scene rigidity [Zhang, Stein, Sawhney, McLauchlan, ...]

    Advantages:

  • Applies to man-made or natural scenes

    Disadvantages:

  • Iterative solutions|require initial estimates


Some problems

A

single distortion parameter is accurate enough for map-building and cinema post production—


Modelling lens distortion

x:xeroxednoxious experimental artifax

p:perfect pinhole perspective pure

Modelling lens distortion

p

p

x

x

Known

Unknown


Single parameter models

Single-parameter models


Single parameter modelling power

Single-parameter modelling power

  • Single-parameter model

  • Radial term only

  • Assumes distortion centre is at centre of image

A one-parameter model suffices


Some problems

A direct solution for k


Look at division model again

Look at division model again


A quick matlab session

A quick matlab session

>> help polyeig

POLYEIG Polynomial eigenvalue problem.

[X,E] = POLYEIG(A0,A1,..,Ap) solves the polynomial eigenvalue problem

of degree p:

(A0 + lambda*A1 + ... + lambda^p*Ap)*x = 0.

The input is [etc etc...]

>>


Algorithm

Algorithm


Some problems

T

his is as stable as computation of the fundamental matrix, so you can use it all the time—


Performance synthetic data

Stable – small errorbars

Biased – not centred on true value

Performance: Synthetic data

0

-0.1

Computed l

-0.2

-0.3

-0.4

0

0.2

0.4

0.6

0.8

1

Noise s (pixels)


Analogy linear ellipse fitting

Best-fit line

Analogy: Linear ellipse fitting

True

Fitted: 10 trials

Data


Performance synthetic data1

Performance: Synthetic data


Performance real sequences

Performance: Real sequences


Some problems

250 pairs

Low distortion

Linear estimate used to initialize nonlinear

Number of inliers changes by [-25..49]

50

40

30

20

10

0

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3


Conclusions

Conclusions


Environment matting

In: magnifying glass moving over background

Out: same magnifying glass, new background

Environment matting


Environment matting why

Learn

light-transport properties of complex optical elements

Previously

Ray tracing geometric models

Calibrated acquisition

Here

Acquisition in situ

Environment matting: why?


Image formation model

Purely 2D-2D

Optical element performs weighted sum of (image of) background at each pixel

suffices for many interesting objects

separate receptive field for each output pixel

Environment matte is collection of all receptive fields—yes, it’s huge.

Image formation model


Image formation model1

Image formation model


Step 1 computing background

Input:

Mosaic:

Step 1: Computing background

Clean plate:

Point tracks:


Step 2 computing w

Step 2: Computing w...

Input:


Computing w x y u v at a single x y

Computing w(x,y,u,v) at a single (x,y)


Assume w i independent

Assume wi independent


Composite over new background

Composite over new background


A more subtle example

Input: Two images

Moving camera

Planar background

- Need priors

A more subtle example


Window example

Window example


Discussion

Works well for non-translucent elements

need to develop for diffuse

Combination assumes independence

ok for large movements: “an edge crosses the pixel”

Need to develop for general backgrounds

Discussion


A clustering problem

A Clustering Problem

  • Watch a movie, recover the cast list

    • Run face detector on every frame

    • Cluster faces

  • Problems

    • Face detector unreliable

    • Large lighting changes

    • Changes in expression

    • Clustering is difficult


A sample sequence

A sample sequence


Detected faces

Detected faces


Face positions

Face positions


Lighting correction

Lighting correction


Clustering pairwise distances

Raw distance

Clustering: pairwise distances


Clustering pairwise distances1

Transform-invariant distance

Clustering: pairwise distances


Clusters tangent distance

Clusters: “tangent distance”


Clusters bayesian tangent distance

Clusters: Bayesian tangent distance


Conclusions1

Extend to feature selection, texton clustering etc

Remove face detector

Conclusions


  • Login