3d reconstruction class 16 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
3D reconstruction Class 16 PowerPoint Presentation
Download Presentation
3D reconstruction Class 16

Loading in 2 Seconds...

play fullscreen
1 / 63

3D reconstruction Class 16 - PowerPoint PPT Presentation


  • 234 Views
  • Uploaded on

3D reconstruction Class 16 3D photography course schedule Papers http://www.unc.edu/courses/2004fall/comp/290b/089/papers/ Ideas for a project? 3D modeling Aligning range images Pairwise Globally Surface reconstruction Single range image Merged

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '3D reconstruction Class 16' - Ava


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
papers
Papers

http://www.unc.edu/courses/2004fall/comp/290b/089/papers/

3d modeling
3D modeling
  • Aligning range images
    • Pairwise
    • Globally
  • Surface reconstruction
    • Single range image
    • Merged

(some slides from S. Rusinkiewicz, J. Ponce,…)

aligning 3d data
Aligning 3D Data
  • If correct correspondences are known,it is possible to find correct relative rotation/translation
slide7

Norm:

Intermezzo: quaternions

q is aquaternion,

a 2R is itsrealpart, and

2R3 is itsimaginarypart.

q = a + 

Operations on quaternions:

  • Sum of quaternions:
  • Multiplication by a scalar:
  • Quaternion product:
  • Conjugate: )

Note:

slide8

Intermezzo: quaternions and rotations

Let R denote the rotation of angle  about the unit vector u.

Define

Then for any vector ,

Reciprocally, if q = a + ( b, c, d )T is a unit quaternion, the corresponding rotation matrix is:

slide9

n

n

E =  | xi’ – R xi – t |2 .

E =  | yi’ – R yi |2

i=1

i=1

n

 ( xi’ – R xi – t ) .

At a minimum: 0 = E/t = –2

Or.. t = x’ – R x.

i=1

If yi = xi –x and yi’ = xi’ –x’, the error is (at a minimum):

n

n

=  | yi’ q – qyi |2

=  | yi’ – q yi q|2|q|2

i=1

i=1

Linear least squares !!

Estimate rigid transformation

Problem: Find the rotation matrix R and the vector t that

minimize

Or E =

aligning 3d data10
Aligning 3D Data
  • How to find corresponding points?
  • Previous systems based on user input,feature matching, surface signatures, etc.
aligning 3d data11
Aligning 3D Data

Alternative: assume closest points correspond to each other, compute the best transform…

aligning 3d data12
Aligning 3D Data

… and iterate to find alignment

Iterated Closest Points (ICP) [Besl & McKay 92]

Converges if starting position “close enough“

icp variants
ICP Variants
  • Classic ICP algorithm not real-time
  • To improve speed: examine stages of ICP and evaluate proposed variants

[Rusinkiewicz & Levoy, 3DIM 2001]

  • Selecting source points (from one or both meshes)
  • Matching to points in the other mesh
  • Weighting the correspondences
  • Rejecting certain (outlier) point pairs
  • Assigning an error metric to the current transform
  • Minimizing the error metric
icp variant point to plane error metric
ICP Variant – Point-to-Plane Error Metric
  • Using point-to-plane distance instead of point-to-point lets flat regions slide along each other more easily [Chen & Medioni 91]
finding corresponding points
Finding Corresponding Points
  • Finding closest point is most expensive stage of ICP
    • Brute force search – O(n)
    • Spatial data structure (e.g., k-d tree) – O(log n)
    • Voxel grid – O(1), but large constant, slow preprocessing
finding corresponding points16
Finding Corresponding Points
  • For range images, simply project point [Blais 95]
    • Constant-time, fast
    • Does not require precomputing a spatial data structure
high speed icp algorithm
High-Speed ICP Algorithm
  • ICP algorithm with projection-based correspondences, point-to-plane matchingcan align meshes in a few tens of ms.(cf. over 1 sec. with closest-point)
  • [Rusinkiewicz & Levoy, 3DIM 2001]
3d global registration19
3D Global registration

The problem:

  • Given: n scans around an object
  • Goal: align them all
  • First attempt: ICP each scan to one other
3d global registration20
3D Global registration
  • Want method for distributing accumulated error among all scans
approach 1 avoid the problem
Approach #1: Avoid the Problem
  • In some cases have 1 scan that covers large part of surface (e.g., cylindrical scan)
  • Align all other scans to this “anchor”
  • Disadvantage: not always practical to obtain anchor scan
approach 2 the greedy solution
Approach #2: The Greedy Solution
  • Align each new scan to all previous scans
  • Disadvantages:
    • Order dependent
    • Doesn’t spread out error
approach 3 brute force solution
Approach #3: Brute-Force Solution
  • While not converged:
    • For each scan:
      • For each point:
        • For every other scan
          • Find closest point
    • Minimize error w.r.t. transforms of all scans
  • Disadvantage:
    • Solve (np)(np) matrix equation, where n is number of scans and p is number of points per scan
approach 3a slightly less brute force
Approach #3a: Slightly Less Brute-Force
  • While not converged:
    • For each scan:
      • For each point:
        • For every other scan
          • Find closest point
      • Minimize error w.r.t. transform of this scan
  • Faster than previous method (matrices are pp)
graph methods
Graph Methods
  • Many globalreg algorithms create a graph of pairwise alignments between scans

Scan 3

Scan 5

Scan 1

Scan 2

Scan 4

Scan 6

pulli s algorithm
Pulli’s Algorithm
  • Perform pairwise ICPs, record sample (e.g. 200) of corresponding points
  • For each scan, starting w. most connected
    • Align scan to existing set
    • While (change in error) > threshold
      • Align each scan to others
  • All alignments during globalreg phase use precomputed corresponding points
sharp et al algorithm
Sharp et al. Algorithm
  • Perform pairwise ICPs, record only optimal rotation/translation for each
  • Decompose alignment graph into cycles
  • While (change in error) > tolerance
    • For each cycle:
      • Spread out error equally among all scans in the cycle
    • For each scan belonging to more than 1 cycle:
      • Assign average transform to scan
lu and milios algorithm
Lu and Milios Algorithm
  • Perform pairwise ICPs, record optimal rotation/translation and covariance for each
  • Least squares simultaneous minimization of all errors (covariance-weighted)
  • Requires linearization of rotations
    • Worse than the ICP case, since don’t converge to (incremental rotation) = 0
open questions in global registration
Open Questions in Global Registration
  • Best way to do correctly-weighted globalreg without linearizing rotations?
  • How to prevent bias (if many scans in one area, few scans in another)?
  • Robust outlier detection
    • For graph methods, pairwise ICP often automated
    • One bad ICP can throw off the entire model
bad icp in globalreg
Bad ICP in Globalreg

One bad ICP can throw off the entire model

Correct Globalreg

Globalreg Including

Bad ICP

huber s automatic modeling system
Huber’s Automatic Modeling System
  • Start with unaligned cans
  • Generate candidate pairwise alignments using spin images
  • Incrementally add scans, checking consistency
    • ICP error in overlapping regions
    • Free space consistency
    • Occupied space consistency
spin images
Spin Images
  • Johnson and Hebert
  • “Signature” that captures local shape
  • Similar shapes  similar spin images
computing spin images
Computing Spin Images
  • Start with a point on a 3D model
  • Find (averaged) surface normal at that point
  • Define coordinate system centered at this point, oriented according to surface normal and two (arbitrary) tangents
  • Express other points (within some distance) in terms of the new coordinates
computing spin images34
Computing Spin Images
  • Compute histogram of locations of other points, in new coordinate system, ignoring rotation around normal:
spin image parameters
Spin Image Parameters
  • Size of neighborhood
    • Determines whether local or global shapeis captured
    • Big neighborhood: more discriminatory power
    • Small neighborhood: resistance to clutter
  • Size of bins in histogram:
    • Big bins: less sensitive to noise
    • Small bins: captures more detail, less storage
verification overlap distance
Verification – Overlap Distance

Views 1 and 2: 48%, 2.1 mm

Views 1 and 9: 15%, 3.1 mm

verification visibility consistency
Verification – Visibility Consistency

surfaces are close

S2 blocks S1

S1 is not observed

S1

S1

S2

S2

S1

S2

C1

C1

C2

C2

C1

C2

consistentsurfaces

free spaceviolation

occupied spaceviolation

automatically modeling the angel

14

15

13

10

9

3

6

6

12

11

top view

5

5

1

8

8

10

9

13

11

15

2

3

2

4

4

7

7

1

12

14

horizontal slice

Automatically Modeling the Angel

initial model(bad matches in red)

connectivity afterlocal verification

inferring new pose relations

4

4

6

6

7

7

5

5

8

8

10

10

9

9

13

13

11

11

15

15

3

3

2

2

1

1

12

12

14

14

Inferring New Pose Relations

30% overlap

1% overlap

surface reconstruction from range images
Surface Reconstruction from Range Images
  • Often an easier problem than reconstruction from arbitrary point clouds
    • Implicit information about adjacency, connectivity
    • Roughly uniform spacing
surface reconstruction from range images45
Surface Reconstruction From Range Images
  • First, construct surface from each range image
  • Then, merge resulting surfaces
    • Obtain average surface in overlapping regions
    • Control point density
range image tesselation
Range Image Tesselation
  • Given a range image, connect up the neighbors
3d surface model
3D surface model

Depth image

Texture image

Triangle mesh

Textured 3D

Wireframe model

range image tesselation48
Range Image Tesselation
  • Caveat #1: can’t be too aggressive
    • Introduce distance threshold for tesselation
range image tesselation49
Range Image Tesselation
  • Caveat #2: Which way to triangulate?
  • Possible heuristics:
    • Shorter diagonal
    • Dihedral angle closer to 180
    • Maximize smallest angle in both triangles
    • Always the same way (best triangle strips)
scan merging using zippering
Scan Merging Using Zippering
  • Turk & Levoy, 1994
  • Erode geometry in overlapping areas
  • Stitch scans together along seam
  • Re-introduce all data
    • Weighted average
point weighting
Point Weighting
  • Higher weights to points facing the camera
    • Favor higher sampling rates
point weighting53
Point Weighting
  • Lower weights(tapering to 0)near boundaries
    • Smooth blendsbetween views
volumetric reconstruction
Volumetric Reconstruction
  • Implicit function defined volumetrically
  • Usually stored sampled on a 3D grid
    • Can be compressed (e.g., using RLE)
    • Another possibility: hierarchical data structures
  • Can extract isosurface (i.e., subset of space where implicit function = some constant)
volumetric reconstruction overview
Volumetric Reconstruction Overview
  • Generate signed distance function (or something close to it) for each scan
  • Compute average (possibly weighted)
  • Extract isosurface
volumetric integration
Volumetric integration

(Curless and Levoy, Siggraph´96)

range surfaces

signed distance

to surface

volume

weight

(~accuracy)

distance

depth

sensor

surface1

  • use voxel space
  • new surface as zero-crossing
  • (find using marching cubes)
  • least-squares estimate
  • (zero derivative=minimum)

surface2

combined estimate

volumetric reconstruction benefits
Volumetric Reconstruction Benefits
  • Always generates a manifold surface
  • Can control sampling density
  • Averaging of signed distance functions corresponds to averaging the surfaces
volumetric reconstruction drawbacks
Volumetric Reconstruction Drawbacks
  • Represent a 3D entity rather than 2D
    • Running time
    • Storage
  • Resampling step – bandlimits the function
  • Generates consistent topology, but not always the topology you wanted
  • Problems with very thin surfaces
from volume to mesh marching cubes
From volume to mesh:Marching Cubes

First 2D, Marching Squares

“Marching Cubes: A High Resolution 3D Surface Construction Algorithm”,William E. Lorensen and Harvey E. Cline,Computer Graphics (Proceedings of SIGGRAPH '87), Vol. 21, No. 4, pp. 163-169.

from volume to mesh marching cubes60
From volume to mesh:Marching Cubes

“Marching Cubes: A High Resolution 3D Surface Construction Algorithm”,William E. Lorensen and Harvey E. Cline,Computer Graphics (Proceedings of SIGGRAPH '87), Vol. 21, No. 4, pp. 163-169.

from volume to mesh marching cubes61
From volume to mesh:Marching Cubes

Improvement

“Marching Cubes: A High Resolution 3D Surface Construction Algorithm”,William E. Lorensen and Harvey E. Cline,Computer Graphics (Proceedings of SIGGRAPH '87), Vol. 21, No. 4, pp. 163-169.

slide62

Volumetric 3D integration

Multiple depth images

Volumetric integration