cv perceiving 3d from 2d n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
CV: Perceiving 3D from 2D PowerPoint Presentation
Download Presentation
CV: Perceiving 3D from 2D

Loading in 2 Seconds...

play fullscreen
1 / 35

CV: Perceiving 3D from 2D - PowerPoint PPT Presentation


  • 131 Views
  • Uploaded on

CV: Perceiving 3D from 2D. Many cues from 2D images enable interpretation of the structure of the 3D world producing them. Topic roadmap. Labeling 3D structure in a 2D image Labeling constraints on edge graphs Huffman-Clowes-Waltz labeling Other cues motion parallax

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'CV: Perceiving 3D from 2D' - carolena


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
cv perceiving 3d from 2d

CV: Perceiving 3D from 2D

Many cues from 2D images enable interpretation of the structure of the 3D world producing them

MSU CSE 803 Stockman

topic roadmap
Topic roadmap
  • Labeling 3D structure in a 2D image
  • Labeling constraints on edge graphs
  • Huffman-Clowes-Waltz labeling
  • Other cues

motion parallax

shape from texture or shading

stereo from two 2d images

MSU CSE 803 Stockman

topic roadmap mathematical models
Topic roadmap: mathematical models
  • Shape from shading
  • Depth from stereo
  • Depth from focus
  • Perspective transformation (review)

MSU CSE 803 Stockman

many 3d cues
Many 3D cues

How can humans and other machines reconstruct the 3D nature of a scene from 2D images?

What other world knowledge needs to be added in the process?

MSU CSE 803 Stockman

vocabulary for image labeling

Vocabulary for image labeling

Interpret the local structure of the scene in the image space

MSU CSE 803 Stockman

some terms for local 3d structure
Some terms for local 3D structure

(left) intensity image of 3 blocks (right) result of 5x5 Prewitt operator

Blade: ( > ) as in the blade of a knife, where the normal to the occluding surface element. Occluding and occluded surfaces unrelated.

Crease: (convex + or concave -) formed by an abrupt change to a surface or the joining of two surfaces. Surface on both sides of the crease can be sensed

MSU CSE 803 Stockman

limb smooth object contour
Limb: smooth object contour

egg

Soup can

Limb: (>>) formed by viewing a smooth 3D object, such as an arm or a soup can: when approaching the contour, the surface normal becomes perpendicular to the line of sight.

(Right side of arrow is the occluding surface.)

MSU CSE 803 Stockman

albedo and lighting
Albedo and lighting
  • mark: (M) surface mark or change of “albedo” (reflectance) and not the 3d surface, creating an intensity contour in the image
  • shading: (S) illumination change due to a change in lighting or shadow on the surface, creating an intensity contour in the image

MSU CSE 803 Stockman

labeling image contours interprets the 3d scene structure
Labeling image contours interprets the 3D scene structure

“shadow” relates to illumination, not material

Logo on cup is a “mark” on the material

An egg and a thin cup on a table top lighted from the top right

+

MSU CSE 803 Stockman

intrinsic image stores 3d info in pixels and not intensity
“Intrinsic Image” stores 3D info in “pixels” and not intensity.

For each point of the image, we want depth to the 3D surface point, surface normal at that point, albedo of the surface material, and illumination of that surface point.

MSU CSE 803 Stockman

practice labeling the contours
Practice labeling the contours

(left) an unopened can of Brand X soda is a solid blue can with a bright orange block ‘X’. (right) an empty box with all four flaps open so even the bottom of the box is visible.

MSU CSE 803 Stockman

3d scene versus 2d image
Creases

Corners

Faces

Occlusions (for some viewpoint)

Edges

Junctions

Regions

Blades, limbs, T’s

3D scene versus 2D image

MSU CSE 803 Stockman

labeling of simple polyhedra
Labeling of simple polyhedra

Labeling of a block floating in space. BJ and KI are convex creases. Blades AB, BC, CD, etc model the occlusion of the background. Junction K is a convex trihedral corner. Junction D is a T-junction modeling the occlusion of blade CD by blade JE.

MSU CSE 803 Stockman

trihedral blocks world image junctions only 16 cases
Trihedral Blocks World Image Junctions: only 16 cases!

Only 16 possible junctions in 2D formed by viewing 3D corners formed by 3 planes and viewed from a general viewpoint! From top to bottom: L-junctions, arrows, forks, and T-junctions.

MSU CSE 803 Stockman

challenge

Challenge !

Create a scene shot of no more than 2 blocks that creates all 16 junctions in the image

MSU CSE 803 Stockman

how do we obtain the catalog
How do we obtain the catalog?
  • think about solid/empty assignments to the 8 octants about the X-Y-Z-origin
  • think about non-accidental viewpoints
  • account for all possible topologies of junctions and edges
  • then handle T-junction occlusions

MSU CSE 803 Stockman

blocks world labeling
Blocks world labeling

Left: block floating in space

Right: block glued to a wall at the back

MSU CSE 803 Stockman

try labeling these interpret the 3d structure then label parts
Try labeling these: interpret the 3D structure, then label parts

What does it mean if we can’t label them? If we can label them?

MSU CSE 803 Stockman

1975 researchers very excited
1975 researchers very excited
  • very strong constraints on interpretations
  • several hundred in catalogue when cracks and shadows allowed (Waltz): algorithm works very well with them
  • but, world is not made of blocks!
  • later on, curved blocks world work done but not as interesting

MSU CSE 803 Stockman

interpretation tree search
Interpretation tree search

MSU CSE 803 Stockman

necker cube has multiple interpretations
“Necker cube” has multiple interpretations

Label the different interpretations

A human staring at one of these cubes typically experiences changing interpretations. The interpretation of the two forks (G and H) flip-flops between “front corner” and “back corner”. What is the explanation?

MSU CSE 803 Stockman

depth cues in 2d images

Depth cues in 2D images

MSU CSE 803 Stockman

interposition cue
“Interposition” cue

Def: Interposition occurs when one object occludes another object, thus indicating that the occluding object is closer to the viewer than the occluded object.

MSU CSE 803 Stockman

interposition
interposition
  • T-junctions indicate occlusion: top is occluding edge while bar is the occluded edge
  • Bench occludes lamp post
  • leg occludes bench
  • lamp post occludes fence
  • railing occludes trees
  • trees occlude steeple

MSU CSE 803 Stockman

slide27

Perspective scaling: railing looks smaller at the left; bench looks smaller at the right; 2 steeples are far away

  • Forshortening: the bench is sharply angled relative to the viewpoint; image length is affected accordingly

MSU CSE 803 Stockman

texture gradient reveals surface orientation
Texture gradient reveals surface orientation

( In East Lansing, we call it “corn” not “maize’. )

Note also that the rows appear to converge in 2D

Texture Gradient: change of image texture along some direction, often corresponding to a change in distance or orientation in the 3D world containing the objects creating the texture.

MSU CSE 803 Stockman

3d cues from perspective
3D Cues from Perspective

MSU CSE 803 Stockman

3d cues from perspective1
3D Cues from perspective

MSU CSE 803 Stockman

more 3d cues
More 3D cues

Virtual lines

Falsely perceived interposition

MSU CSE 803 Stockman

more 3d cues1
More 3D cues

2D alignment usually means 3d alignment

2D image curves create perception of 3D surface

MSU CSE 803 Stockman

structured light can enhance surfaces in industrial vision
“structured light” can enhance surfaces in industrial vision

Potatoes with light stripes

Sculpted object

MSU CSE 803 Stockman

shape normals from shading
Shape (normals) from shading

Clearly intensity encodes shape in this case

Cylinder with white paper and pen stripes

Intensities plotted as a surface

MSU CSE 803 Stockman

shape normals from shading1
Shape (normals) from shading

Plot of intensity of one image row reveals the 3D shape of these diffusely reflecting objects.

MSU CSE 803 Stockman