computational photography light field rendering n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Computational Photography Light Field Rendering PowerPoint Presentation
Download Presentation
Computational Photography Light Field Rendering

Loading in 2 Seconds...

play fullscreen
1 / 51

Computational Photography Light Field Rendering - PowerPoint PPT Presentation


  • 128 Views
  • Uploaded on

Computational Photography Light Field Rendering. Jinxiang Chai. Image-based Modeling: Challenging Scenes. Why will they produce poor results? lack of discernible features occlusions difficult to capture high-level structure illumination changes specular surfaces. Some Solutions.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Computational Photography Light Field Rendering' - kolya


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
image based modeling challenging scenes
Image-based Modeling: Challenging Scenes
  • Why will they produce poor results?
  • lack of discernible features
  • occlusions
  • difficult to capture high-level structure
  • illumination changes
  • specular surfaces
some solutions
Some Solutions
  • Use priors to constrain the modeling space
  • Aid modeling process with minimal user interaction
  • Combine image-based modeling with other modeling approaches
videos
Videos
  • Morphable face (click here)
  • Image-based tree modeling (click here)
  • Video trace (click here)
  • 3D modeling by ortho-images (Click here)
spectrum of ibmr
Spectrum of IBMR

Model

Panoroma

Image-based rendering

Image based modeling

Images + Depth

Geometry+ Images

Camera + geometry

Imagesuser input range scans

Images

Light field

Geometry+ Materials

Kinematics

Dynamics

Etc.

outline
Outline
  • Light field rendering [Levoy and Hanranhan SIG96]
  • 3D light field (concentric mosaics) [Shum and He Sig99]
plenoptic function
Plenoptic Function

Can reconstruct every possible view, at every moment, from every position, at every wavelength

Contains every photograph, every movie, everything that anyone has ever seen! it completely captures our visual reality!

An image is a 2D sample of plenoptic function!

P(x,y,z,θ,φ,λ,t)

slide8
Ray
  • Let’s not worry about time and color:
  • 5D
    • 3D position
    • 2D direction

P(x,y,z,q,f)

how can we use this
How can we use this?

Static Lighting

No Change in

Radiance

Static object

Camera

how can we use this1
How can we use this?

Static Lighting

No Change in

Radiance

Static object

Camera

ray reuse
Ray Reuse
  • Infinite line
    • Assume light is constant (vacuum)
  • 4D
    • 2D direction
    • 2D position
    • non-dispersive medium

Slide by Rick Szeliski and Michael Cohen

synthesizing novel views
Synthesizing novel views

Assume we capture every ray in 3D space!

light field lumigraph
Light field / Lumigraph
  • Outside convex space
  • 4D

Empty

Stuff

light field
Light Field
  • How to represent rays?
  • How to capture rays?
  • How to use captured rays for rendering
light field1
Light Field
  • How to represent rays?
  • How to capture rays?
  • How to use captured rays for rendering
light field organization
Light field - Organization
  • 2D position
  • 2D direction

s

q

slide19

Light field - Organization

2D position

2D position

2 plane parameterization

u

s

slide20

Light field - Organization

s,t

u,v

s,t

u,v

2D position

2D position

2 plane parameterization

t

v

u

s

slide21

Light field - Organization

Hold u,v constant

Let s,t vary

What do we get?

u,v

s,t

slide22

Lumigraph - Organization

Hold s,t constant

Let u,v vary

An image

u,v

s,t

slide24

Light field/lumigraph - Capture

  • Idea 1
    • Move camera carefully over u,v plane
    • Gantry
      • see Light field paper

u,v

s,t

stanford multi camera array
Stanford multi-camera array
  • 640 × 480 pixels ×30 fps × 128 cameras
  • synchronized timing
  • continuous streaming
  • flexible arrangement
slide26

Light field/lumigraph - rendering

  • For each output pixel
    • determine s,t,u,v
    • either
      • use closest discrete RGB
      • interpolate near values

s

u

light field lumigraph rendering

s

u

Light field/lumigraph - rendering
  • Nearest
    • closest s
    • closest u
    • draw it
  • Blend 16 nearest
    • quadrilinear interpolation
ray interpolation

s

u

Ray interpolation

Nearest neighbor

Quadrilinear interpolation

Linear interpolation in S-T

light field lumigraph rendering1

Camera Plane

Light Field/Lumigraph Rendering

Light Field

Capture

Rendering

Image Plane

light fields
Light fields
  • Advantages:
    • No geometry needed
    • Simpler computation vs. traditional CG
    • Cost independent of scene complexity
    • Cost independent of material properties and other optical effects
  • Disadvantages:
    • Static geometry
    • Fixed lighting
    • High storage cost
3d plenoptic function
3D plenoptic function
  • Image is 2D
  • Light field/lumigraph is 4D
  • What happens to 3D?
  • - 3D light field subset
  • - Concentric mosaic [Shum and He]
3d light field
3D light field
  • One row of s,t plane
    • i.e., hold t constant

s,t

u,v

3d light field1
3D light field
  • One row of s,t plane
    • i.e., hold t constant
    • thus s,u,v
    • a “row of images”

s

u,v

concentric mosaics shum and he
Concentric mosaics [Shum and He]

Polar coordinate system:

- hold r constant

- thus (θ,u,v)

concentric mosaics
Concentric mosaics

Why concentric mosaic?

- easy to capture

- relatively small in storage size

concentric mosaics1
Concentric mosaics

From above

How to captured images?

concentric mosaics2
Concentric mosaics

From above

How to render a new image?

concentric mosaics3
Concentric mosaics

From above

How to render a new image?

- for each ray, retrieval the closest captured rays

concentric mosaics4
Concentric mosaics

From above

How to render a new image?

- for each ray, retrieval the closest captured rays

concentric mosaics5
Concentric mosaics

From above

How to render a new image?

- for each ray, retrieval the closest captured rays

concentric mosaics6
Concentric mosaics

From above

object

How to retrieval the closest rays?

concentric mosaics7
Concentric mosaics

From above

object

(s,t) interpolation plane

How to retrieve the closest rays?

concentric mosaics8
Concentric mosaics

From above

object

(s,t) interpolation plane

How to retrieve the closest rays?

concentric mosaics9
Concentric mosaics

From above

object

(s,t) interpolation plane

How to retrieve the closest rays?

concentric mosaics10
Concentric mosaics

From above

object

(s,t) interpolation plane

How to retrieve the closest rays?

concentric mosaics11
Concentric mosaics

From above

object

(s,t) interpolation plane

How to synthesize the color of rays?

concentric mosaics12
Concentric mosaics

From above

object

(s,t) interpolation plane

How to synthesize the color of rays?

- bilinear interpolation

concentric mosaics15
Concentric mosaics
  • What are limitations?
concentric mosaics16
Concentric mosaics
  • What are limitations?
  • - limited rendering region?
  • - large vertical distortion