slide1
Download
Skip this Video
Download Presentation
3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes

Loading in 2 Seconds...

play fullscreen
1 / 69

PPT - PowerPoint PPT Presentation


  • 428 Views
  • Uploaded on

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes Wojciech Matusik, MERL Hanspeter Pfister, MERL Just like a window! 3D TV – Our Vision Immersive Unobtrusive Multi-user 3D TV – Our Goals

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'PPT' - andrew


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes

Wojciech Matusik, MERL

Hanspeter Pfister, MERL

3d tv our vision

Just like a window!

3D TV – Our Vision
  • Immersive
  • Unobtrusive
  • Multi-user
3d tv our goals
3D TV – Our Goals
  • Capture light flowing through a “window”
  • Real-time transmission
  • Multiview autostereoscopic light field display
challenges
Challenges
  • Bandwidth
  • Processing
  • Real-time
  • Multiview autostereoscopic display
  • End-to-end system
  • Automatic setup and calibration
contributions
Contributions
  • Real-time end-to-end 3D TV system
  • Distributed, scalable architecture
  • Multiview video rendering
  • Computational alignment for 3D displays
outline
Outline
  • Previous Work
  • 3D Display
  • System Architecture
  • Display Calibration
  • Rendering
  • Future Work and Conclusions
previous work early beginnings
Previous Work – Early Beginnings
  • Stereoscope [Wheatstone 1838]
  • Parallax stereogram [Ives 1903]
  • Integral display [Lippman 1908]
  • Parallax panoramagram [Ives 1928]
  • Lenticular sheets, 1930s
  • Hologram [Gabor 1948]
previous work acquisition
Previous Work - Acquisition
  • Model-based systems

[Kanade 97], [Gross 03], [Carranza 03]

  • Light-field systems

[Levoy 96], [Gortler 96], [Schirmacher 01], [Yang 02],

[Wilburn 02], [Naemura 02]

  • Multiview video compression and transmission

[Fehn 02], [Magnor 03], [Ramanathan 03], [Yang 02],

[Tanimoto 03], [Zitnick 04], [Smolic 03]

previous work displays
Previous Work – Displays
  • Holographic displays

[St.-Hillaire 95], [Maeno 96], [Kajiki 96], [Stanley 00],

[Huebschman 03]

  • Volumetric displays

[McKay 00], [Favalora 01]

  • Parallax displays

[Nakajima 01], [Liao 02], [Moore 96], [Perlin 00]

  • Multi-projector displays

[Raskar 98], [Li 02], [Humphreys 02]

outline11
Outline
  • Previous Work
  • 3D Display
  • System Architecture
  • Display Calibration
  • Rendering
  • Future Work and Conclusions
regular pixels
Regular Pixels
  • Ideally emit the same light in all directions

Emitted Light

view dependent pixels
View-dependent Pixels
  • Emit different amounts of light/color in different directions

Emitted Light

view dependent pixels14
View-dependent Pixels

Lens or Pinhole = Pixel

High Resolution Screen

view dependent pixels15
View-dependent Pixels

Emitted Light

Lens or Pinhole = Pixel

High Resolution Screen

rear projection design
Rear Projection Design

Lens = Pixel

Semi-transparent Material

Lens

rear projection design17
Rear Projection Design

Lens = Pixel

Semi-transparent Material

Lens

rear projection design18
Rear Projection Design

Emitted Light

Lens = Pixel

Semi-transparent Material

Lens

front projection design
Front Projection Design

Lens

Reflective Material

front projection design21
Front Projection Design

Lens

Reflective Material

front projection design22
Front Projection Design

Emitted Light

Lens

Reflective Material

display trade offs
Display Trade-offs
  • Horizontal and vertical parallax 3D TV
    • requires O(n2) bandwidth, computation, & cost
  • Horizontal parallax only 3D TV
    • requires O(n) bandwidth, computation, & cost
    • still produces immersive and convincing 3D experience
display trade offs30
Display Trade-offs
  • Horizontal and vertical parallax 3D TV
    • requires O(n2) bandwidth, computation, & cost
  • Horizontal parallax only 3D TV
    • requires O(n) bandwidth, computation, & cost
    • still produces immersive and convincing 3D experience
outline31
Outline
  • Previous Work
  • 3D Display
  • System Architecture
  • Display Calibration
  • Rendering
  • Future Work and Conclusions
system architecture

Acquisition

Compression

Transmission

3D Display

System Architecture
acquisition
Acquisition
  • Array (16) of hardware synchronized, calibrated, cameras
  • Distributed acquisition
compression transmission
Compression & Transmission
  • Temporal encoding
    • Each stream encoded separately
    • Uses existing video standards (MPEG-2)
    • Scalable
  • Spatial (multiview) encoding
    • Not scalable
    • Not real-time
  • Both temporal & spatial encoding
    • Best compression
compression transmission35
Compression & Transmission
  • Temporal encoding
    • Each stream encoded separately
    • Uses existing video standards (MPEG-2)
    • Scalable
  • Spatial (multiview) encoding
    • Not scalable
    • Not real-time
  • Both temporal & spatial encoding
    • Best compression
distributed display
Distributed Display
  • Decoders
    • decode video streams
    • send pixel streams to consumers
  • Consumers
    • render video streams
distributed display38
Distributed Display
  • Controller
    • decides where to send pixels
    • ensures data flow to each consumer is at most kx video stream (k = 3)
    • allows interactively changing display parameters
outline40
Outline
  • Previous Work
  • 3D Display
  • System Architecture
  • Display Calibration
  • Rendering
  • Future Work and Conclusions
display calibration
Display Calibration
  • Geometric calibration
    • Project checkerboard pattern

Display Plane

Calibration

Camera

Projector Array

display calibration42
Display Calibration
  • Geometric calibration
    • Project checkerboard pattern
    • Compute homographies

Display Plane

Calibration

Camera

Projector Array

display calibration43
Display Calibration
  • Geometric calibration
    • Project checkerboard pattern
    • Compute homographies
    • Compute intersection
display calibration44
Display Calibration
  • Geometric calibration
    • Project checkerboard pattern
    • Compute homographies
    • Compute intersection
    • Compute maximum rectangle
display calibration45
Display Calibration
  • Photometric Calibration
    • Compute minimum intensity
display calibration46
Display Calibration
  • Photometric Calibration
    • Compute minimum intensity
    • Equalize intensities
outline47
Outline
  • Previous Work
  • 3D Display
  • System Architecture
  • Display Calibration
  • Rendering
  • Future Work and Conclusions
simple system limitations
Simple System Limitations
  • Physical alignment is impossible
  • No flexibility
lightfield rendering
Lightfield Rendering
  • Unstructured Lumigraph Rendering [Buehler 01]

– Blend 3 rays / pixel

  • Closest ray
    • 1 ray / pixel
lightfield rendering51
Lightfield Rendering
  • Flexible and interactive control of
    • Proxy Plane
    • Zero-disparity Plane
outline66
Outline
  • Previous Work
  • 3D Display
  • System Architecture
  • Display Calibration
  • Rendering
  • Future Work and Conclusions
future work
Future Work
  • Computational display
  • Rendering algorithms
  • Color and dynamic range reproduction
  • Scalable compression
conclusions
Conclusions
  • Real-time end-to-end 3D TV system
  • Distributed, scalable architecture
  • 3D TV is technically feasible and economically practical today
acknowledgements
Acknowledgements
  • Joe Marks
  • Leonard McMillan
  • Marc Levoy
  • Jennifer Roderick Pfister
  • Morgan McGuire
  • Peter Sibley
  • Matt Loper
  • Charles Han
  • John Barnwell
  • Bill Yerazunis
  • Tim Weirich
  • Microlens Technology
  • Big 3D
  • E-tech Team
ad