3D TV: A Scalable System for
Download
1 / 69

PPT - PowerPoint PPT Presentation


  • 428 Views
  • Updated On :

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes Wojciech Matusik, MERL Hanspeter Pfister, MERL Just like a window! 3D TV – Our Vision Immersive Unobtrusive Multi-user 3D TV – Our Goals

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'PPT' - andrew


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes

Wojciech Matusik, MERL

Hanspeter Pfister, MERL


3d tv our vision l.jpg

Just like a window!

3D TV – Our Vision

  • Immersive

  • Unobtrusive

  • Multi-user


3d tv our goals l.jpg
3D TV – Our Goals

  • Capture light flowing through a “window”

  • Real-time transmission

  • Multiview autostereoscopic light field display


Challenges l.jpg
Challenges

  • Bandwidth

  • Processing

  • Real-time

  • Multiview autostereoscopic display

  • End-to-end system

  • Automatic setup and calibration


Contributions l.jpg
Contributions

  • Real-time end-to-end 3D TV system

  • Distributed, scalable architecture

  • Multiview video rendering

  • Computational alignment for 3D displays



Outline l.jpg
Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


Previous work early beginnings l.jpg
Previous Work – Early Beginnings

  • Stereoscope [Wheatstone 1838]

  • Parallax stereogram [Ives 1903]

  • Integral display [Lippman 1908]

  • Parallax panoramagram [Ives 1928]

  • Lenticular sheets, 1930s

  • Hologram [Gabor 1948]


Previous work acquisition l.jpg
Previous Work - Acquisition

  • Model-based systems

    [Kanade 97], [Gross 03], [Carranza 03]

  • Light-field systems

    [Levoy 96], [Gortler 96], [Schirmacher 01], [Yang 02],

    [Wilburn 02], [Naemura 02]

  • Multiview video compression and transmission

    [Fehn 02], [Magnor 03], [Ramanathan 03], [Yang 02],

    [Tanimoto 03], [Zitnick 04], [Smolic 03]


Previous work displays l.jpg
Previous Work – Displays

  • Holographic displays

    [St.-Hillaire 95], [Maeno 96], [Kajiki 96], [Stanley 00],

    [Huebschman 03]

  • Volumetric displays

    [McKay 00], [Favalora 01]

  • Parallax displays

    [Nakajima 01], [Liao 02], [Moore 96], [Perlin 00]

  • Multi-projector displays

    [Raskar 98], [Li 02], [Humphreys 02]


Outline11 l.jpg
Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


Regular pixels l.jpg
Regular Pixels

  • Ideally emit the same light in all directions

Emitted Light


View dependent pixels l.jpg
View-dependent Pixels

  • Emit different amounts of light/color in different directions

Emitted Light


View dependent pixels14 l.jpg
View-dependent Pixels

Lens or Pinhole = Pixel

High Resolution Screen


View dependent pixels15 l.jpg
View-dependent Pixels

Emitted Light

Lens or Pinhole = Pixel

High Resolution Screen


Rear projection design l.jpg
Rear Projection Design

Lens = Pixel

Semi-transparent Material

Lens


Rear projection design17 l.jpg
Rear Projection Design

Lens = Pixel

Semi-transparent Material

Lens


Rear projection design18 l.jpg
Rear Projection Design

Emitted Light

Lens = Pixel

Semi-transparent Material

Lens



Front projection design l.jpg
Front Projection Design

Lens

Reflective Material


Front projection design21 l.jpg
Front Projection Design

Lens

Reflective Material


Front projection design22 l.jpg
Front Projection Design

Emitted Light

Lens

Reflective Material



Limitations field of view l.jpg
Limitations – Field of View


Limitations field of view25 l.jpg
Limitations – Field of View


Limitations field of view26 l.jpg
Limitations – Field of View


Limitations field of view27 l.jpg
Limitations – Field of View


Limitations discretization cross talk l.jpg
Limitations – Discretization & Cross-talk


Display trade offs l.jpg
Display Trade-offs

  • Horizontal and vertical parallax 3D TV

    • requires O(n2) bandwidth, computation, & cost

  • Horizontal parallax only 3D TV

    • requires O(n) bandwidth, computation, & cost

    • still produces immersive and convincing 3D experience


Display trade offs30 l.jpg
Display Trade-offs

  • Horizontal and vertical parallax 3D TV

    • requires O(n2) bandwidth, computation, & cost

  • Horizontal parallax only 3D TV

    • requires O(n) bandwidth, computation, & cost

    • still produces immersive and convincing 3D experience


Outline31 l.jpg
Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


System architecture l.jpg

Acquisition

Compression

Transmission

3D Display

System Architecture


Acquisition l.jpg
Acquisition

  • Array (16) of hardware synchronized, calibrated, cameras

  • Distributed acquisition


Compression transmission l.jpg
Compression & Transmission

  • Temporal encoding

    • Each stream encoded separately

    • Uses existing video standards (MPEG-2)

    • Scalable

  • Spatial (multiview) encoding

    • Not scalable

    • Not real-time

  • Both temporal & spatial encoding

    • Best compression


Compression transmission35 l.jpg
Compression & Transmission

  • Temporal encoding

    • Each stream encoded separately

    • Uses existing video standards (MPEG-2)

    • Scalable

  • Spatial (multiview) encoding

    • Not scalable

    • Not real-time

  • Both temporal & spatial encoding

    • Best compression



Distributed display l.jpg
Distributed Display

  • Decoders

    • decode video streams

    • send pixel streams to consumers

  • Consumers

    • render video streams


Distributed display38 l.jpg
Distributed Display

  • Controller

    • decides where to send pixels

    • ensures data flow to each consumer is at most kx video stream (k = 3)

    • allows interactively changing display parameters



Outline40 l.jpg
Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


Display calibration l.jpg
Display Calibration

  • Geometric calibration

    • Project checkerboard pattern

Display Plane

Calibration

Camera

Projector Array


Display calibration42 l.jpg
Display Calibration

  • Geometric calibration

    • Project checkerboard pattern

    • Compute homographies

Display Plane

Calibration

Camera

Projector Array


Display calibration43 l.jpg
Display Calibration

  • Geometric calibration

    • Project checkerboard pattern

    • Compute homographies

    • Compute intersection


Display calibration44 l.jpg
Display Calibration

  • Geometric calibration

    • Project checkerboard pattern

    • Compute homographies

    • Compute intersection

    • Compute maximum rectangle


Display calibration45 l.jpg
Display Calibration

  • Photometric Calibration

    • Compute minimum intensity


Display calibration46 l.jpg
Display Calibration

  • Photometric Calibration

    • Compute minimum intensity

    • Equalize intensities


Outline47 l.jpg
Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions



Simple system limitations l.jpg
Simple System Limitations

  • Physical alignment is impossible

  • No flexibility


Lightfield rendering l.jpg
Lightfield Rendering

  • Unstructured Lumigraph Rendering [Buehler 01]

    – Blend 3 rays / pixel

  • Closest ray

    • 1 ray / pixel


Lightfield rendering51 l.jpg
Lightfield Rendering

  • Flexible and interactive control of

    • Proxy Plane

    • Zero-disparity Plane




Lightfield rendering54 l.jpg

Proxy

Lightfield Rendering

Scene


Lightfield rendering55 l.jpg

Proxy

Lightfield Rendering

Scene


Lightfield rendering56 l.jpg

Proxy

Lightfield Rendering

Scene






Zero disparity plane l.jpg

Zero-disparity

Plane

Zero-disparity Plane


Zero disparity plane62 l.jpg

Zero-disparity

Plane

Zero-disparity Plane


Zero disparity plane63 l.jpg

Zero-disparity

Plane

Zero-disparity Plane


Zero disparity proxy plane l.jpg

Zero-disparity

Plane

Zero-disparity & Proxy Plane

Proxy


Zero disparity proxy plane65 l.jpg

Zero-disparity

Plane

Zero-disparity & Proxy Plane

Proxy


Outline66 l.jpg
Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


Future work l.jpg
Future Work

  • Computational display

  • Rendering algorithms

  • Color and dynamic range reproduction

  • Scalable compression


Conclusions l.jpg
Conclusions

  • Real-time end-to-end 3D TV system

  • Distributed, scalable architecture

  • 3D TV is technically feasible and economically practical today


Acknowledgements l.jpg
Acknowledgements

  • Joe Marks

  • Leonard McMillan

  • Marc Levoy

  • Jennifer Roderick Pfister

  • Morgan McGuire

  • Peter Sibley

  • Matt Loper

  • Charles Han

  • John Barnwell

  • Bill Yerazunis

  • Tim Weirich

  • Microlens Technology

  • Big 3D

  • E-tech Team


ad