3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes - PowerPoint PPT Presentation

Slide1 l.jpg
Download
1 / 69

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes Wojciech Matusik, MERL Hanspeter Pfister, MERL Just like a window! 3D TV – Our Vision Immersive Unobtrusive Multi-user 3D TV – Our Goals

Related searches for 3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide1 l.jpg

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes

Wojciech Matusik, MERL

Hanspeter Pfister, MERL


3d tv our vision l.jpg

Just like a window!

3D TV – Our Vision

  • Immersive

  • Unobtrusive

  • Multi-user


3d tv our goals l.jpg

3D TV – Our Goals

  • Capture light flowing through a “window”

  • Real-time transmission

  • Multiview autostereoscopic light field display


Challenges l.jpg

Challenges

  • Bandwidth

  • Processing

  • Real-time

  • Multiview autostereoscopic display

  • End-to-end system

  • Automatic setup and calibration


Contributions l.jpg

Contributions

  • Real-time end-to-end 3D TV system

  • Distributed, scalable architecture

  • Multiview video rendering

  • Computational alignment for 3D displays


Emerging technologies exhibit l.jpg

Emerging Technologies Exhibit


Outline l.jpg

Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


Previous work early beginnings l.jpg

Previous Work – Early Beginnings

  • Stereoscope [Wheatstone 1838]

  • Parallax stereogram [Ives 1903]

  • Integral display [Lippman 1908]

  • Parallax panoramagram [Ives 1928]

  • Lenticular sheets, 1930s

  • Hologram [Gabor 1948]


Previous work acquisition l.jpg

Previous Work - Acquisition

  • Model-based systems

    [Kanade 97], [Gross 03], [Carranza 03]

  • Light-field systems

    [Levoy 96], [Gortler 96], [Schirmacher 01], [Yang 02],

    [Wilburn 02], [Naemura 02]

  • Multiview video compression and transmission

    [Fehn 02], [Magnor 03], [Ramanathan 03], [Yang 02],

    [Tanimoto 03], [Zitnick 04], [Smolic 03]


Previous work displays l.jpg

Previous Work – Displays

  • Holographic displays

    [St.-Hillaire 95], [Maeno 96], [Kajiki 96], [Stanley 00],

    [Huebschman 03]

  • Volumetric displays

    [McKay 00], [Favalora 01]

  • Parallax displays

    [Nakajima 01], [Liao 02], [Moore 96], [Perlin 00]

  • Multi-projector displays

    [Raskar 98], [Li 02], [Humphreys 02]


Outline11 l.jpg

Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


Regular pixels l.jpg

Regular Pixels

  • Ideally emit the same light in all directions

Emitted Light


View dependent pixels l.jpg

View-dependent Pixels

  • Emit different amounts of light/color in different directions

Emitted Light


View dependent pixels14 l.jpg

View-dependent Pixels

Lens or Pinhole = Pixel

High Resolution Screen


View dependent pixels15 l.jpg

View-dependent Pixels

Emitted Light

Lens or Pinhole = Pixel

High Resolution Screen


Rear projection design l.jpg

Rear Projection Design

Lens = Pixel

Semi-transparent Material

Lens


Rear projection design17 l.jpg

Rear Projection Design

Lens = Pixel

Semi-transparent Material

Lens


Rear projection design18 l.jpg

Rear Projection Design

Emitted Light

Lens = Pixel

Semi-transparent Material

Lens


Our rear projection display l.jpg

Our Rear Projection Display


Front projection design l.jpg

Front Projection Design

Lens

Reflective Material


Front projection design21 l.jpg

Front Projection Design

Lens

Reflective Material


Front projection design22 l.jpg

Front Projection Design

Emitted Light

Lens

Reflective Material


Our front projection display l.jpg

Our Front Projection Display


Limitations field of view l.jpg

Limitations – Field of View


Limitations field of view25 l.jpg

Limitations – Field of View


Limitations field of view26 l.jpg

Limitations – Field of View


Limitations field of view27 l.jpg

Limitations – Field of View


Limitations discretization cross talk l.jpg

Limitations – Discretization & Cross-talk


Display trade offs l.jpg

Display Trade-offs

  • Horizontal and vertical parallax 3D TV

    • requires O(n2) bandwidth, computation, & cost

  • Horizontal parallax only 3D TV

    • requires O(n) bandwidth, computation, & cost

    • still produces immersive and convincing 3D experience


Display trade offs30 l.jpg

Display Trade-offs

  • Horizontal and vertical parallax 3D TV

    • requires O(n2) bandwidth, computation, & cost

  • Horizontal parallax only 3D TV

    • requires O(n) bandwidth, computation, & cost

    • still produces immersive and convincing 3D experience


Outline31 l.jpg

Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


System architecture l.jpg

Acquisition

Compression

Transmission

3D Display

System Architecture


Acquisition l.jpg

Acquisition

  • Array (16) of hardware synchronized, calibrated, cameras

  • Distributed acquisition


Compression transmission l.jpg

Compression & Transmission

  • Temporal encoding

    • Each stream encoded separately

    • Uses existing video standards (MPEG-2)

    • Scalable

  • Spatial (multiview) encoding

    • Not scalable

    • Not real-time

  • Both temporal & spatial encoding

    • Best compression


Compression transmission35 l.jpg

Compression & Transmission

  • Temporal encoding

    • Each stream encoded separately

    • Uses existing video standards (MPEG-2)

    • Scalable

  • Spatial (multiview) encoding

    • Not scalable

    • Not real-time

  • Both temporal & spatial encoding

    • Best compression


Compression transmission36 l.jpg

Compression & Transmission


Distributed display l.jpg

Distributed Display

  • Decoders

    • decode video streams

    • send pixel streams to consumers

  • Consumers

    • render video streams


Distributed display38 l.jpg

Distributed Display

  • Controller

    • decides where to send pixels

    • ensures data flow to each consumer is at most kx video stream (k = 3)

    • allows interactively changing display parameters


Our system l.jpg

Our System


Outline40 l.jpg

Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


Display calibration l.jpg

Display Calibration

  • Geometric calibration

    • Project checkerboard pattern

Display Plane

Calibration

Camera

Projector Array


Display calibration42 l.jpg

Display Calibration

  • Geometric calibration

    • Project checkerboard pattern

    • Compute homographies

Display Plane

Calibration

Camera

Projector Array


Display calibration43 l.jpg

Display Calibration

  • Geometric calibration

    • Project checkerboard pattern

    • Compute homographies

    • Compute intersection


Display calibration44 l.jpg

Display Calibration

  • Geometric calibration

    • Project checkerboard pattern

    • Compute homographies

    • Compute intersection

    • Compute maximum rectangle


Display calibration45 l.jpg

Display Calibration

  • Photometric Calibration

    • Compute minimum intensity


Display calibration46 l.jpg

Display Calibration

  • Photometric Calibration

    • Compute minimum intensity

    • Equalize intensities


Outline47 l.jpg

Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


No rendering l.jpg

No Rendering


Simple system limitations l.jpg

Simple System Limitations

  • Physical alignment is impossible

  • No flexibility


Lightfield rendering l.jpg

Lightfield Rendering

  • Unstructured Lumigraph Rendering [Buehler 01]

    – Blend 3 rays / pixel

  • Closest ray

    • 1 ray / pixel


Lightfield rendering51 l.jpg

Lightfield Rendering

  • Flexible and interactive control of

    • Proxy Plane

    • Zero-disparity Plane


Lightfield rendering52 l.jpg

Lightfield Rendering

Scene


Lightfield rendering53 l.jpg

Lightfield Rendering

Scene

Proxy


Lightfield rendering54 l.jpg

Proxy

Lightfield Rendering

Scene


Lightfield rendering55 l.jpg

Proxy

Lightfield Rendering

Scene


Lightfield rendering56 l.jpg

Proxy

Lightfield Rendering

Scene


Effect of display disparities l.jpg

Effect of Display Disparities

Display

Plane


Effect of display disparities58 l.jpg

Effect of Display Disparities

Display

Plane


Effect of display disparities59 l.jpg

Effect of Display Disparities

Display

Plane


Effect of display disparities60 l.jpg

Effect of Display Disparities

Display

Plane


Zero disparity plane l.jpg

Zero-disparity

Plane

Zero-disparity Plane


Zero disparity plane62 l.jpg

Zero-disparity

Plane

Zero-disparity Plane


Zero disparity plane63 l.jpg

Zero-disparity

Plane

Zero-disparity Plane


Zero disparity proxy plane l.jpg

Zero-disparity

Plane

Zero-disparity & Proxy Plane

Proxy


Zero disparity proxy plane65 l.jpg

Zero-disparity

Plane

Zero-disparity & Proxy Plane

Proxy


Outline66 l.jpg

Outline

  • Previous Work

  • 3D Display

  • System Architecture

  • Display Calibration

  • Rendering

  • Future Work and Conclusions


Future work l.jpg

Future Work

  • Computational display

  • Rendering algorithms

  • Color and dynamic range reproduction

  • Scalable compression


Conclusions l.jpg

Conclusions

  • Real-time end-to-end 3D TV system

  • Distributed, scalable architecture

  • 3D TV is technically feasible and economically practical today


Acknowledgements l.jpg

Acknowledgements

  • Joe Marks

  • Leonard McMillan

  • Marc Levoy

  • Jennifer Roderick Pfister

  • Morgan McGuire

  • Peter Sibley

  • Matt Loper

  • Charles Han

  • John Barnwell

  • Bill Yerazunis

  • Tim Weirich

  • Microlens Technology

  • Big 3D

  • E-tech Team


  • Login