1 / 22

Streaming of Panoramas

Treffen der ITG FG 3.1.2 Sci-Worx, Hannover, 2002. Streaming of Panoramas. Dr.-Ing. Peter Eisert Computer Vision & Graphics Image Processing Department Heinrich-Hertz-Institute, Berlin. Motivation. 3D Streaming / Mobile Rendering. Interactive Navigation in Virtual Worlds on Mobile Devices

rowa
Download Presentation

Streaming of Panoramas

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Treffen der ITG FG 3.1.2 Sci-Worx, Hannover, 2002 Streaming of Panoramas Dr.-Ing. Peter Eisert Computer Vision & Graphics Image Processing Department Heinrich-Hertz-Institute, Berlin

  2. Motivation 3D Streaming / Mobile Rendering • Interactive Navigation in Virtual Worlds on Mobile Devices • Virtual city guides (position dependent) • Tourism applications • E-commerce • Requirements / Constraints • 3D scene description located on a server(cylindrical panoramas, concentric mosaics, light fields) • Interactive navigation with low delay(Transmission of navigation decision / position of user) • Low data-rate of wireless channels • Restricted computational power and storage of PDAs

  3. Overview • Image-based rendering • Cylindrical panoramas • Concentric mosaics • Light fields • Streaming of image-based datasets • First implementation: • MPEG-4 client/server system • Streaming of high resolution cylindrical panoramas • Demo

  4. Image-based Rendering Real Picture Computer Graphics • Simple acquisition • Fast rendering • Natural impression • Modeling time consuming • Powerful rendering necessary • Artificial appearance

  5. Cylindrical Panoramas • Simple acquisition • Rotating camera or mirror system • Vertical field of view limited • Navigation: Rotation and zoom • QuickTimeVR

  6. Concentric Mosaics • 3D plenoptic function • Navigation: Rotation, translation in a plane and zoom • Camera motion constrained on planar concentric circles • Acquisition of n+1 vertical lines (slit images)at different radial positions horizontal parallax • Each pushbroom camera contributes one cylindrical panorama

  7. Example of a Concentric Mosaic • a) Panorama acquired at the rotation center (CM0) • b) Panorama acquired at the outmost position (CMn) • c) Panorama as in b) but looking in the opposite direction a) b) c)

  8. Rendering of new Views II

  9. Simplified Acquisition • One single off-center camera rotating on an arm with radius Rn • a) Free movement inside a circle of radius • b) Free movement inside the ring Normal Viewing Direction Tangential Viewing Direction

  10. Demo: Concentric Mosaics (Zhang, Li)

  11. Light Fields • Static 3D scene • Known viewing positions • 2D array of images • 4D plenoptic function(4D array of rays) • no time dependency • no wavelength • no scattering in air (bounding box) • Rendering • arbitrary viewing position and direction • new view interpolated from stored images • no geometry required

  12. Light Field Representation ST array of UV images UV array of ST images

  13. Interpolation • 8 x 8 images of resolution 256 x 256 • Aliasing!! • Many images needed  large data sets

  14. Example: Dragon (Stanford) • 4 slabs • 24 x 24 images each slab • resolution 256 x 256 pixels

  15. 3D Streaming • System for streaming image-based data • Virtual walk-throughs • E-commerce • Free navigation in 3D scenes • Visualization on portable devices • Limited band-width and storage • Efficient coding and transmission necessary

  16. Interaction of all Components Photorealistic Visualization • Image-based representation • Large amount of data Free navigation Representation • Fast transmission of requested data • Prefetching necessary (RTT>100ms) • Intelligent selection of transmitted data • Efficient coding necessary • Scalability • Adapted to representation Coding Navigation Streaming • Efficient packet-based protokoll • RD optimization with delay constraints Bit-Rates: ~ 100 kbit/s - 2 Mbit/s

  17. Current Implementation • Cylindrical, high resolution panoramas • MPEG-4 BIFS scene Representation • JPEG • JPEG-2000 Coding Navigation • MPEG-4 BIFS Player Streaming • MPEG-4 client/server architecture • TCP / UDP

  18. Patches 360°-Panorama, BIFS Scene Cylindrical Panoramas • Navigation: Rotation / Zoom • Random access to image patches • Pre-fetching realized by visibility sensors

  19. Current Viewing Area Visibility Sensor Visibility Sensors • Pre-fetching of neighboring patches • Removal of patches that are no longer used

  20. Demo: Cylindrical Panorama

  21. Summary • Image-based rendering provides tools for photo-realistic visualization of 3D scenes • Cylindrical panoramas / concentric mosaics / light fields • Large datasets require efficient encoding and transmission • Interactive navigation on mobile devices • New applications • virtual guides • e-commerce • MPEG-4 client/server system for streaming high resolution panoramas

  22. Thank you

More Related