1 / 40

Seamless Multi-Projector Display on Curved Screens

Seamless Multi-Projector Display on Curved Screens. Jeroen van Bar, Thomas Willwacher, Srinivas Rao, Ramesh Raskar Mitsubishi Electric Research Labs Cambridge, MA USA. Curved Screen Displays. Multiple overlapping projectors on curved screens Goal : Replace single-proj

rafi
Download Presentation

Seamless Multi-Projector Display on Curved Screens

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Seamless Multi-Projector Display on Curved Screens Jeroen van Bar, Thomas Willwacher, Srinivas Rao, Ramesh Raskar Mitsubishi Electric Research Labs Cambridge, MA USA

  2. Curved Screen Displays Multiple overlapping projectors on curved screens Goal : • Replace single-proj • Higher resoltn/brightness • Sub-pixel auto-alignment • Parametric solution • Low cost infrastructure Markets Planetarium Curved screens

  3. Dome Projection Techniques Edge-Blended (Tiled/Mosaic) Display • Sub-Frames w/Spherical Mapping & Edge-Blends

  4. 1 Dome Screen 2 4 3

  5. OutlineSeamless Curved Display • Multi-projector low cost method • Simplified Quadric Image Transfer • Calibration with camera-pair • Parametric Rendering solution

  6. Related Work • Conventional Displays • Manual alignment, expensive infrastructure • [Jupiter,Trimensions, CAVE, Planetaria, Flight Simulators] • Planar Screens • Camera in loop, auto calibration, low cost • Exploit homography parameters • [Raskar98,Surati99,Chen00,Brown02 ……] • Curved Screens • Non-parametric solutions • [Jarvis97,Raskar98,Yang01 …] • Parametric • ?, Siggraph 2003

  7. Parametric Approach • Advantages • Lower camera resolution • Tolerance for pixel localization errors • Faster calibration • Efficient well-defined warping • Avoid look up tables

  8. X j i Parametric Image Transfer X i j Planar Homography Quadric Transfer

  9. Planar projective transfer What is homography ? • Two images of 3D points on a plane are related by a 3x3 matrix M i j = A3 x 3 i j Proj 1 Proj 2

  10. What is homography ? Two images of 3D points on a plane Related by a 3x3 matrix ~ j = A3 x 3 i A3 x 3 a1 a2 a3 b1 b2 b3 c1 c2 c3 jx jy 1 ix iy 1 = k j i jx = (a •i)/ (c •i) Proj 1 Proj 2 jy = (b •i)/ (c •i)

  11. 1 2 3 1 4 2 3 4 Planar Displays Current Multi-Cube System MERL Projector Planar Mosaic

  12. Curved ScreensView for a Sweet-spot Projector Sweet spot (Static user)

  13. Calibration for a Sweet-spot Projector Camera at Sweet spot

  14. Discretized non-parametric approach Projector Image p1 p6 Projector c1 Camera at Sweet spot c6 Camera Image Desired Image =

  15. Off-Axis Spherical Distortion Offset Viewpoint Ideal Viewpoint

  16. Fish-eye ProjectionPlanetaria and Digital Dome Theaters • Immersive Production Software • Spitz - PolyDome™ • SkySkan - DigiDome™

  17. OutlineSeamless Curved Display • Multi-projector low cost method • Simplified Quadric Image Transfer • Calibration with camera-pair • Parametric Rendering solution

  18. Ruled quadrics: hyperboloids of one sheet Degenerate ruled quadrics: cone two planes Curved projective transferQuadric classification Projectively equivalent to sphere: sphere ellipsoid paraboloid hyperboloid of two sheets

  19. Quadrics For 3D points X on Quadric X Q : 4x4 symmetric matrix, Nine d.o.f Q In general 9 points in 3D define quadric

  20. Quadric Image Transfer • Quadratic image transfer function [Shashua97] X Quadric written as x’ x 21 params, 4 more than necessary !

  21. Simplified Quadric Image Transfer Our Solution X Based on observation .. x’ x 17 param warp

  22. Simplified Quadric Image Transfer X 17 param warp x’ x Planar homography: 4 corresponding pixels Quadric transfer: 9 corresponding pixels

  23. OutlineSeamless Curved Display • Multi-projector low cost method • Simplified Quadric Image Transfer • Calibration with camera-pair • Parametric Rendering solution

  24. Calibration of Quadric Screens

  25. Approach Calibration • At each projector i, • Project structured pattern • View with stereo camera • Finding camera to projector quadric transfer, Run-time • At each projector i, • Pre-warp input image using

  26. CalibrationFinding relationship between camera and projector Low-res Camera 640x480 images But each Projector 1024x768

  27. Non-linear Refinement Linear Estimation Error ~10 pixels NonLinear Refinement Error ~ 1.0 pixels

  28. Before Blending

  29. After Blending

  30. Intensity Correction in Overlap Projector Framebuffers

  31. Projector Framebuffers

  32. Projector Framebuffer Intensity Weights

  33. OutlineSeamless Curved Display • Multi-projector low cost method • Simplified Quadric Image Transfer • Calibration with camera-pair • Parametric Rendering solution

  34. Rendering a 3D Scene Steps at each projector (Pre-distort vertex 3D location) • For each triangle T with vertices {Mj} • For each vertex M • Find pixel m via VirtualViewProjection( M ) • Find warped pixel m’ via quadricTransferof m • Replace M with m’

  35. Vertex Shader for Quadric Transfer in Cg vertout main( appin IN, uniform float4x4 modelViewProj, uniform float4 constColor, uniform float3x3 A, uniform float3x3 E, uniform float3 e) { vertout OUT; float4 m1 = float4(IN.position.x, IN.position.y, IN.position.z, 1.0f ); float4 m, mi ; float3 m2,mp; float scale; m = mul( modelViewProj, m1); m2.x = m.x/m.w; m2.y = m.y/m.w; m2.z = 1; scale = mul(m2, mul(E,m2)); mp = mul(A,m2) + sqrt(scale)*e; mi.x = m.w * (mp.x)/(mp.z); mi.y = m.w * (mp.y)/(mp.z); mi.zw = m.zw; OUT.position = mi; OUT.color0 = IN.color0; // Use the original per-vertex color specified return OUT; } ParametricWarp

  36. Rendering 2D + 3D scene Concave Dome Convex Dome

  37. Details I Skipped .. • Estimating camera and projector params • Internal and External params • Issue with near-planar 3D points • Finding pixels weights for blending • Non-linear optimization • Rendering • Warping and Depth buffer issues

  38. Seamless Curved Display • Multi-projector low cost method • Simplified Quadric Image Transfer • Complete Parametric calib+rendering solution More info :www.raskar.com/Projector/

  39. Projector Mailing List majordomo@cs.unc.edusubscribe projector Projector bibliography www.raskar.com/Projector/

  40. Advantages • Parametric warp • Lower camera resolution • Tolerance for pixel localization errors • Faster calibration • Efficient well-defined warping

More Related