Pixel Aligned Warping for Multiprojector Tiled Displays

1. Pixel Aligned WarpingforMultiprojector Tiled Displays Mark Hereld and Rick Stevens Argonne National Laboratory Computation Institute University of Chicago

2. Introduction • Warping image content into projector frame buffers is a fact of life • Driven by geometrical constraints • Never driven by clarity constraint • except in that geometrical constraints are sometimes driven by alignment with nearby or overlapping projected content • What if content crispness is more important?

3. Pixel Exploit • Nyquist and the pixel • Lone illuminated pixel contains high frequency components well beyond Nyquist cutoff • Anti-aliasing techniques are the standard antidote when sharp features don’t fall on pixel boundaries • Often exploited to represent thin, crisp features • Graphics: single pixel width horizontal and vertical lines, and points • Text: sharp and sometimes small fonts

4. Example

5. Warped

6. Rift Line Plate Nearest Neighbor

7. Warped w/Rifts

8. Detail

9. rift dX of warp plate plate X coordinate of pixel Possible Approaches • Modified nearest neighbor: soften rift boundary • Stretch rift lines to avoid cutting through important stuff • Perturb warp according to local measure of steepness • Minimize an energy function that includes terms reflecting affinity of features for pixel boundaries and gives weight to keeping proximal features together • Crystallize pixel values around nucleation centers related to locations of high-contrast features in the image • Segment the image into unwarpable islands of content

10. Possible Approaches • Modified nearest neighbor: soften rift boundary • Stretch rift lines to avoid cutting through important stuff • Perturb warp according to local measure of steepness • Minimize an energy function that includes terms reflecting affinity of features for pixel boundaries and gives weight to keeping proximal features together • Crystallize pixel values around nucleation centers related to locations of high-contrast features in the image • Segment the image into unwarpable islands of content

11. Possible Approaches • Modified nearest neighbor: soften rift boundary • Stretch rift lines to avoid cutting through important stuff • Perturb warp according to local measure of steepness • Minimize an energy function that includes terms reflecting affinity of features for pixel boundaries and gives weight to keeping proximal features together • Crystallize pixel values around nucleation centers related to locations of high-contrast features in the image • Segment the image into unwarpable islands of content

12. Possible Approaches • Modified nearest neighbor: soften rift boundary • Stretch rift lines to avoid cutting through important stuff • Perturb warp according to local measure of steepness • Minimize an energy function that includes terms reflecting affinity of features for pixel boundaries and gives weight to keeping proximal features together • Crystallize pixel values around nucleation centers related to locations of high-contrast features in the image • Segment the image into unwarpable islands of content x,y feature blocking energy Ci-1.Ci Ci.Ci+1 fu,fv k k k f0u,f0v target warp term pixel mis-alignment penalty u,v

13. Possible Approaches • Modified nearest neighbor: soften rift boundary • Stretch rift lines to avoid cutting through important stuff • Perturb warp according to local measure of steepness • Minimize an energy function that includes terms reflecting affinity of features for pixel boundaries and gives weight to keeping proximal features together • Crystallize pixel values around nucleation centers related to locations of high-contrast features in the image • Segment the image into unwarpable islands of content

14. Possible Approaches • Modified nearest neighbor: soften rift boundary • Stretch rift lines to avoid cutting through important stuff • Perturb warp according to local measure of steepness • Minimize an energy function that includes terms reflecting affinity of features for pixel boundaries and gives weight to keeping proximal features together • Crystallize pixel values around nucleation centers related to locations of high-contrast features in the image • Segment the image into unwarpable islands of content

15. Islands in the Ocean • Lock contiguous blocks of high contrast pixels, islands, to one another and to nearest plate • Warp everything else, the ocean, in the usual fashion • Don’t worry about possible artifacts around the edges of the islands, the beaches

16. Algorithm • Compute contrast of each pixel • w.r.t. 3x3 neighborhood • Binarize • Consolidate into islands • Morphological fill & open • Pin island coordinates • to plate containing island centroid • Warp ocean • Transfer islands into warped ocean • cut-and-paste unwarped

17. Example

18. Contrast

19. Labeled

20. Labeled w/Rifts

21. PAW

22. Warped

23. Interesting notion Simple algorithm Islands in the Ocean Very effective Except when it’s not See “Weaknesses” Large warps Rotation Keystone Tiling introduces new problems Overlap or abutting Large islands Collisions Results Strengths Weaknesses

24. Future • Robust rules for handling pathological cases gracefully • Fast implementation • compatible with hardware accelerators • Handle blending areas in tiled displays • Awareness of horizontal and vertical features • allow lines to stretch or shrink but not kink • Apply to image scaling • Particularly high res laptop display driving XGA projector

25. Summary • Pixel edges are commonly exploited to represent sub-Nyquist features • Warping can degrade this crispness • Many approaches to pixel aligned warping that could preserve much of this crispness at the expense of geometrical accuracy • Islands in the Ocean is effective