Hardware Accelerated Voxel Coloring

1. Background estimation Reconstruction by Adaptive Space Carving Image capture Object segmentation Camera calibration Images and segmentation Visibility and noise maps Space Carving Adaptive Space Carving Fixed pre-calibrated cameras setup Occlusion tolerant Calibration Background estimation and segmentation 3D object reconstruction is one of the most investigated topics in computer graphics and vision. Among different techniques, image based reconstruction is considered one of the most promising as high quality digital cameras are becoming a commodity hardware. • Adaptive space carving: • Works on an octree representation of the scene space. • The reconstruction is obtained by a refinement process based on photo-consistency tests. • Uses photometric and silhouette information in multiresolution to detect coarse empty regions as soon as possible. • Classification of the cells: CONSISTENT, INCONSISTENT and UNDEFINED. • Undefined cells are subdivided and classified in later stages. Hardware Accelerated Voxel Coloring Input image Background image Homography H K1Rtm1 K2Rtm2 Registered backgrounds Segmentation problem at the pattern lines due to alignement errors (a). Solution:the interval of confidence for a pixel p(i,j) in the target image is calculated by sampling the pixels from the registered images at a neighborhood of (i,j) whose color is the closest to p(i,j) (b). Z=0 H =K1Rtm1(Rtm2)-1 (K2)-1 Segmentationbased on intervals of confidence Calibration by model recongnition Volumetric carving is a very common technique use for image based reconstruction. It may use silhouette and/or photometric information. Silhouette based methods were successfully used in real-time reconstructions. This is not the case when we consider photometric approaches. =0 Algorithm Levels of refinement Anselmo A. Montenegro†, Luiz Velho†, Paulo Carvalho† and Marcelo Gattass‡ †[anselmo,lvelho,pcezar]@visgraf.impa.br, ‡ mgattass@tecgraf.puc-rio.br wrong (a) correct(a) Initialize the octree root cell with the bounding box of the scene Hand-held camera reconstruction results Fixed cameras reconstruction results Determine the registration planes at the current level Level 5 Level 6 Project images on the current registration plane with resolution compatible to the octree level • Problems with photometric approaches: • Registration and evaluation of thousands of individual elements. • Solution: • Registration based on projective texture mapping. • Photo-consitency evaluation done by GPU programming. Test the consistency of the non- classified cells intersected by the current registration plane Process next octree refinement level Process next registration plane Level 8 Level7 Subdivide undefined cells and colorize photo-consistent cells. Update visibility maps. Final considerations Hand-held camera setup • Still some problems: • Too much elementsMemory waste • Solution: • Hierarchical representation of scene space • Refinement approach • Adaptive Carving • Problems: • Calibration • Background estimation • Solution: • Insert model in the scene • Background estimation by warping images of the scene without the object Last registration plane of the level? In this work we only explored convencional GPU hardware accelerated operations, as in the registration step by projective texture mapping . The mechanism of copying framebuffer information to main memory introduces significant overhead to the overall processing time. We believe that by combining our adaptive approach with photo-consistency test done by GPU programming we can obtain considerable gains in efficiency. NO YES No cell subdivided ? NO YES Zoom Finish