
Improved Census Transforms for Resource-Optimized Stereo Vision Wade S. Fife, Member, IEEE, James K. Archibald, Senior Member, IEEE IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 23, NO. 1, JANUARY 2013
Outline • Introduction • Related Work • Proposed Algorithm • Sparse Census Transform • Generalized Census Transform • Hardware Implementation • Experimental Results • Conclusion
Introduction • The challenges: • The enormous amount of computation required to identify the corresponding points in the images. • It is critical to… • maximize the accuracy and throughput of the stereo system • while minimizing the resource requirements
Objective • Propose the sparse census transforms : • Reduce the resource requirements of census-based systems • Maintain correlation accuracy • Propose the generalized census transforms : • A new class of census-like transforms • Increase the robustness and flexibility
Related Work • CensusTransform : • Color • Gradient
Related Work • After aggregation step: Census on colors Census on gradients
Related Work • Sparse census[6] : • Half of the bits The computation costs for the hamming distances are quite large. [6] C. Zinner, M. Humenberger, K. Ambrosch, and W. Kubinger, “An optimized software-based implementation of a census-based stereo matching algorithm,” in Proc. 4th ISVC, 2008, pp. 216–227.
Related Work • Mini-census[8] : [8] N.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang,“Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 792–805, Jun. 2010.
Related Work • Mini-census[8] : • Mini-census adaptive support weight [8] N.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang,“Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 792–805, Jun. 2010.
Related Work • Mini-census[8] :
Sparse Census Transform • Definition : • N: the set of points within a T T window around p • : a new set of N
Transform Point Selection • Goal :minimize the size of the census transform vector • Challenge: Must quantify how much each point in the transform window contributes to overall correlation accuracy • Test correlation accuracy: • Define a sparse census transform consisting of a single point (| | = 1) • Determine how consistently this point leads to correct correlation • 13 13 correlation window (aggregation)
Transform Point Selection • Go Tsukuba Venus Average Bright: Higher correlation accuracy 25 25 neighborhood Teddy Cones
Transform Point Selection • Further from the center : value decreasing • Very near the center : less effective • It is best to choose points that are neither too far from nor too close to the center pixel. • Optimal distance : 2 pixels • If the image is noisy should be slightly further from the center
Transform Point Selection With Gaussian noise ( = 5.12) Tsukuba Venus Tsukuba Venus Average Bright: Higher correlation accuracy 37 37 neighborhood Teddy Teddy Cones Cones
Proposed Sparse Census Transform • Very good correlation accuracy can be achieved using very sparse transforms. 16-point 12-point 8-point 4-point 2-point 1-point
Generalized Census Transform • Goal :greater freedom in choosing the census transform design • Definition : redrawing the transform as a graph 3 3 correlation (aggregation) 3 3 census
Generalized Census Transform • As.. • (1)transform neighborhoods become more and more sparse • (2)fewer pixels are used in the correlation process • selection of points to include in the transform becomes more critical Horizontal + Vertical 2-edge 2-point
Generalized Census Transform symmetric
Proposed Generalized Census Transform • Benefits : • Often require a smaller census transform window (memory) • Increased robustness under varying conditions (noise) 16-edge 12-edge 8-edge 4-edge 2-edge 1-edge
Hardware Implementation • Pipelining :to increase throughput in an FPGA implementation (Field Programmable Gate Array) 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 One input pixel per clock cycle & Output one disparity result per clock cycle Range : 0~3
Hardware Implementation • Correlation window sum (Aggregation) :
Full 7x7 census Ground Truth Left Image 12-edge 4-edge
Full 7x7 census Ground Truth Left Image 12-edge 4-edge
Left Image Full 7x7 census 12-edge 4-edge
Experimental Results LUTs (look-up tables) : the amount of logic required to implement the method FFs : the number of 1-bit registers (the amount of pipelining used) RAMs : the number of 18-kbit block memories Freq. : the maximum operating frequency reported by synthesis
Conclusion • Proposed and analyzed in this paper: • A range of sparse census transforms • reduce hardware resource requirements • attempting to maximize correlation accuracy. • often better than or nearly as good as the full census • Generalized census transforms • increased robustness in the presence of image noise