Distributed Compression for Large Camera Arrays

1. Wyner-Ziv Cameras Conventional Cameras … … Distributed Encoding WZ-ENC WZ-ENC Centralized Decoding Geometry Reconstruction WZ-DEC WZ-DEC Geometry Information Rendering Side Information Rate-PSNR Curve Reconstructed Images Wyner-Ziv Coder SA-DCT Coder JPEG2000 Rate = 0.11 bpp PSNR = 39.87 dB Rate = 0.12 bpp PSNR = 38.89 dB Rate = 0.11 bpp PSNR = 37.43 dB Rate = 0.13 bpp PSNR = 44.08 dB Rate = 0.15 bpp PSNR = 41.86 dB Rate = 0.13 bpp PSNR = 42.68 dB Distributed Compression for Large Camera Arrays Xiaoqing Zhu, Anne Aaron and Bernd Girod Department of Electrical Engineering, Stanford University Introduction Wyner-Ziv Codec System Description Wyner-Ziv Decoder Wyner-Ziv Encoder • Large Camera Arrays • Capture multi-viewpoint images of a scene/object. • Potential applications abound: • surveillance, special movie effects. • Image-based rendering [Levoy ’96] • Joint encoding of multiple views cannot be used Shape Architecture Parity Bits Turbo Decoder Turbo Coder Scaler Quantizer Buffer Reconstruction Request Bits [Aaron ’02] • Shape Adaptation • Only encode pixels within the object shape • Object shapes are obtained by chroma keying, compressed with JBIG, and then transmitted to the decoder. Stanford Camera Array, Courtesy of Computer Graphics Lab, Stanford • Rendering of Side Information • The geometry model is reconstructed from silhouette information of the conventional camera views • Side information of the Wyner-Ziv camera views are rendered based on pixel correspondences derived from the geometry. • Proposed Scheme • Apply Wyner-Ziv coding to multi-viewpoint images • Distributed encoding and joint decoding of the images, hence to benefit from the inter-viewpoint coherence. Encoder Complexity Compression Performance Conclusions • Basic Operations • The Wyner-Ziv encoder needs: • 1 quantization step and 3 look-up-table procedures per pixel • shape extraction and coding • The JPEG2000 compressor needs: • Multi-level 2-D DWT: ~ 5 multiplications per pixel • Content-based arithmetic coding • The Wyner-Ziv coder in comparison with JPEG2000 and a SA-DCT coder, using the synthetic Buddha and the real-world Garfield data sets. • Shape information is derived from perfect geometry for Buddha and coded at 0.0814 bpp for Garfield. The overhead of shape coding is counted in the Wyner-Ziv coder and the SA-DCT coder • A distributed compression scheme for large camera arrays. • Low-complexity Wyner-Ziv encoder • Allows independent encoding of each camera view but centralized decoding to exploit inter-viewpoint image similarities. • The existence of rendered side information and the use of shape adaptation techniques enhances compression efficiency. • Experimental results show superior rate-PSNR performance over JPEG2000 and a JPEG-like SA-DCT coder, especially at low bit rates. • Pixel domain coding and shape adaptation help to avoid blurry edges around the object (e.g., in JPEG2000) and blocky artifacts from block-based transform (e.g., in the SA-DCT coder). [Ramanathan ‘01] CPU Execution Time milliseconds(ms) per picture Contact: zhuxq@stanford.edu