Compression with Side Information using Turbo Codes

1. Compression with Side Information usingTurbo Codes Anne Aaron and Bernd Girod Information Systems Laboratory Stanford University Data Compression Conference April 3, 2002

2. Overview • Introduction • Turbo Coder and Decoder • Compression of Binary Sequences • Extension to Continuous-valued Sequences • Joint Source-Channel Coding • Conclusion Compression with Side Information Using Turbo Codes April 3, 2002

3. Encoder Encoder Statistically dependent Decoder Slepian-Wolf Theorem Distributed Source Coding Compression with Side Information Using Turbo Codes April 3, 2002

4. Research Problem • Motivation • Slepian-Wolf theorem: It is possible to compress statistically dependent signals in a distributed manner to the same rate as with a system where the signals are compressed jointly. • Objective • Design practical codes which achieve compression close to the Slepian-Wolf bound Compression with Side Information Using Turbo Codes April 3, 2002

5. Encoder Statistically dependent Decoder Asymmetric Scenario – Compression with Side Information • Compression techniques to send at rate close to H(Y) are well known • Can perform some type of switching for more symmetric rates Compression with Side Information Using Turbo Codes April 3, 2002

6. Our Approach: Turbo Codes • Turbo Codes • Developed for channel coding • Perform close to Shannon channel capacity limit (Berrou, et al., 1993) • Similar work • Garcia-Frias and Zhao, 2001 (Univ. of Delaware) • Bajcsy and Mitran, 2001 (McGill Univ.) Compression with Side Information Using Turbo Codes April 3, 2002

7. Encoder Statistically dependent Decoder System Set-up • X and Y are i.i.d binary sequences X1X2…XL and Y1Y2…YL with equally probable ones and zeros. Let Xi be independent of Yj for ij, but dependent on Yi. X and Y dependency described by pmf P(x|y). • Y is sent at rate RYH(Y) and is available as side information at the decoder Compression with Side Information Using Turbo Codes April 3, 2002

8. L bits in L bits Systematic Convolutional Encoder Rate Systematic Convolutional Encoder Rate Discarded bits bits Interleaverlength L Discarded L bits Turbo Coder Compression with Side Information Using Turbo Codes April 3, 2002

9. Channel probabilities calculations Channel probabilities calculations bits in bits in Decision Interleaverlength L Interleaverlength L Deinterleaverlength L Deinterleaverlength L Turbo Decoder Pchannel SISO Decoder Pa posteriori Pa priori Pextrinsic L bits out Pextrinsic Pa priori SISO Decoder Pchannel Pa posteriori Compression with Side Information Using Turbo Codes April 3, 2002

10. Simulation: Binary Sequences • X-Y relationship • P(Xi=Yi)=1-p and P(XiYi)=p • System • 16-state, Rate 4/5 constituent convolutional codes; • RX=0.5 bit per input bit with no puncturing • Theoretically, must be able to send X without error when H(X|Y)0.5 Compression with Side Information Using Turbo Codes April 3, 2002

11. Results: Compression of Binary Sequences RX=0.5 0.15 bit Compression with Side Information Using Turbo Codes April 3, 2002

12. Results for different rates • Punctured the parity bits to achieve lower rates Compression with Side Information Using Turbo Codes April 3, 2002

13. Turbo Coder Turbo Coder L values ML bits L symbols Interleaverlength L Quantize to 2M levels Convert to bits Convert to bits ML bits Extension to Continuous-Valued Sequences • X and Y are sequences of i.i.d continuous-valued random variables X1X2…XL and Y1Y2…YL. Let Xi be independent of Yj for ij, but dependent on Yi. X and Y dependency described by pdf f(x|y). • Y is known as side information at the decoder To decoder Compression with Side Information Using Turbo Codes April 3, 2002

14. Simulation: Gaussian Sequences • X-Y relationship • X is a sequence of i.i.d Gaussian random variables • Yi=Xi+Zi, where Z is also a sequence of i.i.d Gaussian random variables, independent of X. f(x|y) is a Gaussian probability density function • System • 4-level Lloyd-Max scalar quantizer • 16-state, rate 4/5 constituent convolutional codes • No puncturing so rate is 1 bit/source sample Compression with Side Information Using Turbo Codes April 3, 2002

15. Results: Compression of Gaussian Sequences RX=1 bit/sample 2.8 dB CSNR = ratio of the variance of X and Z Compression with Side Information Using Turbo Codes April 3, 2002

16. Joint Source-Channel Coding • Assume that the parity bits pass through a memoryless channel with capacity C • We can include the channel statistics in the decoder calculations for Pchannel. • From Slepian-Wolf theorem and definition of Channel capacity Compression with Side Information Using Turbo Codes April 3, 2002

17. Results: Joint Source-Channel Coding RX=0.5 BSC with q=0.03 0.15 bit 0.12 bit Compression with Side Information Using Turbo Codes April 3, 2002

18. Conclusion • We can use turbo codes for compression of binary sequences. Can perform close to the Slepian-Wolf bound for lossless distributed source coding. • We can apply the system for compression of distributed continuous-valued sequences. Performs better than previous techniques. • Easy extension to joint source-channel coding Compression with Side Information Using Turbo Codes April 3, 2002