1 / 26

Distributed Source Coding Using Syndromes (DISCUS): Design and Construction - PowerPoint PPT Presentation

Distributed Source Coding Using Syndromes (DISCUS): Design and Construction. S.Sandeep Pradhan, Kannan Ramchandran IEEE Transactions on Information Theory, vol. 49, no.3, pp.626-643, Mar 2003. Outline. Introduction Preliminaries Encoding with a Fidelity Criterion Problem Formulation

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

PowerPoint Slideshow about 'Distributed Source Coding Using Syndromes (DISCUS): Design and Construction' - bela

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Distributed Source Coding Using Syndromes (DISCUS): Design and Construction

IEEE Transactions on Information Theory,

vol. 49, no.3, pp.626-643, Mar 2003

• Introduction

• Preliminaries

• Encoding with a Fidelity Criterion

• Problem Formulation

• Design Algorithm

• Constructions based on Trellis Codes

• Simulation Results

• Conclusion

• Slepian-Wolf theorem:

By knowing joint distribution of X and Y, without explicitly knowing Y, encoder of X can perform as well as encoder who knows Y.

• Wyner-Ziv Problem:

If decoder knows Y, then the information-theoretic rate-distortion performance for coding X is identical, no matter encoder knows Y or not.(X &Y are Gaussian.)

• Prior work on source quantizer design.

• Contributions:

• Construction of a framework resting on algebraic channel coding principles

• Performance analysis on Gaussian signals.

Source: discrete-alphabet  continuous-valued

Compression: lossless  lossy

• Introduction

• Preliminaries

• Encoding with a Fidelity Criterion

• Problem Formulation

• Design Algorithm

• Constructions based on Trellis Codes

• Simulation Results

• Conclusion

• Example:

X, Y: equiprobable 3-bit binary words

Hamming distance is no more than 1.

Y is available to decoder.

Solution?

Cosets: {000,111},{100,011},{010,101},{001, 110}

Only transmit coset index/syndrome.

-0.5

1

Preliminaries

• Quantization:

Digitizes an analog signal.

Two parameters: a partition and a codebook.

Codebook: [-2, 0.4, 2.3, 6]

yi-2

ai

yi

ai-1

yi-1

Preliminaries

• Lloyd Max Quantization:

partition: ai are midpoints.

codebook: yiare centroids.

Optimal scalar quantization.

• Trellis Coded Quantization (TCQ):[24]

• Dual of TCM

• Example:

• Uniformly distributed source in [-A, A]

• Implemented by Viterbi algorithm

[24] M.W. Marcellin and T. R. Fischer, “Trellis coded quantization of memoryless and Gauss-Markov sources,” IEEE Trans. Commun., vol. 38, pp.82–93, Jan. 1990.

• Introduction

• Preliminaries

• Encoding with a Fidelity Criterion

• Problem Formulation

• Design Algorithm

• Constructions based on Trellis Codes

• Simulation Results

• Conclusion

• Problem Formulation

• X, Y: correlated, memoryless, i.i.d distributed sequences

• Yi = Xi + Ni

• Xi, Yi, Ni: continuous-valued

• Ni: i.i.d distributed, independent from X

• Xi, Ni: zero-mean Gaussian random variables with known variance

• Goal: Form best approximation to X given R bits per sample

• Encoding in blocks of length L

• Distortion measure:

• Min R, s.t. reconstruction fidelity is less than given value D.

System Model: encoder and decoder.

Interplay of source coding, channel coding and estimation

• Design Algorithm

• Source Coding (M1, M2):

• Partition source space:

• Defining source codebook (S)

• Characterizing active codeword by W (r.v.)

• Estimation (M3):

Get best estimate of X (minimizing distortion) conditioned on outcome of Y and the element in .

• Channel Coding (M4, M5):

• Transmit over an error-free channel with rate R (less than Rs)

• Doable: I(W;Y) > 0, so H(W|Y) = H(W) – I(W;Y)

• Build channel code with rate Rc on channel P(Y|W)

• R = Rs – Rc.

• Summary of Design Algorithm:

• M1 and M3:

• minimize Rs, s.t. reconstruction distortion within given criterion.

• M2: maximize I(W;Y).

• M4:

• maximize Rc, s.t. error probability meets a desired tolerance level.

• M5: minimize decoding computational complexity.

• Scalar Quantization and Memoryless Coset Construction (C1):

• Lloyd-Max (memoryless) quantizer

• Memoryless coset partition (M4)

• Example:

L=1, (sample by sample)

Quantization codebook: {r0, r1, …, r7}, (Rs = 3)

Channel coding codebook: {r0, r2, r4, r6}, {r1, r3, r5, r7}. (Rc = 2)

R = Rs – Rc = 1 bit/sample.

• Scalar Quantization and Trellis-Based Coset Construction (C2):

• Scalar quantizer for {Xi}i=1L

• Coset partition (M4) by trellis code.

Codebook (size of 8L), Rs = 3 bits/sample, two cosets

• Example:

Computing syndrome (Rs = 3, Rc = 2)

outcome of quantization be 7, 3, 2, 1, 4.

L = 5,

Syndrome is given by 10110 for 5 samples.

• Trellis-Based Quantization and Memoryless Coset Construction (C3):

• Trellis coded quantizer

• Memoryless coset partition

• Example:

Quantization codebook: Rs = 2

D0={r0, r4}, D1={r1, r5}, D2={r2, r6}, D3={r3, r7}.

Memoryless channel code: Rc = 1

1 coded bit with another 1 uncoded bit (from Y) to recover Di.

• Trellis-Based Quantization and Trellis-Based Coset Coset Construction (C4):

• Trellis coded quantizer

• Trellis coded coset partition

Comparison between C3 and C4.

• Distance Property

• Given a uniform partition, four cases of coset constructions have same distance property.

• Non-uniform quantizer, analyze performance by simulations.

• Introduction

• Preliminaries

• Encoding with a Fidelity Criterion

• Problem Formulation

• Design Algorithm

• Four Constructions

• Simulation Results

• Conclusion

Quantization levels decrease distortion. (C1)

Correlation

-SNR:

ratio of X’s

variance and

N’s variance.

Correlation

-SNR:

ratio of X’s

variance and

N’s variance.

Quantization levels increase prob. Of error. (C1)

Correlation

-SNR:

ratio of X’s

variance and

N’s variance.

Error probability comparison of C1 and C2

(3-4dB gain)

Correlation

-SNR:

ratio of X’s

variance and

N’s variance.

Error probability of C4 codes.

• Constructive practical framework based on algebraic trellis codes.

• Promising performance.