1 / 33

# Extremal Problems of Information Combining - PowerPoint PPT Presentation

Extremal Problems of Information Combining. Information Combining: formulation of the problem Mutual Information Function for the Single Parity Check Codes More Extremal Problems of Information Combining Solutions (with the help of Tchebysheff Systems)

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Extremal Problems of Information Combining ' - Sophia

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Extremal Problems of Information Combining

• Information Combining: formulation of the problem

• Mutual Information Function for

the Single Parity Check Codes

• More Extremal Problems of Information Combining

• Solutions (with the help of Tchebysheff Systems)

for the Single Parity Check Codes

Alexei Ashikhmin

Joint work with Yibo Jiang, Ralf Koetter, Andrew Singer

Encoder

Channel

Channel

Information Transmission

APP

Decoder

Density function of the channel is not known

We only know

We assume that

and that the channel is symmetric

Problem 1

Among all probability distributions such that

determine the probability distribution that maximizes (minimizes)

the mutual information at the output of the optimal decoder

Check nodes processing

Variable nodes processing

From variable nodes

Decoder of

single parity

check code

Input from channel

Interleaver

1. I.Land, P. Hoeher, S.Huettinger, J. Huber, 2003

2. I.Sutskover, S. Shamai, J. Ziv, 2003

Repetition code:

The Binary Erasure Channel (BEC) is the best

The Binary Symmetric Channel (BSC) is the worst

Single Parity Check Code:

is Dual ofRepetition Code

The Binary Erasure Channel (BEC) is the worst

The Binary Symmetric Channel (BSC) is the best

• We would like to solve the optimization problem for the Single Parity Check Codes directly (without using duality)

• Get some improvements

Soft Bits

We callsoft bit, it has support on

Gaussian Channel:

Single Parity

Check Code

Encoder

Single Parity

Check Code

Channel

Channel

Channel

E.Sharon, A. Ashikhmin, S. Litsyn

Results:

Lemma

• is nonnegative and nonincreasing

2. The ratio sequence is nonincreasing

Lemma

In the Binary Erasure Channel all moments are the same

Among all T-consistent probability distributions on [0,1]

such that

determine the probability distribution that maximizes

(minimizes) the second moment

Theorem

Among all binary-input symmetric-output channel

distributions with a fixed mutual information

Binary Symmetric Channel maximizes

and

Binary Erasure Channel minimizes

the second moment

Proof: We use the theory of Tchebysheff Systems

Binary Symmetric, Binary Erasure and an arbitrary channel

with the same mutual information have the following layout of

Let and

1)

2) if for and for

then

This is exactly our case

Problem 1 on extremum of mutual information

and

Problem 2 on extremum of the second moment

are equivalent

Extrema of MMSE

It is known that the channel soft bit is the MMSE estimator fo

thechannel input

Theorem Among all binary-input symmetric-output channels with

fixed the Binary Symmetric Channel has

the minimum MMSE: and the Binary

Erasure Channel has the maximum MMSE:

Single Parity

Check Code

Encoder

Single Parity

Check Code

Channel

Channel

Channel

Problem 3

1)

2)

Among all T-consistent channels find that maximizes

(minimizes)

Among all T-consistent probability distributions on [0,1]

such that

1)

2)

determine the probability distribution that maximizes

(minimizes) the fourth moment

The distribution with mass at , mass at

and mass at 0 maximizes

The distribution with mass at , mass at

and mass at 1 minimizes

Maximizing

Minimizing:

Channel with minimum and maximum and an arbitrary

channel with the same mutual information have the followin

layout of

Problem 3 on extremum of mutual information

and

Problem 4 on extremum of the fourth moment

are equivalent

and is the same as in AWGN channel with this

Definition

A set of real continues functions is called

Tchebysheff system (T-system) if for any real the linear

combination has at most distinct roots at

Definition

A distribution is a nondecreasing, right-continues function

The moment space, defined by

( is the set of valid distributions), is a closed convex cone.

For define

For a given find

that maximizes (minimizes)

If and are T-systems,

and then the extrema are attained uniquely with

distrtibutions and with finitely many mass points

Lower principal

representation

Upper principal

representation

Soft Bits

We callsoft bit, it has support on

Lemma (Sharon, Ashikhmin, Litsyn)

If then

Random variables with this property are called T-consistent

Under constrains

Systems and are T-systems on [0,1].

---------------------------------------------------------------------------------

the distribution that maximizes

has only one mass point at :

has probability mass at

and at

This is exactly the Binary Symmetric Channel