Extremal Problems of Information Combining

1 / 33

# Extremal Problems of Information Combining - PowerPoint PPT Presentation

Extremal Problems of Information Combining. Information Combining: formulation of the problem Mutual Information Function for the Single Parity Check Codes More Extremal Problems of Information Combining Solutions (with the help of Tchebysheff Systems)

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Extremal Problems of Information Combining' - Sophia

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Extremal Problems of Information Combining

• Information Combining: formulation of the problem
• Mutual Information Function for

the Single Parity Check Codes

• More Extremal Problems of Information Combining
• Solutions (with the help of Tchebysheff Systems)

for the Single Parity Check Codes

Alexei Ashikhmin

Joint work with Yibo Jiang, Ralf Koetter, Andrew Singer

Channel

Encoder

Channel

Channel

Information Transmission

APP

Decoder

Density function of the channel is not known

We only know

Optimization Problem

We assume that

and that the channel is symmetric

Problem 1

Among all probability distributions such that

determine the probability distribution that maximizes (minimizes)

the mutual information at the output of the optimal decoder

To variable nodes

Check nodes processing

Variable nodes processing

From variable nodes

Decoder of

single parity

check code

Input from channel

Interleaver

1. I.Land, P. Hoeher, S.Huettinger, J. Huber, 2003

2. I.Sutskover, S. Shamai, J. Ziv, 2003

erasure

Repetition code:

The Binary Erasure Channel (BEC) is the best

The Binary Symmetric Channel (BSC) is the worst

Single Parity Check Code:

is Dual ofRepetition Code

The Binary Erasure Channel (BEC) is the worst

The Binary Symmetric Channel (BSC) is the best

Our Goals
• We would like to solve the optimization problem for the Single Parity Check Codes directly (without using duality)
• Get some improvements

Channel

Soft Bits

We callsoft bit, it has support on

Binary symmetric channel,

Gaussian Channel:

Decoder

Single Parity

Check Code

Encoder

Single Parity

Check Code

Channel

Channel

Channel

E.Sharon, A. Ashikhmin, S. Litsyn

Results:

Properties of the moments

Lemma

• is nonnegative and nonincreasing

2. The ratio sequence is nonincreasing

Lemma

In the Binary Erasure Channel all moments are the same

Problem 2

Among all T-consistent probability distributions on [0,1]

such that

determine the probability distribution that maximizes

(minimizes) the second moment

Solution of Problem 2

Theorem

Among all binary-input symmetric-output channel

distributions with a fixed mutual information

Binary Symmetric Channel maximizes

and

Binary Erasure Channel minimizes

the second moment

Proof: We use the theory of Tchebysheff Systems

Lemma

Binary Symmetric, Binary Erasure and an arbitrary channel

with the same mutual information have the following layout of

Lemma

Let and

1)

2) if for and for

then

Problem 1 on extremum of mutual information

and

Problem 2 on extremum of the second moment

are equivalent

Channel

Extrema of MMSE

It is known that the channel soft bit is the MMSE estimator fo

thechannel input

Theorem Among all binary-input symmetric-output channels with

fixed the Binary Symmetric Channel has

the minimum MMSE: and the Binary

Erasure Channel has the maximum MMSE:

Decoder

Single Parity

Check Code

Encoder

Single Parity

Check Code

Channel

Channel

Channel

Problem 3

1)

2)

Among all T-consistent channels find that maximizes

(minimizes)

Problem 4

Among all T-consistent probability distributions on [0,1]

such that

1)

2)

determine the probability distribution that maximizes

(minimizes) the fourth moment

Theorem

The distribution with mass at , mass at

and mass at 0 maximizes

The distribution with mass at , mass at

and mass at 1 minimizes

Extremum densities

Maximizing

Minimizing:

Lemma

Channel with minimum and maximum and an arbitrary

channel with the same mutual information have the followin

layout of

Problem 3 on extremum of mutual information

and

Problem 4 on extremum of the fourth moment

are equivalent

Assume that

and is the same as in AWGN channel with this

Tchebysheff Systems

Definition

A set of real continues functions is called

Tchebysheff system (T-system) if for any real the linear

combination has at most distinct roots at

Definition

A distribution is a nondecreasing, right-continues function

The moment space, defined by

( is the set of valid distributions), is a closed convex cone.

For define

Problem

For a given find

that maximizes (minimizes)

Theorem

If and are T-systems,

and then the extrema are attained uniquely with

distrtibutions and with finitely many mass points

Lower principal

representation

Upper principal

representation

Channel

Soft Bits

We callsoft bit, it has support on

Lemma (Sharon, Ashikhmin, Litsyn)

If then

Random variables with this property are called T-consistent

Find extrema of

Under constrains

Theorem

Systems and are T-systems on [0,1].

---------------------------------------------------------------------------------

the distribution that maximizes

has only one mass point at :

has probability mass at

and at

This is exactly the Binary Symmetric Channel