Extremal problems of information combining
Download
1 / 33

Extremal Problems of Information Combining - PowerPoint PPT Presentation


  • 208 Views
  • Uploaded on

Extremal Problems of Information Combining. Information Combining: formulation of the problem Mutual Information Function for the Single Parity Check Codes More Extremal Problems of Information Combining Solutions (with the help of Tchebysheff Systems)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Extremal Problems of Information Combining ' - Sophia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Extremal problems of information combining l.jpg

Extremal Problems of Information Combining

  • Information Combining: formulation of the problem

  • Mutual Information Function for

    the Single Parity Check Codes

  • More Extremal Problems of Information Combining

  • Solutions (with the help of Tchebysheff Systems)

    for the Single Parity Check Codes

Alexei Ashikhmin

Joint work with Yibo Jiang, Ralf Koetter, Andrew Singer


Information transmission l.jpg

Channel

Encoder

Channel

Channel

Information Transmission

APP

Decoder

Density function of the channel is not known

We only know


Optimization problem l.jpg
Optimization Problem

We assume that

and that the channel is symmetric

Problem 1

Among all probability distributions such that

determine the probability distribution that maximizes (minimizes)

the mutual information at the output of the optimal decoder


Slide4 l.jpg

To variable nodes

Check nodes processing

Variable nodes processing

From variable nodes

Decoder of

single parity

check code

Input from channel

Interleaver


Problem is solved already l.jpg
Problem is Solved Already

1. I.Land, P. Hoeher, S.Huettinger, J. Huber, 2003

2. I.Sutskover, S. Shamai, J. Ziv, 2003


Slide6 l.jpg

erasure

Repetition code:

The Binary Erasure Channel (BEC) is the best

The Binary Symmetric Channel (BSC) is the worst

Single Parity Check Code:

is Dual ofRepetition Code

The Binary Erasure Channel (BEC) is the worst

The Binary Symmetric Channel (BSC) is the best


Our goals l.jpg
Our Goals

  • We would like to solve the optimization problem for the Single Parity Check Codes directly (without using duality)

  • Get some improvements


Soft bits l.jpg

Channel

Soft Bits

We callsoft bit, it has support on



Slide10 l.jpg

Binary symmetric channel,

Gaussian Channel:


Slide11 l.jpg

Decoder

Single Parity

Check Code

Encoder

Single Parity

Check Code

Channel

Channel

Channel

E.Sharon, A. Ashikhmin, S. Litsyn

Results:


Properties of the moments l.jpg
Properties of the moments

Lemma

  • is nonnegative and nonincreasing

    2. The ratio sequence is nonincreasing

    Lemma

    In the Binary Erasure Channel all moments are the same


Problem 2 l.jpg
Problem 2

Among all T-consistent probability distributions on [0,1]

such that

determine the probability distribution that maximizes

(minimizes) the second moment


Solution of problem 2 l.jpg
Solution of Problem 2

Theorem

Among all binary-input symmetric-output channel

distributions with a fixed mutual information

Binary Symmetric Channel maximizes

and

Binary Erasure Channel minimizes

the second moment

Proof: We use the theory of Tchebysheff Systems


Slide15 l.jpg

Lemma

Binary Symmetric, Binary Erasure and an arbitrary channel

with the same mutual information have the following layout of


Slide16 l.jpg

Lemma

Let and

1)

2) if for and for

then



Slide18 l.jpg

Problem 1 on extremum of mutual information

and

Problem 2 on extremum of the second moment

are equivalent


Extrema of mmse l.jpg

Channel

Extrema of MMSE

It is known that the channel soft bit is the MMSE estimator fo

thechannel input

Theorem Among all binary-input symmetric-output channels with

fixed the Binary Symmetric Channel has

the minimum MMSE: and the Binary

Erasure Channel has the maximum MMSE:



Problem 3 l.jpg

Decoder

Single Parity

Check Code

Encoder

Single Parity

Check Code

Channel

Channel

Channel

Problem 3

1)

2)

Among all T-consistent channels find that maximizes

(minimizes)


Problem 4 l.jpg
Problem 4

Among all T-consistent probability distributions on [0,1]

such that

1)

2)

determine the probability distribution that maximizes

(minimizes) the fourth moment


Slide23 l.jpg

Theorem

The distribution with mass at , mass at

and mass at 0 maximizes

The distribution with mass at , mass at

and mass at 1 minimizes


Extremum densities l.jpg
Extremum densities

Maximizing

Minimizing:


Slide25 l.jpg

Lemma

Channel with minimum and maximum and an arbitrary

channel with the same mutual information have the followin

layout of


Slide26 l.jpg

Problem 3 on extremum of mutual information

and

Problem 4 on extremum of the fourth moment

are equivalent


Slide27 l.jpg

Assume that

and is the same as in AWGN channel with this


Tchebysheff systems l.jpg
Tchebysheff Systems

Definition

A set of real continues functions is called

Tchebysheff system (T-system) if for any real the linear

combination has at most distinct roots at

Definition

A distribution is a nondecreasing, right-continues function

The moment space, defined by

( is the set of valid distributions), is a closed convex cone.

For define


Slide29 l.jpg

Problem

For a given find

that maximizes (minimizes)


Slide30 l.jpg

Theorem

If and are T-systems,

and then the extrema are attained uniquely with

distrtibutions and with finitely many mass points

Lower principal

representation

Upper principal

representation


Soft bits31 l.jpg

Channel

Soft Bits

We callsoft bit, it has support on

Lemma (Sharon, Ashikhmin, Litsyn)

If then

Random variables with this property are called T-consistent


Slide32 l.jpg

Find extrema of

Under constrains


Slide33 l.jpg

Theorem

Systems and are T-systems on [0,1].

---------------------------------------------------------------------------------

the distribution that maximizes

has only one mass point at :

has probability mass at

and at

This is exactly the Binary Symmetric Channel


ad