extremal problems of information combining
Download
Skip this Video
Download Presentation
Extremal Problems of Information Combining

Loading in 2 Seconds...

play fullscreen
1 / 33

Extremal Problems of Information Combining - PowerPoint PPT Presentation


  • 208 Views
  • Uploaded on

Extremal Problems of Information Combining. Information Combining: formulation of the problem Mutual Information Function for the Single Parity Check Codes More Extremal Problems of Information Combining Solutions (with the help of Tchebysheff Systems)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Extremal Problems of Information Combining ' - Sophia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
extremal problems of information combining

Extremal Problems of Information Combining

  • Information Combining: formulation of the problem
  • Mutual Information Function for

the Single Parity Check Codes

  • More Extremal Problems of Information Combining
  • Solutions (with the help of Tchebysheff Systems)

for the Single Parity Check Codes

Alexei Ashikhmin

Joint work with Yibo Jiang, Ralf Koetter, Andrew Singer

information transmission

Channel

Encoder

Channel

Channel

Information Transmission

APP

Decoder

Density function of the channel is not known

We only know

optimization problem
Optimization Problem

We assume that

and that the channel is symmetric

Problem 1

Among all probability distributions such that

determine the probability distribution that maximizes (minimizes)

the mutual information at the output of the optimal decoder

slide4

To variable nodes

Check nodes processing

Variable nodes processing

From variable nodes

Decoder of

single parity

check code

Input from channel

Interleaver

problem is solved already
Problem is Solved Already

1. I.Land, P. Hoeher, S.Huettinger, J. Huber, 2003

2. I.Sutskover, S. Shamai, J. Ziv, 2003

slide6

erasure

Repetition code:

The Binary Erasure Channel (BEC) is the best

The Binary Symmetric Channel (BSC) is the worst

Single Parity Check Code:

is Dual ofRepetition Code

The Binary Erasure Channel (BEC) is the worst

The Binary Symmetric Channel (BSC) is the best

our goals
Our Goals
  • We would like to solve the optimization problem for the Single Parity Check Codes directly (without using duality)
  • Get some improvements
soft bits

Channel

Soft Bits

We callsoft bit, it has support on

slide10

Binary symmetric channel,

Gaussian Channel:

slide11

Decoder

Single Parity

Check Code

Encoder

Single Parity

Check Code

Channel

Channel

Channel

E.Sharon, A. Ashikhmin, S. Litsyn

Results:

properties of the moments
Properties of the moments

Lemma

  • is nonnegative and nonincreasing

2. The ratio sequence is nonincreasing

Lemma

In the Binary Erasure Channel all moments are the same

problem 2
Problem 2

Among all T-consistent probability distributions on [0,1]

such that

determine the probability distribution that maximizes

(minimizes) the second moment

solution of problem 2
Solution of Problem 2

Theorem

Among all binary-input symmetric-output channel

distributions with a fixed mutual information

Binary Symmetric Channel maximizes

and

Binary Erasure Channel minimizes

the second moment

Proof: We use the theory of Tchebysheff Systems

slide15
Lemma

Binary Symmetric, Binary Erasure and an arbitrary channel

with the same mutual information have the following layout of

slide16
Lemma

Let and

1)

2) if for and for

then

slide18
Problem 1 on extremum of mutual information

and

Problem 2 on extremum of the second moment

are equivalent

extrema of mmse

Channel

Extrema of MMSE

It is known that the channel soft bit is the MMSE estimator fo

thechannel input

Theorem Among all binary-input symmetric-output channels with

fixed the Binary Symmetric Channel has

the minimum MMSE: and the Binary

Erasure Channel has the maximum MMSE:

problem 3

Decoder

Single Parity

Check Code

Encoder

Single Parity

Check Code

Channel

Channel

Channel

Problem 3

1)

2)

Among all T-consistent channels find that maximizes

(minimizes)

problem 4
Problem 4

Among all T-consistent probability distributions on [0,1]

such that

1)

2)

determine the probability distribution that maximizes

(minimizes) the fourth moment

slide23
Theorem

The distribution with mass at , mass at

and mass at 0 maximizes

The distribution with mass at , mass at

and mass at 1 minimizes

extremum densities
Extremum densities

Maximizing

Minimizing:

slide25
Lemma

Channel with minimum and maximum and an arbitrary

channel with the same mutual information have the followin

layout of

slide26
Problem 3 on extremum of mutual information

and

Problem 4 on extremum of the fourth moment

are equivalent

slide27

Assume that

and is the same as in AWGN channel with this

tchebysheff systems
Tchebysheff Systems

Definition

A set of real continues functions is called

Tchebysheff system (T-system) if for any real the linear

combination has at most distinct roots at

Definition

A distribution is a nondecreasing, right-continues function

The moment space, defined by

( is the set of valid distributions), is a closed convex cone.

For define

slide29
Problem

For a given find

that maximizes (minimizes)

slide30
Theorem

If and are T-systems,

and then the extrema are attained uniquely with

distrtibutions and with finitely many mass points

Lower principal

representation

Upper principal

representation

soft bits31

Channel

Soft Bits

We callsoft bit, it has support on

Lemma (Sharon, Ashikhmin, Litsyn)

If then

Random variables with this property are called T-consistent

slide32

Find extrema of

Under constrains

slide33
Theorem

Systems and are T-systems on [0,1].

---------------------------------------------------------------------------------

the distribution that maximizes

has only one mass point at :

has probability mass at

and at

This is exactly the Binary Symmetric Channel

ad