Loading in 2 Seconds...

A new bound for discrete distributions based on Maximum Entropy

Loading in 2 Seconds...

- 140 Views
- Uploaded on

Download Presentation
## A new bound for discrete distributions based on Maximum Entropy

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

CNRS, Paris, France, July 8-13, 2006

A new bound for discrete distributions

based on Maximum Entropy

†Gzyl H., ‡Novi Inverardi P.L. and ‡Tagliani A.

†USB and IESA - Caracas (Venezuela)

‡Department of Computer and Managements Sciences

University of Trento (Italy)

Aim of the paper

Our aim is to compare some classical bounds for nonnegative integer-valued random variables for estimating the survival probability

and a new tighter bound obtained through the Maximum Entropy method constrained by fractional moments given by

We will show the superiority of this last bound with respect to the others.

A new bound for discrete distributions based on Maximum Entropy

Some classical distribution bounds

- Most used candidates as an upper bound of are:
- the Chernoff’s bound (Chernoff (1952))
- the moment bound (Philips and Nelson (1995))
- the factorial moment bound
- These three bounds come from the Markov Inequality and involve only
- integer moments or moment generating function ( mgf ).

A new bound for discrete distributions based on Maximum Entropy

Akhiezer’s bound

Given two distribution F and G sharing the first 2Q moments, the following distribution bound is well known in literature (Akhiezer (1965)),

where ,

and

is the Hankel matrix.

For any real value of x,Q(x) represents the maximum mass can be concentrated at the point x under the condition that 2Q moments are met; being Q(x)the reciprocal of a polynomial of degree 2Q in x, the bound goes to zero at the rate x -2Q as x goes to infinity: this behavior gives relatively sharp tail information but no much on the central part of the distribution.

A new bound for discrete distributions based on Maximum Entropy

Improving Akhiezer’s bound

The main question is

how to choose

the approximantdistribution sharing the same 2Q moments of F(x) which allows

the bound improvement.

A key role in achieving this improvement is played by the MaxEnt technique.

constrained by integer moments

MaxEnt

constrained by fractional moments

A new bound for discrete distributions based on Maximum Entropy

Improving Akhiezer’s bound:integer moments

If

- X is a discrete r.v. with pmf with assigned
- is the MaxEntM-approximant of P based on

the first M integer moments

thencombining

and

we prove that the Akhiezer’s bound can be replaced by the following uniform

bound

This bound is tighter than Akhiezer’s bound because .

Further, H[P] is unknown but, it can be estimated through Aitken D2- method.

A new bound for discrete distributions based on Maximum Entropy

Improving Akhiezer’s bound:fractional moments

- If
- X is a non negative discrete r.v. with pmf
- a sequence of M+1fractional moments
- with chosen according to
- is the MaxEntM-approximant of P, based on
- the previous M fractional moments
- then the chain of inequalities, similarly in the case of integer moments, gives

Numerical evidence proves that , even for small M, then the Akhiezer’s bound can be replaced by the following new uniform and computablebound

A new bound for discrete distributions based on Maximum Entropy

Improving Akhiezer’s bound: a new bound

The proposed bound is sharper than Akhiezer’s bound in the central part of the distribution and vice versa, the Akhiezer’s bound is sharper than it in the tails. Combining Akhiezer’s and our MaxEntbound, we have a sharper upper

bounds, valid for X ≥ 0 and M which guarantees :

where is obtained from through a convergence accelerating process, so that may be assumed. Here the maximum value allowed of Q stems from the number of given moments or from numerical stability requirements.

A new bound for discrete distributions based on Maximum Entropy

How to calculate fractional moments?

As seen, the fractional moments play the role of building blocks of the

proposed procedure.

But, how tocalculate them?

Several scenarios will be analyzed, depending on the available information on

the distribution of . This latter is assumed given by a finite or infinite

sequence of moments and/or by the mgf.

Here we present three cases:

both and are known;

assigned and existing mgf ;

known, R finite or infinite

A new bound for discrete distributions based on Maximum Entropy

Case a): both and are known

A new bound for discrete distributions based on Maximum Entropy

Case b) assigned and existing mgf

A new bound for discrete distributions based on Maximum Entropy

Case c) known, R finite or infinite

MaxEnt 2006

A new bound for discrete distributions based on Maximum Entropy

Density reconstruction via ME technique

A new bound for discrete distributions based on Maximum Entropy

Density reconstruction via ME technique

- But it is not the unique choice!

A new bound for discrete distributions based on Maximum Entropy

How to choose fractional moments?

A new bound for discrete distributions based on Maximum Entropy

Comparing and combining bounds

A new bound for discrete distributions based on Maximum Entropy

Download Presentation

Connecting to Server..