A new bound for discrete distributions based on Maximum Entropy

1. MaxEnt 2006 CNRS, Paris, France, July 8-13, 2006 A new bound for discrete distributions based on Maximum Entropy †Gzyl H., ‡Novi Inverardi P.L. and ‡Tagliani A. †USB and IESA - Caracas (Venezuela) ‡Department of Computer and Managements Sciences University of Trento (Italy)

2. MaxEnt 2006 Aim of the paper Our aim is to compare some classical bounds for nonnegative integer-valued random variables for estimating the survival probability and a new tighter bound obtained through the Maximum Entropy method constrained by fractional moments given by We will show the superiority of this last bound with respect to the others. A new bound for discrete distributions based on Maximum Entropy

3. MaxEnt 2006 Some classical distribution bounds • Most used candidates as an upper bound of are: • the Chernoff’s bound (Chernoff (1952)) • the moment bound (Philips and Nelson (1995)) • the factorial moment bound • These three bounds come from the Markov Inequality and involve only • integer moments or moment generating function ( mgf ). A new bound for discrete distributions based on Maximum Entropy

4. MaxEnt 2006 Akhiezer’s bound Given two distribution F and G sharing the first 2Q moments, the following distribution bound is well known in literature (Akhiezer (1965)), where , and is the Hankel matrix. For any real value of x,Q(x) represents the maximum mass can be concentrated at the point x under the condition that 2Q moments are met; being Q(x)the reciprocal of a polynomial of degree 2Q in x, the bound goes to zero at the rate x -2Q as x goes to infinity: this behavior gives relatively sharp tail information but no much on the central part of the distribution. A new bound for discrete distributions based on Maximum Entropy

5. MaxEnt 2006 Improving Akhiezer’s bound The main question is how to choose the approximantdistribution sharing the same 2Q moments of F(x) which allows the bound improvement. A key role in achieving this improvement is played by the MaxEnt technique. constrained by integer moments MaxEnt constrained by fractional moments A new bound for discrete distributions based on Maximum Entropy

6. MaxEnt 2006 Improving Akhiezer’s bound:integer moments If • X is a discrete r.v. with pmf with assigned • is the MaxEntM-approximant of P based on the first M integer moments thencombining and we prove that the Akhiezer’s bound can be replaced by the following uniform bound This bound is tighter than Akhiezer’s bound because . Further, H[P] is unknown but, it can be estimated through Aitken D2- method. A new bound for discrete distributions based on Maximum Entropy

7. MaxEnt 2006 Improving Akhiezer’s bound:fractional moments • If • X is a non negative discrete r.v. with pmf • a sequence of M+1fractional moments • with chosen according to • is the MaxEntM-approximant of P, based on • the previous M fractional moments • then the chain of inequalities, similarly in the case of integer moments, gives Numerical evidence proves that , even for small M, then the Akhiezer’s bound can be replaced by the following new uniform and computablebound A new bound for discrete distributions based on Maximum Entropy

8. MaxEnt 2006 Improving Akhiezer’s bound: a new bound The proposed bound is sharper than Akhiezer’s bound in the central part of the distribution and vice versa, the Akhiezer’s bound is sharper than it in the tails. Combining Akhiezer’s and our MaxEntbound, we have a sharper upper bounds, valid for X ≥ 0 and M which guarantees : where is obtained from through a convergence accelerating process, so that may be assumed. Here the maximum value allowed of Q stems from the number of given moments or from numerical stability requirements. A new bound for discrete distributions based on Maximum Entropy

9. MaxEnt 2006 How to calculate fractional moments? As seen, the fractional moments play the role of building blocks of the proposed procedure. But, how tocalculate them? Several scenarios will be analyzed, depending on the available information on the distribution of . This latter is assumed given by a finite or infinite sequence of moments and/or by the mgf. Here we present three cases: both and are known; assigned and existing mgf ; known, R finite or infinite A new bound for discrete distributions based on Maximum Entropy

10. MaxEnt 2006 Case a): both and are known A new bound for discrete distributions based on Maximum Entropy

11. MaxEnt 2006 Case b) assigned and existing mgf A new bound for discrete distributions based on Maximum Entropy

12. Case c) known, R finite or infinite MaxEnt 2006 A new bound for discrete distributions based on Maximum Entropy

13. MaxEnt 2006 A numerical example (1) A new bound for discrete distributions based on Maximum Entropy

14. MaxEnt 2006 A numerical example (1) A new bound for discrete distributions based on Maximum Entropy

15. MaxEnt 2006 A numerical example (1) A new bound for discrete distributions based on Maximum Entropy

16. MaxEnt 2006 A numerical example (2) A new bound for discrete distributions based on Maximum Entropy

17. MaxEnt 2006 A numerical example (2) A new bound for discrete distributions based on Maximum Entropy

18. THANK YOU for your attention! A new bound for discrete distributions based on Maximum Entropy

19. MaxEnt 2006 Density reconstruction via ME technique A new bound for discrete distributions based on Maximum Entropy

20. MaxEnt 2006 Density reconstruction via ME technique • But it is not the unique choice! A new bound for discrete distributions based on Maximum Entropy

21. MaxEnt 2006 An uniform bound A new bound for discrete distributions based on Maximum Entropy

22. MaxEnt 2006 Aitken D2-method A new bound for discrete distributions based on Maximum Entropy

23. MaxEnt 2006 How to choose fractional moments? A new bound for discrete distributions based on Maximum Entropy

24. MaxEnt 2006 Fractional moments A new bound for discrete distributions based on Maximum Entropy

25. MaxEnt 2006 Comparing and combining bounds A new bound for discrete distributions based on Maximum Entropy