Günter Zech. Likelihood intervals vs. coverage intervals. (What is better: exclude parameter with high likelihood by coverage interval or have bad coverage in a likelihood ratio interval?). Likelihood ratio. Coverage. Parameters are included if.
Likelihood intervals vs. coverage intervals
(What is better: exclude parameter with high likelihood by coverage interval
or have bad coverage in a likelihood ratio interval?)
Parameters are included if
observation has relatively high p.d.f. (competition)
observation has high
g.o.f.-probability (no competition)
About 68 % of intervals contain true value. They cannot be combined.
but only half a st.dev off the H2 prediction
H1: N(7.5, 2.5)
H2: N(50, 100)
in favor of H1
Likelihood ratio favors hypothesis 1
Frequentist view excludes hypothesis 1 at 1 st. dev. but supports hypothesis 2
Assume following theory:
(includes discrete hypotheses H1 and H2 for specific values of m)
The likelihood ratio limit does not cover.
The coverage inteval excludes hig likelihood values.
What is preferable?
Asume earthquake theory following a similar formula as shown in the previous slide with different constants, and specific parameter values:
H1: predicts eathquake at 2008, Feb. 13 +- 1 day
H2: predicts eathquake at 2010, Feb. 13 +- 2 years
Assume earthquake happens at 2008, Feb. 15
Frequentist would exclude H1 by 2 st.dev.!
It is clear that H1 should not be excluded but why exclude H2 even though the observation is only half a standard deviation off the prediction?
Answer: Only one parameter is correct. If the parameter corresponding to H1 is very likely, H2 must be unlikely.
We should not require likelihood ratio intervals to cover.
Coverage intervals excluding regions of high likelihood are very problematic.
There is nothing wrong with conservative upper limits.
(It is better to present a conservative limit than a doubtfull one)
Including systematic errors
More specifically: calibration uncertainties, theoretical uncertainties, uncertainties which are not improving with 1/sqrt(N)
The only reasonable way is to invent a Bayesian p.d.f. and to integrate out the corresponding parameter. For upper limits one should be conservative. (It is better to overestimate the errors than to underestimate them.
Priors should incorporate prior knowledge and not be selected such that they produce the output you like.