Information criteria. What function fits best ?. The more free parameters a model has the higher will be R 2 . The more parsimonious a model is the lesser is the bias towards type I errors. Explained variance. Bias. The optimal number of model parameters.
Themorefreeparameters a model hasthehigher will be R2.
Themoreparsimonious a model isthelesseristhebiastowardstype I errors.
Theoptimalnumber of model parameters
We have to find a compromisbetweengoodness of fit and bias!
k: number of model parameters +1
L: maximum likelihood estimate of the model
The preferred model is the one with the lowest AIC.
If the parameter errors are normal and independent we get
n: number data points
RSS: residual sums of squares
If we fit using R2:
If we fit using c2:
At small sample size we should use the following correction
We getthesurprisingresultthattheseeminglyworstfitting model appears to be thepreferred one.
A single outliermakesthedifference. The single high residualmakestheexponentialfittingworse
ApproximatelyDAIC isstatisticalysignificantinfavor of the model withthesmaller AIC atthe 5% errorbenchmarkif |DAIC| > 2.
The last model is not significantly (5% level) different from the second model.
AIC model selectionserves to find the bestdescriptor of observedstructure.
It is a hypothesisgeneratingmethod.
It does not test for significance
Model selectionusingsignificancelevelsis a hypothesistestingmethod.
Significancelevels and AIC must not be usedtogether.