Bootstrap Confidence Intervals for Three-way Component Methods Henk A.L. Kiers University of Groningen The Netherlands. three- way data X. i = 1 . . . . . . I. SUBJECTs. K. OCCASIONS. k=1 . j=1 . . . . . . . J VARIABLES. three- way data X. i = 1 . . . . . . I.
Henk A.L. Kiers
University of Groningen
i = 1
j=1 . . . . . . . J
i = 1
j=1 . . . . . . . J
Is solutions stable? Methods
Is solution ‘reliable’? Would it also hold for population?
Kiers & Van Mechelen report split-half stability results:
Split-half results: rather global stability measures
Bootstrap procedure: Methods
Lots of possibilities, depends on interpretation
Not too bad
Simple effective procedure
Identify solution completely: Methods
uniquely defined outcome parameters
bootstrap straightforward (CI’s directly available)
CP and Tucker3 (principal axes or simple structure)
- solution identified up to scaling/permutation
- further identification needed
does not affect fit Methods
Identify solution up to permutation/reflection
outcome parameters b may differ much, but maybe only due to ordering or sign
bootstrap CI’s unrealistically broad !
how to make b’s comparable?
reorder and reflect columns in (e.g.) Bb, Cbsuch that Bb, Cb optimally resemble B, C
e.g., two equally Methods
direct bootstrap CI’s
takes orientation, order, (too?!) seriously
Identified up to perm./refl.
more realistic solution
cannot fully mimic sample & analysis process
*) thanks to program by Patrick Groenen (procedure by Meulman & Heiser, 1983)
What can go wrong when you take orientation too seriously?
Two-way Example Data: 100 x 8 Data set
PCA: 2 Components
Eigenvalues: 4.04, 3.96, 0.0002, (first two close to each other)
PCA (unrotated) solutions for variables (a,b,c,d,e,f,g,h)
bootstrap 95% confidence ellipses*
What caused these enormous ellipses? Meulman & Heiser, 1983)
Look at loadings for data and some bootstraps:
… leading to standard errors: ...
Conclusion: Meulman & Heiser, 1983)solutions very unstable, hence: loadings seem very uncertain
Configurations of subsamples very similar
So: Weshould’ve considered the whole configuration !
comparable across bootstraps
Identify solution up to nonsingular transformations Meulman & Heiser, 1983)
transform Bb, Cb,Gb so as to optimally resemble B, C, G
BootstrapMethod Meulman & Heiser, 1983)
mean se (B)
mean se (C)
mean se (G)
Some summary results:
Now what CI’s did we find for Anxiety data Meulman & Heiser, 1983)
Plot of confidence ellipses for first two and last two B components
Confidence intervals for Situation Loadings Meulman & Heiser, 1983)
A bit small.... Meulman & Heiser, 1983)
Confidence intervals for Higest Core Values
Should be close Meulman & Heiser, 1983) to 95%
Here are the results
Some details: Meulman & Heiser, 1983) ranges of values per cell in design (and associated se’s)
3. How deal with computational problems (if any) Meulman & Heiser, 1983)
Is there a problem?
Computation times per 500 boostraps:
(Note: largest data size: 100 8 20)
CP: min 4 s, max 452 s
Tucker3 (SimpStr): min 3 s, max 30 s
Tucker3 (OrthogMatch): min 1 s, max 23 s
Problem most severe with CP
How deal with computational problems for CP? Meulman & Heiser, 1983)
some first tests show that this works Meulman & Heiser, 1983)
some first tests show that this does not work