Non-Parametric Statistical Permutation Tests for Local Shape Analysis

1. Non-Parametric Statistical Permutation Tests for Local Shape Analysis Martin Styner, UNC Dimitrios Pantazis, Richard Leahy, USC LA Tom Nichols, University of Michigan Ann Arbor

2. TOC • Motivation local shape analysis • Local shape difference/distance measures • Statistical significance maps • Problem: Multiple correlated comparisons • 1st approach: It’s a hack! • 2nd approach: Let’s do it right! • Template free - Hotelling T2 measures • Example Results • Conclusions & Outlook

3. Motivation Shape Analysis • Anatomical studies of brain structures • Changes between patient and healthy controls • Detection, Enhanced understanding, course of disease, pathology • Normal neuro-development • interest in diseases with brain changes • Schizophrenia, autism, fragile-X, Alzheimer's • Information additional to volume • Both volumetric and shape analysis • Shape analysis: where and how?

4. Shape Distances • Shape description: • SPHARM-PDM • M-rep • Normalization: • Rigid Procrustes, brain size normalized • Local scalar distance • Euclidean distance • “Radius” difference • Signed vs absolute

5. Local Shape Analysis • Distance to template • Distance between subject pairs • Sets of distance-maps • Significance map • Statistical test at each point • Mean difference test • P-values • Significance threshold

6. Multiple Comparisons • Lots of correlated statistical tests → Overly optimistic • M-rep: 2x24 tests, SPHARM: 2252 tests • Same problem with other shape descriptions and other difference analysis schemes • Correction needed, overly optimistic • Test locally at given level (e.g. α = 0.05) • Globally incorrect false-positive rate • Bonferroni correction, worst case, assumption: 0% correlation • Correct False-Positive rate at α/n = 0.05/4000 = 0.0000125 • Correct False-Positive rate at 1-(1- α)1/n = 0.0000128

7. 1st Approach: SnPM • Statistical non-Parametric Maps in SPM (SPIE 2004) • Decomposition of distance map into separate images for processing in SnPM • 75% overlap necessary due to distortions • Each image is tested separately in SnPM • ONE BIG HACK: • 6 correlated tests • Averaging in overlap

8. 2nd Approach: Permutations • Non-parametric permutation test using spatially summarized statistics, ISBI 2004 • Correct false positive control (Type II) • Summary: • Random permutations of the group labels • Metric for difference between populations • Spatial normalization for uniform spatial sensitivity • Summarize statistics across whole shape • Choose threshold in summary statistic

9. Statistical Problem • 2 groups: a & b, #member na, nb • Each member: p-features (e.g. 4000) • Test: Is the mean of each feature in the 2 populations the same? • Null hypothesis: The mean of each feature is the same • Permutations of group label leave distributions unchanged under null hypothesis • M permutations • Specific test • Correct false positive rate

10. diff norm Histogram norm diff Summary Statistic Min/Max Non-parametric Permutation Tests • Goal: significance for a vector with 4’000 correlated variables • 50’000 to 100’000 permutations • Extrema statistic: controls false-positive

11. α Single Feature Example • Feature fA,1-fA,n1 vs fB,1-fB,n1 • Compute difference: T0 =|A- B| • Permute group label → A’i,B’I → Ti • Make Histogram of Ti • Histogram = pdf • Sum histogram = cdf • Cdf at 1-α = Threshold

12. Multiple features • Testing a single feature → no problem • Testing multiple features together as a whole, NOT individually • Summary is necessary of all features across the surface • For correct Type II, use an extrema measurement • Right sided distance metrics → Maxima • Left sided distance metrics → Minima

13. Spatial Normalization • Extremal summary is most influenced by regions with higher variance • Assume 2 regions with same difference, but one has larger variance • Region with larger variance contributes more to extremal statistics and thus sensitivity in that region is higher • Normalization of local statistical distributions is necessary for spatially uniform sensitivity

14. α 1-α Spatial Normalization • A) local p-values, non-parametric • Minimum, (1-α) thresh • B) standard deviation, parametric • Maximum, α thresh • C) q-th quantile, non-parametric • q = 68% ~ if Gaussian • Maximum, α thresh • Assumptions: A > C > B • Uniform sensitivity: A > C ~ B • Numerical pdf: C > B > A • Use A • Many permutations • High computation + space costs Shape difference metric Extrema statistics Norm p-value Min-stat Norm  Max-stat

15. Raw vs Corrected P-values • Raw significance map: • 4000 elements, 5% → 200 will be significant at 5% by pure chance, if locations are uncorrelated. • Corrected significance map • Correct control of false negative • Single location significant → whole shape significant • No assumption over local covariance • Overly pessimistic • There is room for improvement!

16. Raw vs Corrected P-values • Raw p-values are comparable • But visualization of raw p-value map is misleading even without statement about significance • Too optimistic, often viewed using linear colormap • P-value correction is non-linear ! Correction factor: F = Raw-P / Corr-P

17. Metric for Group difference • Scalar Local difference: • Signed/Unsigned Euclidean distance • Thickness difference • Pairs, Template • Difference of mean metric → Statistical feature T = |A- B| • Needed: Positive scalar + shape difference metric between populations PDM: Mean difference of Euclidean distance at a selected point Gaussian, passed Lilliefors test 0.01

18. Template Free Stats • No need for a scalar value at each location for each subject • Positive scalar difference value between populations • SPHARM-PDM • So far: Signed/absolute Euclidean distance at each location to template → Scalar field analysis • New: Difference vectors to template → Vector field analysis • Better: Location vector at each location → Template free analysis → Length of difference vector between mean vectors of populations → Hotelling T2 distance between populations = Hotelling T2 is mean difference 2 vector weighted with the pooled Covariance matrix T2 = (μa – μ b) Σa,b (μa –μb) Σa,b = ( (na - 1) Σa + (nb -1) Σb ) / (na +nb - 2)

19. Hotelling T2 histogram Hotelling T2 distance of locations (template free) → 2

20. Results • SnPM hack vs Correct permutation tests • Sample Hippocampus study: Stanley study, resp/non-resp SZ (56) vs Cnt (26) • Both M-rep & PDM • Other example tests

21. SnPM-Hack vs Correct Stat SnPM 0.001 • SnPM too optimistic • relatively good agreement L R 0.05

22. Hippocampus SZ Study Left Right

23. M-rep Shape Analysis Left Right

24. Vector Field Analysis Raw Significance Maps Corr Significance Maps T2 location 0.05 0.001 T2 template difference Abs template distance (scalar)

25. Conclusions of Methods • Multiple comparison correction scheme for local shape analysis • Non-parametric, Permutation-based • Globally correct for false-positive across whole object • Applicable to scalar, vectors, any Euclidean space measures • Black box • Pessimistic estimate

26. NAMIC kit • StatNonParamTestPDM • Command line tool, Win/Linux/MacOSX • E.g. StatNonParamTestPDM <listfile> -out <basename> -surfList -numPerms 50000 -signLevel 0.05 -signSteps 1000 • Output (for meshes) • P-value of global shape difference between the populations (mean T2 across surface) • Mean difference map (effect size) • Hotelling T2 map using robust T2 formula • Raw significance map • Corrected significance map • Mean surfaces of the 2 groups

27. StatNonParamTestPDM • Input: File with list of ITK mesh files • Generic features also supported using customizable text-file input option • Currently in NAMIC-Sandbox (open) • Next: submission to Insight Journal • MeshVisu, combination of Mesh and maps Map Txt 0.011 0.2324 0.123 …..

28. That’s it folks… • Questions

29. No norm max stat L R Corrected Analysis – Spatial Normalization • Without normalization → incorrect, unless uniformity is assumed • High variability → overestimation of significance • Low variability → underestimation of significance •  -normalization ~ 68% normalization