Engaging in Statistical Practice in Academia is Honorable. Michael J. Schell Moffitt Cancer Center & Research Institute. The Methodologist and the Practitioner. The roles of a statistical methodologist and a statistical practitioner, while overlapping somewhat, are distinct.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Michael J. Schell
Moffitt Cancer Center &
The roles of a statistical methodologist and a statistical practitioner, while overlapping somewhat, are distinct.
Academia still needs to learn how to properly define the proper expectations and rewards for the statistical practitioner
Two of following four departments should move to the new building: Mathematics, Meteorology, Oceanography, Statistics. What split would be best?
Apparently some departments are too academically too close for comfort.
I'd like to say a word for the farmer, He come out west and made a lot of changes
He come out west and built a lot of fences, And built 'em right acrost our cattle ranges.
The farmer should be sociable with the cowboy if he rides by and asks for food and water.Don't treat him like a louse make him welcome in your house.
Pro-farmerBut be sure that you lock up your wife and daughters!
While systems designed to cull the best from the rest might be desirable in situations like baseball’s World Series, this ought not to be our design in the training of medical students. We want each medical school graduate to have mastered all the material considered important. At present, those doctors in need of remedial assistance are often not identified until some kind of ex-post screening process is undertaken, such as in-service examinations, specialty board certifications, or malpractice lawsuits.
Innovator’s Prescription, p. 352-3
What roles are today’s leading medical schools playing in training the caregivers we’ll need? In fact, they’re training more and more of the doctors we won’t need, and leaving others to train the professionals we will need. The medical education system in the United States has no means of coordinated planning for training doctors for societal needs.
Innovator’s Prescription, p. 354
The only resource allocation mechanism is that medical students, like the rest of us, choose careers that are intellectually and emotionally engaging, with the most attractive incomes and lifestyles. … This has led to a rapid expansion of subspecialists and a dearth of primary care physicians. …
As for the shortage of primary care physicians, America is turning to immigrants from foreign nursing schools
Innovator’s Prescription, p. 354-5, 357
We anticipate that, given the need for more primary care doctors and nurses that our medical and nursing schools are not meeting, major integrated health-care providers will begin training their own caregivers.
Innovator’s Prescription, p. 360End Result: Disruption of Medical Schools
Arbitrarily small departures from normality result in low power; even when distributions are normal, heteroscedasticity can seriously lower the power of standard ANOVA and regression methods.
… most quantitative articles tend to be too technical for applied researchers.
If the goal is to avoid low power, the worst method is the ANOVA F test.
The scandal of poor medical research, 1994
Why are errors so common? Put simply, much poor research arise because researchers feel compelled for career reasons to carry out research that they are ill equipped to perform, and nobody stops them.
Statistics and ethics in medical research. The misuse of statistics is unethical, 1980
RD Remington: “The discipline of statistics, which emphasizes the development of new methodology and new theoretical structures, is perhaps comparable to its parent discipline, mathematics …”
“The practice of statistics on the other hand, is a profession, involving at every stage judgment and informed choice.”
“The guidelines themselves give only slight notice to a central feature of professional practice of any type – professional judgment.”
HV Roberts: “The Guidelines will help to remind statisticians … that statistical practice requires integrity as well as professional skill. But [the Guidelines] sound bland.”
Temptations are pervasive, yet subtle:
1. to modify one’s best evaluation of the data by what the audience or client wants to hear.
3. to reject needed tools on the grounds that they will prove too difficult to explain.
4. to be lax in seeking out the most appropriate statistical tools.
7. to neglect checks and safeguards against data problems, model failure, and processing errors.
In an environment where ideas are not marketed, first on the market wins
Kolmogorov-Smirnov test, 1937
Duncan’s test, 1950
Kaplan-Meier curves, 1958
Cox regression, 1972
Traditional answer: Bonferroni
Improved answer: Holm
J or Scandinavian Statistics, 1979
3,432 total citations (3/16/09)
288 in 2007
386 in 2008
Personal thanks: Gary Koch and Alex Dmietrienko
Traditional answer: Wilcoxon rank sum test for small samples, t-test for large samples
Better practice: Wilcoxon or normal scores test for large samples, t-test for small samples (such as n=3, but assumptions are critical!)
Hodges and Lehmann (1956) proved that the ARE of the Wilcoxon rank sum test is at least .864
Chernoff and Savage (1958) proved that the ARE of the normal scores test is at least 1
“The above results suggest that on the basis of power, at least for large samples, both the Wilcoxon and normal scores tests are preferable to the t-test for general use.”
Assessed validity only, using three distributions, normal, uniform, exponential and three sample size pairs
Conclusion: t-test is fine.
Later discovery: exponential simulation was done wrong
Highest citation count on any subsequent simulation study (39 articles thru 2000) = 96
J of Modern Applied Statistical Methods, 2005
by SS Sawilowsky
“The knowledge about the large sample asymptotic theory “had even penetrated to the level of a book review written in 1968!”
“The Wilcoxon rank-sum test … show[s] only slight losses in both large and small sample efficiency relative to the t-test in the normal case, while in many non-normal cases, efficiency exceeds 100%”.
But who was listening?
Comment by Diaconis and Lehmann
… even under slight deviations from normality, the t-test can be far from optimal. The poor performance of the t-test, particularly for distributions with heavy tails, can be seen in comparison with nonparametric tests, such as the Wilcoxon or normal scores tests.
… for all distributions with finite variance, the asymptotic relative efficiency relative to t is ≥ .864 for Wilcoxon and ≥ 1 for normal scores.
Basic Practice of Statistics, 4th Ed. 2006 David S. Moore (728 pages)
Non-parametric tests don’t make the book; they appear in the virtual appendix.
Statistics: A Biomedical Introduction, 1977
Hollander and Wolfe
T-test in Chapter 5; Wilcoxon in Chapter 13
Biostatistics, 2nd Ed. van Belle, Fisher, et al., 2004
T-test in Chapter 5; Wilcoxon in Chapter 8
Traditional answer: chi-square test unless expected cell count < 5, then Fisher exact test
Better answer: It depends on the design (what marginals are fixed), For most practical situations: unconditional test
It cheats (is invalid) sometimes and we now can tell where by exact or Monte Carlo results.
The rule of thumb reduces the violations but doesn’t eliminate them.
D’Agostino, Chase, and Belanger recommend that one can use the chi-square test all the time! (TAS, 1988)
1945,7 Barnard introduces test
1949 Barnard renounces own test
1979 Kempthorne: “The importance of the topic cannot be stressed too heavily .. 2x2 contigency tables are the most elemental structures leading to ideas of association.
… It is remarkable that a consensus has not been reached”.
1985 Suissa and Shuster re-introduce test
1985? Mehta (at JSM conference) “I’m not sure I believe in it … but if people want it (chuckles), I might include it in StatXact.”
Campbell (Stat Med, 2007) recommends the Mantel-Haenszel chi-square when expected cell size are 1 or more
Lydersen, Fagerland, and Laake (Stat Med, 2009) recommend the unconditional test unless both margins are fixed
Proschan, and Nason (Bmcs, 2009) “One lesson is that when the design dictates the margins … one should condition on them.”
Ethical Guidelines for Statistical Practice, ’99
“The use of statistics in medical and biomedical research may affect whether individuals live or die”
“… society depends on sound statistical practice … many unresolved issues that deserve frank discussion.”
Educators have an ethical responsibility to properly train their “tool user” students in best practices
“Tool user” statisticians have an ethical responsibility to seek best practice information
… students would rotate from one clerkship to the next in an identical sequence – rather than crisscrossing back and forth as they presently do, and where the quality of students’ educational experiences depends to a frustrating degree on “the luck of the draw”.
Innovator’s Prescription, p. 348
Although there are clear needs for sustaining improvements to medical education, … we are pessimistic that our leading medical schools will be able to act decisively on either front. … The reason lies in the mechanisms of governance in these institutions – which are largely collegial and consensus-driven.
Innovator’s Prescription, p. 358
Territory folks should stick together, Territory folks should all be pals. Cowboys dance with farmer's daughters, Farmers dance with the ranchers' gals.