engaging in statistical practice in academia is honorable l.
Skip this Video
Loading SlideShow in 5 Seconds..
Engaging in Statistical Practice in Academia is Honorable PowerPoint Presentation
Download Presentation
Engaging in Statistical Practice in Academia is Honorable

Loading in 2 Seconds...

play fullscreen
1 / 36

Engaging in Statistical Practice in Academia is Honorable - PowerPoint PPT Presentation

  • Uploaded on

Engaging in Statistical Practice in Academia is Honorable. Michael J. Schell Moffitt Cancer Center & Research Institute. The Methodologist and the Practitioner. The roles of a statistical methodologist and a statistical practitioner, while overlapping somewhat, are distinct.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Engaging in Statistical Practice in Academia is Honorable

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
engaging in statistical practice in academia is honorable

Engaging in Statistical Practice in Academia is Honorable

Michael J. Schell

Moffitt Cancer Center &

Research Institute

the methodologist and the practitioner
The Methodologist and the Practitioner

The roles of a statistical methodologist and a statistical practitioner, while overlapping somewhat, are distinct.

Academia still needs to learn how to properly define the proper expectations and rewards for the statistical practitioner

fsu builds a new academic building circa 1978
FSU Builds a New Academic Building Circa 1978

Two of following four departments should move to the new building: Mathematics, Meteorology, Oceanography, Statistics. What split would be best?

Apparently some departments are too academically too close for comfort.

the farmer and the cowman should be friends oklahoma rodgers and hammerstein
The Farmer and the Cowman Should be FriendsOklahoma – Rodgers and Hammerstein


I'd like to say a word for the farmer, He come out west and made a lot of changes


He come out west and built a lot of fences, And built 'em right acrost our cattle ranges.

the farmer and the cowman should be friends
The Farmer and the Cowman Should be Friends


The farmer should be sociable with the cowboy if he rides by and asks for food and water.Don't treat him like a louse make him welcome in your house.

Pro-farmerBut be sure that you lock up your wife and daughters!

societal costs to poor medical training
Societal Costs to Poor Medical Training

While systems designed to cull the best from the rest might be desirable in situations like baseball’s World Series, this ought not to be our design in the training of medical students. We want each medical school graduate to have mastered all the material considered important. At present, those doctors in need of remedial assistance are often not identified until some kind of ex-post screening process is undertaken, such as in-service examinations, specialty board certifications, or malpractice lawsuits.

Innovator’s Prescription, p. 352-3

medical schools in la la land
Medical Schools in La-La Land

What roles are today’s leading medical schools playing in training the caregivers we’ll need? In fact, they’re training more and more of the doctors we won’t need, and leaving others to train the professionals we will need. The medical education system in the United States has no means of coordinated planning for training doctors for societal needs.

Innovator’s Prescription, p. 354

improper reward structure in medical education
Improper Reward Structure in Medical Education

The only resource allocation mechanism is that medical students, like the rest of us, choose careers that are intellectually and emotionally engaging, with the most attractive incomes and lifestyles. … This has led to a rapid expansion of subspecialists and a dearth of primary care physicians. …

As for the shortage of primary care physicians, America is turning to immigrants from foreign nursing schools

Innovator’s Prescription, p. 354-5, 357

end result disruption of medical schools
We anticipate that, given the need for more primary care doctors and nurses that our medical and nursing schools are not meeting, major integrated health-care providers will begin training their own caregivers.

Innovator’s Prescription, p. 360

End Result: Disruption of Medical Schools
How Many Discoveries Have Been Lost by Ignoring Modern Statistical Methods?Rand R. Wilcox, American Psychologist, 1998

Arbitrarily small departures from normality result in low power; even when distributions are normal, heteroscedasticity can seriously lower the power of standard ANOVA and regression methods.

… most quantitative articles tend to be too technical for applied researchers.

If the goal is to avoid low power, the worst method is the ANOVA F test.

british medical journal articles by doug altman
British Medical Journal articles by Doug Altman

The scandal of poor medical research, 1994

Why are errors so common? Put simply, much poor research arise because researchers feel compelled for career reasons to carry out research that they are ill equipped to perform, and nobody stops them.

Statistics and ethics in medical research. The misuse of statistics is unethical, 1980

ethical guidelines discussion jh ellenberg tas 1983
Ethical Guidelines Discussion, (JH Ellenberg, TAS,1983)

RD Remington: “The discipline of statistics, which emphasizes the development of new methodology and new theoretical structures, is perhaps comparable to its parent discipline, mathematics …”

“The practice of statistics on the other hand, is a profession, involving at every stage judgment and informed choice.”

“The guidelines themselves give only slight notice to a central feature of professional practice of any type – professional judgment.”

ethical guidelines discussion 2
Ethical Guidelines Discussion (2)

HV Roberts: “The Guidelines will help to remind statisticians … that statistical practice requires integrity as well as professional skill. But [the Guidelines] sound bland.”

Temptations are pervasive, yet subtle:

1. to modify one’s best evaluation of the data by what the audience or client wants to hear.

3. to reject needed tools on the grounds that they will prove too difficult to explain.

4. to be lax in seeking out the most appropriate statistical tools.

7. to neglect checks and safeguards against data problems, model failure, and processing errors.

marketing of pharmaceuticals
Marketing of Pharmaceuticals
  • Must have the produced the drug and shown its efficacy
  • Need to produce the drug in mass quantities
  • Marketing
marketing of statistical ideas
Marketing of Statistical Ideas
  • Must have derived the statistic and demonstrated its efficacy
  • Need to have available software
  • Need to disseminate the idea
key principle
Key Principle

In an environment where ideas are not marketed, first on the market wins

first on the market winners
First-on-the-market winners

Chi-square test

T-test, 1908


Kolmogorov-Smirnov test, 1937

Duncan’s test, 1950

Kaplan-Meier curves, 1958

Cox regression, 1972

three examples where education and practice lags woefully behind
Three examples where education and practice lags woefully behind
  • Two-sample location problem
  • 2 x 2 contingency table
  • Multiple testing adjustment
multiple testing adjustment
Multiple Testing Adjustment

Traditional answer: Bonferroni

Improved answer: Holm

holm adjustment
Holm adjustment

J or Scandinavian Statistics, 1979

3,432 total citations (3/16/09)

288 in 2007

386 in 2008

Personal thanks: Gary Koch and Alex Dmietrienko

two sample location problem
Two-sample location problem

Traditional answer: Wilcoxon rank sum test for small samples, t-test for large samples

Better practice: Wilcoxon or normal scores test for large samples, t-test for small samples (such as n=3, but assumptions are critical!)

hodges and lehmann 1961 4 th berkeley symposium
Hodges and Lehmann , 19614th Berkeley Symposium

Hodges and Lehmann (1956) proved that the ARE of the Wilcoxon rank sum test is at least .864

Chernoff and Savage (1958) proved that the ARE of the normal scores test is at least 1

“The above results suggest that on the basis of power, at least for large samples, both the Wilcoxon and normal scores tests are preferable to the t-test for general use.”

first simulation on robustness of t test ca boneau 1960
First Simulation on Robustness of t-testCA Boneau, 1960

320 citations

Assessed validity only, using three distributions, normal, uniform, exponential and three sample size pairs

Conclusion: t-test is fine.

Later discovery: exponential simulation was done wrong

Highest citation count on any subsequent simulation study (39 articles thru 2000) = 96

Misconceptions Leading to Choosing the t Test Over the Wilcoxon-Mann Whitney Test for Shift in Location Parameter

J of Modern Applied Statistical Methods, 2005

by SS Sawilowsky

“The knowledge about the large sample asymptotic theory “had even penetrated to the level of a book review written in 1968!”

“The Wilcoxon rank-sum test … show[s] only slight losses in both large and small sample efficiency relative to the t-test in the normal case, while in many non-normal cases, efficiency exceeds 100%”.

who was the book review author
Who was the book review author?

Duane Meeter!

But who was listening?

on student s 1908 article jasa march 2008
On Student’s 1908 ArticleJASA March, 2008

Comment by Diaconis and Lehmann

… even under slight deviations from normality, the t-test can be far from optimal. The poor performance of the t-test, particularly for distributions with heavy tails, can be seen in comparison with nonparametric tests, such as the Wilcoxon or normal scores tests.

… for all distributions with finite variance, the asymptotic relative efficiency relative to t is ≥ .864 for Wilcoxon and ≥ 1 for normal scores.

textbook placement
Textbook Placement

Basic Practice of Statistics, 4th Ed. 2006 David S. Moore (728 pages)

Non-parametric tests don’t make the book; they appear in the virtual appendix.

Statistics: A Biomedical Introduction, 1977

Hollander and Wolfe

T-test in Chapter 5; Wilcoxon in Chapter 13

Biostatistics, 2nd Ed. van Belle, Fisher, et al., 2004

T-test in Chapter 5; Wilcoxon in Chapter 8

2x2 contingency table
2x2 contingency table

Traditional answer: chi-square test unless expected cell count < 5, then Fisher exact test

Better answer: It depends on the design (what marginals are fixed), For most practical situations: unconditional test

problems with chi square
Problems with chi-square

It cheats (is invalid) sometimes and we now can tell where by exact or Monte Carlo results.

The rule of thumb reduces the violations but doesn’t eliminate them.

D’Agostino, Chase, and Belanger recommend that one can use the chi-square test all the time! (TAS, 1988)

enter barnard s unconditional test
Enter Barnard’s unconditional test

1945,7 Barnard introduces test

1949 Barnard renounces own test

1979 Kempthorne: “The importance of the topic cannot be stressed too heavily .. 2x2 contigency tables are the most elemental structures leading to ideas of association.

… It is remarkable that a consensus has not been reached”.

1985 Suissa and Shuster re-introduce test

1985? Mehta (at JSM conference) “I’m not sure I believe in it … but if people want it (chuckles), I might include it in StatXact.”

recent articles
Recent Articles

Campbell (Stat Med, 2007) recommends the Mantel-Haenszel chi-square when expected cell size are 1 or more

Lydersen, Fagerland, and Laake (Stat Med, 2009) recommend the unconditional test unless both margins are fixed

Proschan, and Nason (Bmcs, 2009) “One lesson is that when the design dictates the margins … one should condition on them.”

ethical considerations
Ethical Considerations

Ethical Guidelines for Statistical Practice, ’99

“The use of statistics in medical and biomedical research may affect whether individuals live or die”

“… society depends on sound statistical practice … many unresolved issues that deserve frank discussion.”

Educators have an ethical responsibility to properly train their “tool user” students in best practices

“Tool user” statisticians have an ethical responsibility to seek best practice information

insufficient focus on teaching goals
Insufficient Focus on Teaching Goals

… students would rotate from one clerkship to the next in an identical sequence – rather than crisscrossing back and forth as they presently do, and where the quality of students’ educational experiences depends to a frustrating degree on “the luck of the draw”.

Innovator’s Prescription, p. 348

why medical schools won t change
Why Medical Schools Won’t Change

Although there are clear needs for sustaining improvements to medical education, … we are pessimistic that our leading medical schools will be able to act decisively on either front. … The reason lies in the mechanisms of governance in these institutions – which are largely collegial and consensus-driven.

Innovator’s Prescription, p. 358

oklahoma reprise
Oklahoma Reprise

Territory folks should stick together, Territory folks should all be pals. Cowboys dance with farmer's daughters, Farmers dance with the ranchers' gals.