Engaging in statistical practice in academia is honorable
Download
1 / 36

Engaging in Statistical Practice in Academia is Honorable - PowerPoint PPT Presentation


  • 266 Views
  • Uploaded on

Engaging in Statistical Practice in Academia is Honorable. Michael J. Schell Moffitt Cancer Center & Research Institute. The Methodologist and the Practitioner. The roles of a statistical methodologist and a statistical practitioner, while overlapping somewhat, are distinct.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Engaging in Statistical Practice in Academia is Honorable' - Anita


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Engaging in statistical practice in academia is honorable l.jpg

Engaging in Statistical Practice in Academia is Honorable

Michael J. Schell

Moffitt Cancer Center &

Research Institute


The methodologist and the practitioner l.jpg
The Methodologist and the Practitioner

The roles of a statistical methodologist and a statistical practitioner, while overlapping somewhat, are distinct.

Academia still needs to learn how to properly define the proper expectations and rewards for the statistical practitioner


Fsu builds a new academic building circa 1978 l.jpg
FSU Builds a New Academic Building Circa 1978

Two of following four departments should move to the new building: Mathematics, Meteorology, Oceanography, Statistics. What split would be best?

Apparently some departments are too academically too close for comfort.


The farmer and the cowman should be friends oklahoma rodgers and hammerstein l.jpg
The Farmer and the Cowman Should be FriendsOklahoma – Rodgers and Hammerstein

Pro-farmer

I'd like to say a word for the farmer, He come out west and made a lot of changes

Pro-cowman

He come out west and built a lot of fences, And built 'em right acrost our cattle ranges.


The farmer and the cowman should be friends l.jpg
The Farmer and the Cowman Should be Friends

Pro-cowman

The farmer should be sociable with the cowboy if he rides by and asks for food and water.Don't treat him like a louse make him welcome in your house.

Pro-farmerBut be sure that you lock up your wife and daughters!


Societal costs to poor medical training l.jpg
Societal Costs to Poor Medical Training

While systems designed to cull the best from the rest might be desirable in situations like baseball’s World Series, this ought not to be our design in the training of medical students. We want each medical school graduate to have mastered all the material considered important. At present, those doctors in need of remedial assistance are often not identified until some kind of ex-post screening process is undertaken, such as in-service examinations, specialty board certifications, or malpractice lawsuits.

Innovator’s Prescription, p. 352-3


Medical schools in la la land l.jpg
Medical Schools in La-La Land

What roles are today’s leading medical schools playing in training the caregivers we’ll need? In fact, they’re training more and more of the doctors we won’t need, and leaving others to train the professionals we will need. The medical education system in the United States has no means of coordinated planning for training doctors for societal needs.

Innovator’s Prescription, p. 354


Improper reward structure in medical education l.jpg
Improper Reward Structure in Medical Education

The only resource allocation mechanism is that medical students, like the rest of us, choose careers that are intellectually and emotionally engaging, with the most attractive incomes and lifestyles. … This has led to a rapid expansion of subspecialists and a dearth of primary care physicians. …

As for the shortage of primary care physicians, America is turning to immigrants from foreign nursing schools

Innovator’s Prescription, p. 354-5, 357


End result disruption of medical schools l.jpg

We anticipate that, given the need for more primary care doctors and nurses that our medical and nursing schools are not meeting, major integrated health-care providers will begin training their own caregivers.

Innovator’s Prescription, p. 360

End Result: Disruption of Medical Schools


Slide10 l.jpg
How Many Discoveries Have Been Lost by Ignoring Modern Statistical Methods?Rand R. Wilcox, American Psychologist, 1998

Arbitrarily small departures from normality result in low power; even when distributions are normal, heteroscedasticity can seriously lower the power of standard ANOVA and regression methods.

… most quantitative articles tend to be too technical for applied researchers.

If the goal is to avoid low power, the worst method is the ANOVA F test.


British medical journal articles by doug altman l.jpg
British Medical Journal articles by Doug Altman Statistical Methods?

The scandal of poor medical research, 1994

Why are errors so common? Put simply, much poor research arise because researchers feel compelled for career reasons to carry out research that they are ill equipped to perform, and nobody stops them.

Statistics and ethics in medical research. The misuse of statistics is unethical, 1980


Ethical guidelines discussion jh ellenberg tas 1983 l.jpg
Ethical Guidelines Discussion, Statistical Methods?(JH Ellenberg, TAS,1983)

RD Remington: “The discipline of statistics, which emphasizes the development of new methodology and new theoretical structures, is perhaps comparable to its parent discipline, mathematics …”

“The practice of statistics on the other hand, is a profession, involving at every stage judgment and informed choice.”

“The guidelines themselves give only slight notice to a central feature of professional practice of any type – professional judgment.”


Ethical guidelines discussion 2 l.jpg
Ethical Guidelines Discussion (2) Statistical Methods?

HV Roberts: “The Guidelines will help to remind statisticians … that statistical practice requires integrity as well as professional skill. But [the Guidelines] sound bland.”

Temptations are pervasive, yet subtle:

1. to modify one’s best evaluation of the data by what the audience or client wants to hear.

3. to reject needed tools on the grounds that they will prove too difficult to explain.

4. to be lax in seeking out the most appropriate statistical tools.

7. to neglect checks and safeguards against data problems, model failure, and processing errors.


Marketing of pharmaceuticals l.jpg
Marketing of Pharmaceuticals Statistical Methods?

  • Must have the produced the drug and shown its efficacy

  • Need to produce the drug in mass quantities

  • Marketing


Marketing of statistical ideas l.jpg
Marketing of Statistical Ideas Statistical Methods?

  • Must have derived the statistic and demonstrated its efficacy

  • Need to have available software

  • Need to disseminate the idea


Key principle l.jpg
Key Principle Statistical Methods?

In an environment where ideas are not marketed, first on the market wins


First on the market winners l.jpg
First-on-the-market winners Statistical Methods?

Chi-square test

T-test, 1908

ANOVA

Kolmogorov-Smirnov test, 1937

Duncan’s test, 1950

Kaplan-Meier curves, 1958

Cox regression, 1972


Three examples where education and practice lags woefully behind l.jpg
Three examples where education and practice lags woefully behind

  • Two-sample location problem

  • 2 x 2 contingency table

  • Multiple testing adjustment


Multiple testing adjustment l.jpg
Multiple Testing Adjustment behind

Traditional answer: Bonferroni

Improved answer: Holm


Holm adjustment l.jpg
Holm adjustment behind

J or Scandinavian Statistics, 1979

3,432 total citations (3/16/09)

288 in 2007

386 in 2008

Personal thanks: Gary Koch and Alex Dmietrienko


Two sample location problem l.jpg
Two-sample location problem behind

Traditional answer: Wilcoxon rank sum test for small samples, t-test for large samples

Better practice: Wilcoxon or normal scores test for large samples, t-test for small samples (such as n=3, but assumptions are critical!)


Hodges and lehmann 1961 4 th berkeley symposium l.jpg
Hodges and Lehmann , 1961 behind4th Berkeley Symposium

Hodges and Lehmann (1956) proved that the ARE of the Wilcoxon rank sum test is at least .864

Chernoff and Savage (1958) proved that the ARE of the normal scores test is at least 1

“The above results suggest that on the basis of power, at least for large samples, both the Wilcoxon and normal scores tests are preferable to the t-test for general use.”


First simulation on robustness of t test ca boneau 1960 l.jpg
First Simulation on Robustness of t-test behindCA Boneau, 1960

320 citations

Assessed validity only, using three distributions, normal, uniform, exponential and three sample size pairs

Conclusion: t-test is fine.

Later discovery: exponential simulation was done wrong

Highest citation count on any subsequent simulation study (39 articles thru 2000) = 96


Slide24 l.jpg
Misconceptions Leading to Choosing the t Test Over the Wilcoxon-Mann Whitney Test for Shift in Location Parameter

J of Modern Applied Statistical Methods, 2005

by SS Sawilowsky

“The knowledge about the large sample asymptotic theory “had even penetrated to the level of a book review written in 1968!”

“The Wilcoxon rank-sum test … show[s] only slight losses in both large and small sample efficiency relative to the t-test in the normal case, while in many non-normal cases, efficiency exceeds 100%”.


Who was the book review author l.jpg
Who was the book review author? Wilcoxon-Mann Whitney Test for

Duane Meeter!

But who was listening?


On student s 1908 article jasa march 2008 l.jpg
On Student’s 1908 Article Wilcoxon-Mann Whitney Test for JASA March, 2008

Comment by Diaconis and Lehmann

… even under slight deviations from normality, the t-test can be far from optimal. The poor performance of the t-test, particularly for distributions with heavy tails, can be seen in comparison with nonparametric tests, such as the Wilcoxon or normal scores tests.

… for all distributions with finite variance, the asymptotic relative efficiency relative to t is ≥ .864 for Wilcoxon and ≥ 1 for normal scores.


Textbook placement l.jpg
Textbook Placement Wilcoxon-Mann Whitney Test for

Basic Practice of Statistics, 4th Ed. 2006 David S. Moore (728 pages)

Non-parametric tests don’t make the book; they appear in the virtual appendix.

Statistics: A Biomedical Introduction, 1977

Hollander and Wolfe

T-test in Chapter 5; Wilcoxon in Chapter 13

Biostatistics, 2nd Ed. van Belle, Fisher, et al., 2004

T-test in Chapter 5; Wilcoxon in Chapter 8


2x2 contingency table l.jpg
2x2 contingency table Wilcoxon-Mann Whitney Test for

Traditional answer: chi-square test unless expected cell count < 5, then Fisher exact test

Better answer: It depends on the design (what marginals are fixed), For most practical situations: unconditional test


Problems with chi square l.jpg
Problems with chi-square Wilcoxon-Mann Whitney Test for

It cheats (is invalid) sometimes and we now can tell where by exact or Monte Carlo results.

The rule of thumb reduces the violations but doesn’t eliminate them.

D’Agostino, Chase, and Belanger recommend that one can use the chi-square test all the time! (TAS, 1988)


Enter barnard s unconditional test l.jpg
Enter Barnard’s unconditional test Wilcoxon-Mann Whitney Test for

1945,7 Barnard introduces test

1949 Barnard renounces own test

1979 Kempthorne: “The importance of the topic cannot be stressed too heavily .. 2x2 contigency tables are the most elemental structures leading to ideas of association.

… It is remarkable that a consensus has not been reached”.

1985 Suissa and Shuster re-introduce test

1985? Mehta (at JSM conference) “I’m not sure I believe in it … but if people want it (chuckles), I might include it in StatXact.”


Recent articles l.jpg
Recent Articles Wilcoxon-Mann Whitney Test for

Campbell (Stat Med, 2007) recommends the Mantel-Haenszel chi-square when expected cell size are 1 or more

Lydersen, Fagerland, and Laake (Stat Med, 2009) recommend the unconditional test unless both margins are fixed

Proschan, and Nason (Bmcs, 2009) “One lesson is that when the design dictates the margins … one should condition on them.”


Ethical considerations l.jpg
Ethical Considerations Wilcoxon-Mann Whitney Test for

Ethical Guidelines for Statistical Practice, ’99

“The use of statistics in medical and biomedical research may affect whether individuals live or die”

“… society depends on sound statistical practice … many unresolved issues that deserve frank discussion.”

Educators have an ethical responsibility to properly train their “tool user” students in best practices

“Tool user” statisticians have an ethical responsibility to seek best practice information


Insufficient focus on teaching goals l.jpg
Insufficient Focus on Wilcoxon-Mann Whitney Test for Teaching Goals

… students would rotate from one clerkship to the next in an identical sequence – rather than crisscrossing back and forth as they presently do, and where the quality of students’ educational experiences depends to a frustrating degree on “the luck of the draw”.

Innovator’s Prescription, p. 348


Why medical schools won t change l.jpg
Why Medical Schools Won’t Change Wilcoxon-Mann Whitney Test for

Although there are clear needs for sustaining improvements to medical education, … we are pessimistic that our leading medical schools will be able to act decisively on either front. … The reason lies in the mechanisms of governance in these institutions – which are largely collegial and consensus-driven.

Innovator’s Prescription, p. 358


Oklahoma reprise l.jpg
Oklahoma Reprise Wilcoxon-Mann Whitney Test for

Territory folks should stick together, Territory folks should all be pals. Cowboys dance with farmer's daughters, Farmers dance with the ranchers' gals.