Statistics for science journalists
Download
1 / 72

Statistics for Science Journalists - PowerPoint PPT Presentation


  • 123 Views
  • Uploaded on

Statistics for Science Journalists. Steve Doig Cronkite School of Journalism. Journalists hate math. Definition of journalist: A do-gooder who hates math. “ Word person, not a numbers person. ” 1936 JQ article noting habitual numerical errors in newspapers

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Statistics for Science Journalists' - fausta


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Statistics for science journalists

Statistics for Science Journalists

Steve Doig

Cronkite School of Journalism


Journalists hate math
Journalists hate math

  • Definition of journalist: A do-gooder who hates math.

  • “Word person, not a numbers person.”

  • 1936 JQ article noting habitual numerical errors in newspapers

  • Japanese 6th graders more accurate on math test than applicants to Columbia’s Graduate School of Journalism

  • 20% of journalists got more than half wrong on 25-question “math competency test” (Maier)

  • 18% of 5,100 stories examined by Phil Meyer had math errors


Bad examples abound
Bad examples abound

  • Paulos: 300% decrease in murders

  • Detroit Free Press (2006): Compared ACS to Census data to get false drop in median income

  • KC Star (2000): Priests dying of AIDS at 4 times the rate of all Americans

  • Delaware ZIP Code of infant death

  • NYT: 51% of women without spouses


Common problems
Common problems

  • Numbers that don’t add up

  • Making the reader do the math

  • Failure to ask “Does this make sense?”

  • Over-precision

  • Ignoring sampling error margins

  • Implying that correlation equals causation


Dangers of journalistic innumeracy
Dangers of journalistic innumeracy

  • Misleads math-challenged readers/viewers

  • Hurts credibility among math-capable readers/viewers

  • Leads to charges of bias, even when cause is ignorance

  • Makes reporters vulnerable to being used for the agendas of others


Common research methods
Common Research Methods

  • Randomized experiments: Measure deliberate manipulation of the environment

  • Observational studies: Measure the differences that occur naturally

  • Meta-analyses: Quantitative review of multiple studies

  • Case Study: Descriptive in-depth examination of one or a few individuals


Simple measures
Simple Measures...

...don’t exist!


Measurement variability
Measurement Variability

  • Variable measurements include unpredictable errors or discrepancies that aren’t easily explained.

  • Natural variabilityis the result of the fact that individuals and other things are different.


Reasons for variable measures
Reasons for variable measures

  • Measurement error

  • Natural variability between individuals

  • Natural variability over time in a single individual

    Statistics are tools to help us work with measurements that vary



Deliberate bias
Deliberate Bias?

If you found a wallet with $20, would you:

  • “Keep it?”

    (23% would keep it)

  • “Do the honest thing and return it?”

    (13% would keep it)


Unintentional bias
Unintentional Bias?

  • “Do you use drugs?”

  • “Are you religious?”


Desire to please
Desire to Please?

People routinely say they have voted when they actually haven’t, that they don’t smoke when they do, and that they aren’t prejudiced.

One study six months after an election:

  • 96% of actual voters said they voted.

  • 40% of non-voters said they voted.


Asking the uninformed
Asking the uninformed?

Washington Post poll : “Some people say the 1975 Public Affairs Act should be repealed. Do you agree or disagree that it should be repealed?”

  • 24% said yes

  • 19% said no

  • rest had no opinion


Asking the uninformed1
Asking the uninformed?

Later Washington Post poll: “President Clinton says the 1975 Public Affairs Act should be repealed. Do you agree or disagree that it should be repealed?”

  • 36% of Democrats agreed

  • 16% of Republicans agreed

  • rest had no opinion


Unnecessary complexity
Unnecessary Complexity?

  • “Do you support our soldiers in Iraq so that terrorists won’t strike the U.S. again?”


Question order
Question Order

  • “About how many times a month do you normally go out on a date?”

  • “How happy are you with life in general?”



Margin of error
Margin of Error

95% of the time, a random sample’s characteristics will differ from the population’s by no more than about

where N= sample size


Two important concepts about error margin
Two Important Concepts about Error Margin

  • The larger the sample, the smaller the margin of sampling error.

  • The size of the population being surveyed doesn’t matter.*

    *Unless the sample is a significant fraction of the population.


Sampling realities
Sampling realities

  • Bigger sample means more cost (money and/or time)

  • Diminishing return on error margin improvement as sample increases.

    • N=100: +/- 10 percentage points

    • N=400: +/- 5 percentage points

    • N=900: +/- 3.3 percentage points

  • Sample needs only to be large enough to give a reasonable answer.

  • Sampling error affects subsamples, too.



Three useful features of a set of data
Three Useful Featuresof a Set of Data

  • The Center

  • The Variability

  • The Shape


The center
The Center

  • Mean (average): Total of the values, divided by the number of values

  • Median: The middle value of an ordered list of values

  • Mode: The most common value

  • Outliers: Atypical values far from the center


Example baseball salaries
Example: Baseball Salaries

  • Average: $2,827,104

  • Median: $950,000

  • Mode: $327,000 (also the minimum)

  • Outlier: $21.7 million (Alex Rodriguez of the NY Yankees)


The variability
The Variability

Some measures of variability:

  • Maximum and minimum: Largest and smallest values

  • Range: The distance between the largest and smallest values

  • Quartiles: The medians of each half of the ordered list of values

  • Standard deviation: Think of it as the average distance of all the values from the mean.


What is normal
What is “normal”?

  • Don’t consider the average to be “normal”

  • Variability is normal

  • Anything within about 3 standard deviations of the mean is “normal”



Some characteristics of a normal distribution

mean

Some Characteristics of a Normal Distribution

  • Symmetrical (not skewed)

  • One peak in the middle, at the mean

  • The wider the curve, the greater the standard deviation

  • Area under the curve is 1 (or 100%)


Percentiles
Percentiles

Your percentilefor a particular measure (like height or IQ) is the percentage of the population that falls belowyou.

Compared to other American males:

  • My height (5’ 11”): 75th percentile

  • My weight (230 lbs.): 85thpercentile

  • My age (64): 87th percentile

    Therefore, I am older and heavier than I am tall.


Standardized scores
Standardized Scores

A standardized score(also called the z-score) is simply the number of standard deviations a particular value is either above or below the mean.

The standardized score is:

  • Positive if above the mean

  • Negative if below the mean

    Useful for defining data points as outliers.


The empirical rule
The Empirical Rule

For any normal curve, approximately:

  • 68% of values within one StdDevof the mean

  • 95% of values within two StdDevsof the mean

  • 99.7% of values within three StdDevsof the mean


Outlier
Outlier

  • A value that is more than three standard deviations above or below the mean.



Strength of relationship
Strength of Relationship

Correlation (also called the correlation coefficient or Pearson’s r) is the measure of strength of the linear relationship between two variables.

Think of strength as how closely the data points come to falling on a line drawn through the data.


Features of correlation
Features of Correlation

  • Correlation can range from +1 to -1

  • Positive correlation: As one variable increases, the other increases

  • Negative correlation: As one variable increases, the other decreases

  • Zero correlation means the best line through the data is horizontal

  • Correlation isn’t affected by the units of measurement


Positive correlations

r = +.4

r = +1

r = +.8

Positive Correlations

r = +.1


Negative correlations
Negative Correlations

r = -.4

r = -.1

r = -.8

r = -1


Zero correlation
Zero correlation

r = 0

r = 0


Number of points doesn t matter
Number of PointsDoesn’t Matter

r = .8

r = .8


Important
Important!

Correlation does not imply causation.

(Churches and liquor stores, shoe size and reading ability)


Correlation of variables
Correlation of variables

  • When considering relationships between measurement variables, there are two kinds:

    • Explanatory (or independent) variable: The variable that attempts to explain or is purported to cause (at least partially) differences in the…

    • Response (or dependent or outcome) variable

  • Often, chronology is a guide to distinguishing them (examples: baldness and heart attacks, poverty and test scores)


Some reasons why two variables could be related
Some reasons why two variables could be related

  • The explanatory variable is the direct cause of the response variable

    Example: pollen counts and percent of population suffering allergies, intercourse and babies


Some reasons two variables could be related
Some reasons two variables could be related

  • The response variable is causing a change in the explanatory variable

    Example: hotel occupancy and advertising spending, divorce and alcohol abuse


Some reasons two variables could be related1
Some reasons two variables could be related

  • The explanatory variable is a contributing -- but not sole -- cause

    Example: birth complications and violence, gun in home and homicide, hours studied and grade, diet and cancer


Some reasons two variables could be related2
Some reasons two variables could be related

  • Both variables may result from a common cause

    Example: SAT score and GPA, hot chocolate and tissues, storks and babies, fire losses and firefighters, WWII fighter opposition and bombing accuracy


Some reasons two variables could be related3
Some reasons two variables could be related

  • Both variables are changing over time

    Example: divorces and drug offenses, divorces and suicides


Some reasons two variables could be related4
Some reasons two variables could be related

  • The association may be nothing more than coincidence

    Example: clusters of disease, brain cancer from cell phones


So how can we confirm causation
So how can we confirm causation?

The only way to confirm is with a designed (randomized double-blind) experiment.But non-statistical evidence of a possible connection may include:

  • A reasonable explanation of cause and effect.

  • A connection that happens under varying conditions.

  • Potential confounding variables ruled out.



Linear regression
Linear Regression

In addition to figuring the strength of the relationship, we can create a simple equation that describes the best-fit line (also called the “least-squares” line) through the data.

This equation will help us predict one variable, given the other.





Remember 9th grade algebra
Remember 9th Grade Algebra?

x = horizontal axis y = vertical axis

Equation for a line:

y = slope * x + intercept

or as it often is stated:

y =mx + b


Regression in data journalism
Regression in data journalism

  • Public school test scores

  • Cheating in school test scores

  • Tenure of white vs. black coaches in NBA

  • Racial bias in picking jurors

  • Racial profiling in traffic stops



Confusion of the inverse1
Confusion of the Inverse

Confusing these two:

  • Probability of actually having a condition, given a positive test for it

  • Probability of having a positive test, given actually having the condition

    When the incidence of some disease or condition is very low, and the test for it is not perfect, there will be a high probability that a positive test result is false positive.


Definitions
Definitions

  • Base rate: The probability that someone has a disease or condition, without knowing any test results.

  • Test Sensitivity: Proportion of people who correctly test positive when they have the disease or condition (true positive)

  • Test Specificity: Proportion of people who correctly test negativewhen they don’t have the disease or condition (true negative)


Drug tests
Drug Tests

Consider this scenario:

  • Base rate: 1% of population to be tested uses dangerous drugs

  • You use a test that’s 99% accurate in both sensitivity and specificity

  • 10,000 people are tested


Drug tests1
Drug Tests

Test Test Total

PositiveNegative

Users 100

Not 9,900

Total 10,000


Drug tests2
Drug Tests

Test Test Total

PositiveNegative

Users 99 1 100

Not 9,900

Total 10,000


Drug tests3
Drug Tests

Test Test Total

PositiveNegative

Users 99 1 100

Not 9,801 9,900

Total 9,802 10,000


Drug tests4
Drug Tests

Test Test Total

PositiveNegative

Users 99 1 100

Not ??? 9,801 9,900

Total 9,802 10,000


Drug tests5
Drug Tests

Test Test Total

PositiveNegative

Users 99 1 100

Not 9,801 9,900

Total 198 9,802 10,000

(50% of positives are FALSE!)

99



Confidence intervals
Confidence Intervals

  • Like the error margin around poll results

  • A confidence interval is a tradeoff between certainty and accuracy, like shooting at targets of different sizes

  • The bigger the sample, the smaller the confidence interval at the 95% level

  • When comparing results, if confidence intervals overlap, the results are NOT statistically significant


P values
P-values

  • P-value is the probability that the sample result is significantly different from the true result (i.e., wrong)

  • 95% confidence interval (p < 0.05) is the most commonly used interval in social science research

  • Hard science, particularly medicine, often needs tighter confidence intervals and smaller p-values, like p<0.01

  • Studies are going to be wrong about 5% of the time (and you won’t know when)

  • On the other hand, they probably won’t be very wrong.


How to read a research study
How to read a research study

  • Pay attention to the method: Observational, randomized double-blind experiment, meta-analysis, case study

  • Note the sample size

  • Don’t ignore the confidence intervals

  • Consider the p-value as the probability you’re writing about something that isn’t true

  • Remember correlation doesn’t necessarily mean causation.

  • Consider the quality of the journal (peer reviewed?)

  • Who paid for the research?


Newsroom math bibliography
Newsroom math bibliography

  • “Numbers in the Newsroom”, by Sarah Cohen, IRE

  • “News and Numbers”, by Victor Cohn and Lewis Cope

  • “Precision Journalism (4th edition)”, by Phil Meyer

  • “Innumeracy”, by John Allen Paulos

  • “A Mathematician Reads the Newspaper,” by John Allen Paulos



ad