Interpreting numbers ScotPHO training course March 2011 Dr Gerry McCartney

1. Interpreting numbers ScotPHO training course March 2011 Dr Gerry McCartney Head of Public Health Observatory Division NHS Health Scotland gmccartney@nhs.net

2. Approaching numbers: some questions to ask 68% of Doctors don’t listen to their patients • What is being counted? • Definitions • Type of numbers - counts, means, etc. • Who/where (population): what is the denominator? • When (time): what time period do they cover? • How (source): where did they come from? • Why were they produced: is there an agenda? SMOKERS ON PILL DOUBLE STROKE RISK

3. Incidence and Prevalence • Dealt with in more detail later in course • Incidence describes the number of new cases in the population over a period of time • Prevalence describes the number of cases present in a population at any one point in time

4. Framework for interpreting numbers Could your interpretation be affected by either: • Error • Chance • Confounding (the mixing of two effects) • Bias (systematic departure from truth – either deliberate or unintentional)

5. Example: COPD (lung disease) variation • Is COPD more common in Board A or Board B? • Errors (e.g. different definitions used in each Board)? • Chance (e.g. no confidence intervals used)? • Bias (e.g. are there systematic differences in how disease is recorded)? • Confounding (e.g. are there mixed effects – such as age structure)?

6. Sources of error: • Mistakes in data collection, data recording, data storage, data transmission • Coding errors, transcription errors • Can be random or systematic • Do the numbers add up? • Are the number plausible?

8. Chance and interpreting numbers • Most figures report data for a sample from a larger population • A different sample would give a different result • Year to year fluctuation can be due to chance • The size of the sample dictates the degree to which a difference is likely to be due to chance • Confidence intervals and p-values give estimates of the precision of a value • E.g. Relative risk of heart disease amongst diabetics is 7.4 (95% CI 6.5-8.6) means that there is a less than 1 in 20 chance of the true value lying outwith the range 6.5 to 8.6

9. Bias – identification and interpretation • Bias is a systematic alteration of figures away from the true value Examples • Selection bias – critically appraise sampling strategy, loss to follow-up, response rate • Information bias – completeness of data, calibration, participant self-report, recall time • Publication bias – think about a funnel plot

10. Confounding: when separate effects are mixed together • In this example, the effect of location is mixed with (confounded by) the effect of age • The population of Western Isles is older so has higher rates of CHD admission • Is CHD more common in Western Isles after taking age into account?

11. Methods for dealing with confounding Design • Randomisation (only for experimental studies) • Restriction (e.g. narrow the comparison groups by age, sex, ethnicity, socioeconomic status) • Matching Analysis • Stratification (i.e. compare sub-groups, but has dangers) • Standardisation* • Multivariate analysis* *dealt with in more detail elsewhere

12. Standardisation: brief interpretation • A method of “removing” the effect of other factors to allow a “fair” comparison • The other factors are most commonly age and sex, but standardisation can be used for other factors • Standardisation shows the rates you would get if the population had a “standard” age and sex structure

13. Standardised Mortality Ratios (SMRs): • This is a comparison of mortality in a population with a ‘standard’ population taking account of age structure • The standard population is allocated a value of 100 for whatever the mortality rate is • The age and sex standardised mortality of the population of interest is then divided by that in the standard population to give a figure for comparison with the 100 • An SMR of 150 indicates that mortality is 50% higher after accounting for age and sex differences

14. Interpreting associations: does A cause B? Causal relationship A B A B A C Confounding B Chance A ? B

15. Some quick notes on interpreting graphs: • Beware of: ambiguity, distortion and distraction

16. Data Ambiguity

17. Data distortion (1) 10.1 10.2 10.3 10.4 0 1 2 3 4 5 6 7

18. Data Distortion (2)

19. Data Distortion (3)

20. Data distortion (4) General acute inpatient discharges with an alcohol-related diagnosis in any position, by gender, Scotland, 1982/3 - 2009/10

21. Data distortion (5) General acute inpatient discharges with an alcohol-related diagnosis in any position, by gender, Scotland, 1982/3 - 2009/10

22. Summary • Always ask the questions: what, who, where, when, how and why • Think about possible problems with data: errors, chance, bias and confounding • Even when things are associated they may not be cause and effect • Beware of the possibility of graphs creating distortions, distractions or ambiguity

23. Questions gmccartney@nhs.net