Medical Statistics Team, University of Aberdeen

1. Medical Statistics Team, University of Aberdeen Neil Scott Statistical Consulting

2. Overview • Experience of operating a statistical consulting service (Medical Statistics Team, University of Aberdeen) • Review of relevant literature: • Not systematic • Considers any type of statistical consultancy (not just medical statistics) • General discussion – your experiences with statistical consulting

3. Carter et al. (1986) • Unique role of statistics: both a science and a part of the methodology for all scientific subjects • Statistical thought is needed at every stage of quantitative research investigations • A statistics department may assume academic responsibility beyond home department

4. Types of statistical consulting • Ad hoc advice • Drop-in clinic • Appointment system • Formal agreement to produce data analysis/ report • Overlap with research/teaching roles

5. Who consults? • Students • University staff • NHS staff • External

6. Statistical Consultancy Service - Medical Statistics Team • Students (taught course) • Daily student clinic 1-2pm (nine statisticians contribute to rota) • May need to queue • PhD students/MSc by research • Appointment system (supervisor must attend) • University of Aberdeen staff • Appointment system (first half hour is free, then charged) • NHS Grampian staff • Appointment system (free advice/analysis unless large amount of work) • Project collaboration (sometimes begins as consultancy)

7. Logged consultations (2012)

8. Student consultations by month (2012)

9. When do clients consult a statistician? • Stages of the research process: • Planning • Study design • Sample size calculation • Grant application/ethics form • Data collection • Data management/ setting up a database • Data analysis • Most common time to seek help • Dissemination • Publication process • Responding to reviewer comments

10. "To consult a statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of." R. A. Fisher, in an address to the First Indian Statistical Congress in 1938

11. Organisational issues (personal experience) • Drop-in student clinic has peaks and troughs of demand • Some clients want immediate appointments/ specific times • Many people are “hybrids” (University staff/ NHS staff/ student) • Occasional demand for remote consultation for staff/students based off site: • face-to-face (or phone) preferred to e-mail

12. Common situations with face-to-face consultations • Short time scale: “I need to present this next Tuesday” • Understanding unfamiliar topic area/ clinical jargon overwhelming • Overcomplicated studies (small sample size, too many variables), e.g. waste five minutes searching to find correct variable in SPSS • Repeat clients (regular customers) • Timing of advice (e.g., if they had come at design stage, the analysis stage would have been easier) • Client wants to copy a method used in another published paper: i.e. “me too” publication

13. Common situations (when performing data analysis for client) • Messy data: e.g. Excel spreadsheet with blank spaces / mixture of text and numbers • Hand (2010): • “90% of data analysis is data cleaning” • Analysing the data is a cyclic process: interaction with the client • Timescale to complete work or get back to someone – how long is reasonable? • Unclear/vague projects: • Client has only vague idea of research aims - these may evolve over time • Unclear about definitions of variables/ exactly which variables to analyse • Multiple requests for analysis that add up: “Prof Bloggs wants me to add another three p-values / look at another subgroup” • Clients sending data with identifiable information

14. Clients outside applied health research • Laboratory research: • Designed experiments • Repeated measurements on small number of samples • Psychology: • Everything expressed as effect sizes!

15. Generalist versus specialist • Consultant statistician analogous to being a GP • Questions can be on any area (from trivial to extremely complex/specialised) • May have to emphasise you are not expert on every aspect of statistics: • OK to: • Say you don’t know the answer • Get back to them • Refer to colleague with appropriate knowledge

16. Financial terms • Discussing charging can be awkward • Difficult to judge total time on project • Survey of academic consulting units [Niland (1995)] • Most aim to recover costs • Some degree of conflict between service and research roles of academic statistician: • “To maintain a collegial relationship with investigators, it was suggested that only time spent in more routine tasks such as data entry and programming be charged back to investigators, and more conceptual activities such as proposal design, grant, and manuscript preparation be provided without charge.”

17. Statistical Consulting Cycle [Kennett & Thyregod (2006)] • Problem elicitation • Data collection • Data analysis • Findings formulation • Findings presentation Statistics courses in academia tend to focus on step 3

18. Problem elicitation • Active listening • “Determine what the client wants to know: may not be what they say at first (they are not statisticians)” [Hand (2010)] • First question often is: “How exactly would you describe your research question?” • Common question after client gives too much information to take in at once: “Could you go back a step?”

19. Kennett & Thyregod (2006) • “Hunter (1979)…described that two chemists entering his office, when asked to describe the problem they came to get advice on, entered a lengthy discussion that eventually led to a common reformulation of the problem. • Once they had the problem clearly formulated they knew how to solve it and did not need the help of a statistical consultant any more.”

20. Consultancy skills (American Statistical Association, 2003) • Knowledge of statistics • Problem solving • Communication skills • Flexible/ can meet deadlines • Patience • Experience of topic area • Understands resource constraints • Shows respect for statistical and research ethics

21. Hand (2010) • “Statistical consulting requires skills and expertise completely distinct from academic and technical expertise” • “What you need to be a successful consultant: • Affability • Availability • Ability • in decreasing order of importance!” • “Client relationship is the most important issue” • “Put yourself in the client’s shoes”

22. Johnson and Warner (2004) • Studied factors associated with consultation being success • Used factor analysis/ logistic regression [!] • Most important factors: • Statistical/collaboration skills • Personal qualities • Client requested particular consultant

23. Clinician – statistician communication 甲状腺刺激ホルモン受容体遺伝子の生殖細胞突然変異を 引き起こす自己免疫性常染色体優性甲状腺機能亢進症 Bạn đã cố gắng một mô hình hồi quy hoặc một bài kiểm tra chi bình phương và bạn đã kiểm tra rằng các giả định được đáp ứng?

24. Effective communication • Use appropriate language: ask client about knowledge of statistical tests / p-values to ensure communicate at correct level • “Use the language of the customer” [Kennett &Thyregod (2006)] • “Be aware of the client’s level of technical knowledge • They won’t have your statistical expertise • They will know about the problem and the data • They are paying you • They did not come in for a lecture on statistics” [Hand (2010)]

25. Organisational problems of consultancy [Sprent (1970)] • Lack of time or background knowledge to become a full member of an experimental team • The statistician may not have adequately adapted outlook to the computer era • The statistician may be called in too late or dropped too early • There is little or no training in the art of consultancy • In some universities little account is taken of the consultancy load in staffing and successful consultancy is not always a criterion for promotion • In research institutes there is a danger that over-specialization may lead to stagnation, as modern developments may be overlooked • There is little informed criticism of consultancy, and many of us are not keen on practising the art of self-criticism

26. Client stereotypes • Hyams (1971), Sprent (1970) • “Probabilist” – only interested in a significant p-value • “Sporadic leech” – stops the consultant in the hall for a brief friendly chat • “One-technique amateur statistician”

27. Learning from experience • Both client and statistician learn from experience of consultation • Joliffe (2001) • The consulting activity of statisticians can only enhance their teaching • Wilson (1992) • Problems encountered in consultancy settings can lead to new statistical methods • Carter et al. (1986)

28. Benefits of consulting • Exposure to variety of methods • Real world problem solving • Collaboration with others • Rewarding (helping others/ ensuring quality of research)

29. Potential disadvantages • Recognition - consultancy comes under heading of “service”: less prestige than teaching or research [Gibbons and Freund (1980)] • Not initiating own research • Acknowledgement, authorship or nothing at all?: • May do a lot of work without any reward • Concern that do not get to check published material with your name on it! • Checking a paper thoroughly is time-consuming • Additional work may be required following referee comments • May work mainly on small-scale, low-impact research

30. Russell (2011) • “Advertisements for University consultants frequently offer the position at Lecturer level, or even at Research Fellow level, suggesting that the position is seen as a low-level role requiring not too much statistical maturity. Nothing could be further from the truth.”

31. Discussion • Your experiences with statistical consulting: • Demand for consultancy in your institution • Organisation of consultancy services? • Who consults? Staff/students? • Staff time/costs • Etc.

32. References (1) • American Statistical Association (2003). When you consult a statistician … what to expect • Carter RL, Scheaffer RL, Marks RG (1986). American Statistician, 40(4), 260-264. • Gibbons JD, Freund RT (1980). Organizations for statistical consulting at colleges and universities. American Statistician, 34(3), 140-145. • Hand DJ (2010). Statistical consultancy. • Hyams L (1971). The practical psychology of biostatistical consultation. Biometrics, 27(1), 201-211. • Johnson HD, Warner DA (2004). Factors relating to the degree to which statistical consulting clients deem their consulting experiences to be a success. American Statistician, 58(4), 280-289. • Joliffe F (2001). Learning from experience. In: BataneroC (ed.) Training researchers in the use of statistics, 355-370.

33. References (2) • Kennett R, Thyregod P (2006). Aspects of statistical consulting not taught by academia. StatisticaNeedlandica, 60(3),396-411. • Kirk RE (1991). Statistical consulting in a university: dealing with people and other challenges. American Statistician, 45(1), 28-34. • Niland (1995). A survey of biostatistical consulting units throughout North America. American Statistician, 49(2), 183-189. • Russell (2011). Advice for the potential statistical consultant. Proceedings of the Fourth Annual ASEARC Conference, Parmatta, Australia. • Sprent(1970). Some problems of statistical consultancy. JRSS(A), 133(2), 139-165. • Wilson (1992). Statistical consulting is scholarship. American Statistician, 46(4), 295-298.