Bandit Thinkhamrop, PhD.(Statistics) Dept. of Biostatistics & Demography Faculty of Public Health

1. Biostatistics for Critical Appraisal Bandit Thinkhamrop, PhD.(Statistics) Dept. of Biostatistics & Demography Faculty of Public Health Khon Kaen University

2. Core Questions • What is the research question? • What is the answer or the conclusion? • What is the magnitude of effect being used as the basis of the conclusion? • How large is the effect – conclusive or not? • How likely is the validity of the magnitude of effect - accurate, over or under estimate, questionable, unassessable? • Can the conclusion be acceptable? Why? • How this paper could be improved?

3. 1. What is the research question? • State the research question • Focus on the primary research question or the one that lead to the main conclusion • Example: • What are predictors of low birth weight? • What is the efficacy of the HIV vaccine?

4. 2. What is the answer or the conclusion? • Extract the answer to the primary research question • Usually be obtained from the main conclusion of the study • Example: • Low BMI of mother and not received ANC increase risk of low birth weight • HIV vaccine can prevent HIV infection

5. 3. What is the magnitude of effect being used as the basis of the conclusion? • If reported, transcribe it. If not, calculate it based on the reported results. • Its the 95%CI is essential • Summarize the magnitude of effect as a forest plot-like diagram against the meaningful level is recommended • Example: • OR (95%CI) of each factor on low birth weight : Low BMI of mother (OR: 3.2; 2.50 to 4.50), and not received ANC increase risk of low birth weight (OR: 1.6; 1.02 to 2.18) • HIV vaccine efficacy is 30% (95%CI: 1.1% to 52.2%)

6. Forest plot for the Difference Minimum meaningful level Male (0.9; 0.1 to 1.7) Female (1.4; -0.1 to 2.9) Overall (1.8; 0.6 to 3.0) Flavor Intervention A Flavor Intervention B   -1 1 -3 -2 0 2 3

7. Forest plot for RR, IRR, OR, HR Minimum meaningful level Low BMI of mother (3.20; 2.50 to 4.50) Received ANC (1.60; 1.02 to 2.18) Protective effect Risk effect  0 0.50 2 0.25 0.33 1 3 4

8. Notice the wrong scale

10. 4. How large is the effect – conclusive or not? • Compare the magnitude of both lower and upper boundary of the 95%CI with the meaningful level of effect • To be conclusive, the CI needs NOT include the meaningful cut point • Conclusive findings imply sufficient sample size

11. 5. How likely is the validity of the magnitude of effect-accurate, over or under estimate, questionable, unassessable? • Describe briefly how the magnitude of effect was obtained • Describe how it could be wrong by considering these three issues: • Wrong data? • Selection bias • Information bias • Confounding bias • Wrong analysis? • Applied inappropriate statistical method • Poor or unclear reporting • Important information was not provided

12. 6. Can the conclusion be acceptable? • Answer : Yes/No and Why? • Explain the mechanisms as to how the magnitude of effect could be wrong, if not accepted • Elaborate more by describing what position of this evidence in your evidence-base practice

13. 7. How this paper could be improved? • No research is perfect – there are rooms to improve. • Go beyond what had been reported in the paper to what should have been done to be better under unlimited resources