Dr Rachel Wenke Principal Research Fellow (Allied Health),

1. TREAT (Tailoring Research Evidence And Theory) Journal Clubs in Allied Health: A Randomised Controlled Trial Dr Rachel Wenke Principal Research Fellow (Allied Health), Gold Coast Health, Griffith University, Australia Dr Rae Thomas Centre for Research in Evidence-Based Practice, Bond University, Australia Professor Sharon Mickan Professor of Allied Health, Griffith University, Australia

2. Background • Clinical practice informed by research evidence essential for optimal patient outcomes [1, 2]. BUT in practice….. • Barriers to providing EBP include knowledge & confidence gaps, & lack of time [3-5]. • Journal clubs (JC) are a tool to increase EBP skills & evidence use in clinical practice • Currently minimal research into Allied Health Journal Clubs

3. Key components for effective JC • 2015  Allied HealthEBP Champions prioritised developing Journal clubs • Synthesis of 2 systematic reviews revealed 11 “key components” for effective Journal Clubs. • Only 2 of components consistently incorporated in current Journal Clubs ^= key component suggested in Deenadayalan et al., 2008 *= key component suggested in Harris et al., 2011

4. Aims • To investigate the effectiveness of a structured journal club that is Tailored According to Research Evidence And Theory (TREAT) in improving EBP skills & practice compared to a standard Journal Club format. • To explore the feasibility of implementing TREAT Journal Clubs in regards to clinician perceptions and satisfaction. Design • Explanatory mixed methods • Cluster Randomised controlled trial with nested focus group Trial registration: ACTRN12616000811404 Ethics approval: (HREC/15/QGC/310).

5. Participants • All 13 existing JCs invited to participate • 9 JCs agreed to participate & randomised into TREAT and Standard • TREAT (n=61) Standard (n=64) Focus group: • Purposive sampling used to invite 18 clinicians from TREAT to participate  8 participated.

6. Procedure All Journal Clubs asked to meet 1x month for ~1 hr for 6 months. • TREAT format: incorporated 11 key components for effective JCs • Standard format: continued existing format Outcome measures: Collected pre and post • The Evidence Based Practice Questionnaire (EBPQ). 24 item self report questionnaire re perceptions of practice, attitudes and knowledge • Assessing Competence in EBM (ACE) tool: 15 item objective measure of applied EBP skills to hypothetical clinical scenario Post only: • Tailored satisfaction & influence on clinical practice questionnaire (likert scale and free form responses) • Focus group (participants from TREAT only)

7. Data analyses Quantitative Analysis: • Between group differences analysed using: • ANCOVA: assess changes in EBPQ & ACE tool pre-to-post-assessment • Independent t-tests: JC satisfaction and influence on clinical practice • Chi-squared analyses: frequency of different clinical changes Qualitative analysis: • Content analysis: initial themes from the questionnaires • Thematic coding + discussion, checking: focus group data Mixed methods interpretation: Both quantitative and qualitative data were analysed independently & then brought together for interpretation

8. Results: Participant Demographics • No differences between groups pre-assessment • Majority female (85%) • Majority aged 20-29 years (38%) or 30-39 years (36%) • ~50% of participants 2- 5 years (23%) or 5-10 years (24%) clinical experience. • Attended average 4 of 6 journal club sessions ( 0 to 6) • 64% participants completed post assessment.

9. Results: EBPQ (self-report) • No significant differences between groups • Both groups scored at mid point across practice and knowledge measures pre and post • Both higher end for attitudes pre and post

10. Results: EBP skills- ACE tool • No significant between group differences • Marginal increase in searching literature skills in TREAT group compared to standard (not significant) • Both groups remained at intermediate level across pre-post assessment

11. Results: Satisfaction All participants rated JCs highly for overall satisfaction, usefulness and value TREAT JC participants significantly more satisfied with the • organisation of the journal club (p = 0.011) • agreed more strongly that it should continue (p = 0.039)

12. Results: Influence on clinical practice • Participants in both groups rated Journal Clubs as having a positive influence on clinical practice across sessions • Both reported different practice changes across six sessions

13. Results: Qualitative Themes • Four main themes identified from open ended Questions (both groups) and Focus group (TREAT)

14. Theme 1: Benefits of Journal Club “…in the past we probably have made clinical decisions based on what we think the research is saying and now I have more questions marks about it…” Both groups reported increased skills in appraisal. Unique themes identified from TREAT included: - Helps to question how you apply research • Changed attitudes towards research • More time efficient • Changed confidence in appraisal “less scary” “it[appraisal] was done as a group….previously the [presenter] would have … pre-prepared the appraisal so I think it was more time efficient.” “I’m much more critical and nit-picky... it shows you to [how to] really tear the article apart”

15. Theme 2: Facilitating mechanisms • Having academic facilitator, group appraisal, consistent tool, shared goal setting and library support • Increased discussions and participation “Having the academics… really wrapped up each journal club because we got conclusions out of it…..instead of people almost going a bit blindly..” “…it was more collaborative and … less stressful for the person that's actually brought that article to the group….It was more that the group finds the answers together.” “Having access to the library was good as well…they [librarians] fine tuned it and they were able to get a really useful article.”

16. Discussion First RCT of structured Journal Clubs in Allied Health Lack of quantitative changes: • Previous study comparing EBP interventions also reported lack of change on ACE tool (Illic et al., 2014) • Majority of items in ACE tool evaluate “cognitive knowledge rather than direct application in a clinical context” (p18) • Lack of change in EBPQ may be related to variability between professions using this measure as found previously (Lizarondoet al., 2012) • EBPQ attitudes self ratings also high at baseline so reduced likelihood of significant changes being identified. Positive qualitative responses: • Build on existing research evidence, adding insights for improving format and implementation of JCs for AHPs

17. Conclusions • Feasible to implement additional evidence-based components within a structured Journal Club and it may result in greater clinician satisfaction. • Journal Clubs can support and maintain intermediate level knowledge & skills in EBP and positively influence clinical practicehowevermay not be entire answer for increasing EBP competency • Journal Clubs may be useful intervention to integrate with other multi-faceted strategies to increase EBP skills & evidence uptake

18. Implications for Practice • Journal Clubs may have positive influence on clinical practice • Identified some key evidence based components that may enhance clinicians’ satisfaction with existing Journal Clubs including: • Academic facilitator • Group appraisal approach  promote more active participation, reduce time burden

19. Limitations & Research Implications • Considering all of Allied Health clinicians together may have missed subtle changes within individual clinicians and between professions(Lizarondo et al., 2012) • Rotating clinicians reduced number of consistent attendees across pre-post assessments, reduced power of sample post • Similar pragmatic trials across different settings looking at comparative contributions of different key components may be helpful.

20. Key References LizarondoL, Grimmer-Somers K, Kumar S: A systematic review of the individual determinants of research evidence use in allied health. Journal of Multidisciplinary Healthcare 2011, 4:261-272. Caldwell E, Whitehead M, Fleming J, Moes L: Evidence-based practice in everyday clinical practice: strategies for change in a tertiary occupational therapy department. Australian occupational therapy journal 2008, 55(2):79-84. DeenadayalanY, Grimmer-Somers K, Prior M, Kumar S: How to run an effective journal club: a systematic review. J EvalClinPract2008, 14(5):898-911. LizarondoL, Grimmer-Somers K, Kumar S, Crockett A: Does journal club membership improve research evidence uptake in different allied health disciplines: a pre-post study. BMC Res Notes 2012, 5:588. Honey CP, Baker JA: Exploring the impact of journal clubs: a systematic review. Nurse Educ Today 2011, 31(8):825-831. Harris J, Kearley K, Heneghan C, Meats E, Roberts N, Perera R, Kearley-Shiers K: Are journal clubs effective in supporting evidence-based decision making? A systematic review. BEME Guide No. 16. Med Teach 2011, 33(1):9-23. .Linzer M, Brown JT, Frazier LM, DeLong ER, Siegel WC: Impact of a medical journal club on house-staff reading habits, knowledge, and critical appraisal skills. A randomized control trial. Jama 1988, 260(17):2537-2541. MacraeHM, Regehr G, McKenzie M, Henteleff H, Taylor M, Barkun J, Fitzgerald GW, Hill A, Richard C, Webber EM et al: Teaching practicing surgeons critical appraisal skills with an Internet-based journal club: A randomized, controlled trial. Surgery 2004, 136(3):641-646. LizarondoLM, Kumar S, Grimmer-Somers K: Supporting allied health practitioners in evidence-based practice: a case report... including commentary by Goodfellow LM. International Journal of Therapy & Rehabilitation 2009, 16(4):226-236. Milinkovic D, Field N, Agustin CB: Evaluation of a journal club designed to enhance the professional development of radiation therapists. Radiography 2008, 14(2):120-127. McQueen J, Miller C, Nivison C, Husband V: An investigation into the use of a journal club for evidence-based practice... including commentary by Dobrzanska L and Kanthraj GR. International Journal of Therapy & Rehabilitation 2006, 13(7):311-317. LizarondoL, Grimmer-Somers K, Kumar S: Exploring the perspectives of allied health practitioners toward the use of journal clubs as a medium for promoting evidence-based practice: a qualitative study. BMC Med Educ2011, 11:66. Upton DU, P: Development of an evidence-based practice questionnaire for nurses. Journal of Advanced Nursing 2006, 53(454-458). IllicD, Bin Norden R, Glasziou P, Tilson J, Vaillaneuva E: Development and validation of the ACE tool: assessing medical trainees competencey in evidence based medicine. BMC Med Educ2014, 14:114. IlicD, Nordin RB, Glasziou P, Tilson JK, Villanueva E: A randomised controlled trial of a blended learning education intervention for teaching evidence-based medicine. BMC Med Educ2015, 15(1):39. L. M. Lizarondo KG, Somers, Kumar S: Exploring the perspectives of allied health practitioners toward the use of journal clubs as a medium for promoting evidence-based practice: a qualitative study. BMC Med Educ2011, 11:66.

21. Thankyou for listening Any Questions?

22. Theme 3: Challenges “Competing priorities and clinical caseloads” “ Videoconferencing and technology issues” Theme 4: Suggestions for improvement TREAT participants reported: • Extra training • Ongoing support or “check-ups” with academic facilitator. • Timing of club “basic training… on how to interpret research articles, plot charts, p values etc” “earlier timeslot in the day to improve concentration”