160 likes | 254 Views
Jon Brassey shares frustrations with current evidence practices and proposes enhancements for robust clinical answers. Highlights issues with evidence quality, search paradigms, and information relevance. Recommends a shift towards prioritizing practical utility over methodological purity. Advocates for improved search tools, collaboration with clinicians, and tailored information delivery to meet their needs effectively.
E N D
What frustrates me about evidence and what changes I'd like to see! Jon Brassey, TRIP Database
Background • Worked in information support for over ten years • Started in primary care Q&A - answered over 10,000 clinical questions • Developed the TRIP Database –searched over 50 million times • Main interest – getting robust answers to clinicians queries!
Why am I annoyed? • Evidence is not focussed on supporting clinical practice. For me this can be broken down into: • Problems with the evidence • Problems with search
Problems with evidence • Frequently doesn’t answer clinical questions • Methodological purity seems more important than clinical usefulness
Analysis of dermatology questions • 357 Q&As analysed • How many questions were answered by systematic reviews? • >50% • 25-49% • 10-24% • <10%
Analysis of dermatology questions • Only 3 questions answered by a SR! • 63% therapy, 9% diagnosis, 9% etiology • 20% answered using secondary evidence alone • Not much focus, DUETs aside, at feeding in coal-face information needs with production of evidence
Evidence Updates • Run by Brian Haynes at McMaster via funding from the BMJ • Examine 120 core clinical journals and rate them on: • Quality • Relevance • Newsworthiness • Overall >95% rejected
Problems with search • 100 clinicians • Pain • NHS Evidence
Problems with search – 100 • Information seeking defined by the Google generation • Ask 100 clinicians the support they’d want, how many would say – ‘Give me a search box where I add terms and then return 10 results which I need to read in the hope that they will answer my question’.
Problems with search – NHSE • Nearly £25 million budget • Around £15 million on non-content, including £730,000 marketing • 385,000 searches per month or 4,620,000 annually (excludes CKS and SLs/SCs) • >£3 per search • TRIP Database < 1p
Summary of problems • Evidence not fit for purpose • Not findable • Usefulness medical information = (relevance x validity)/ work (Slawson & Shaughnessy)
Solutions – better evidence • Build on the work of DUETs and create a system to record clinical uncertainty • Better understand the clinical information needs of ‘normal’ clinicians • Better prioritisation of SRs and primary research • Less is more
Solutions – improve search • Work with clinicians and throw away the existing search paradigm • Work better with the search intentions • Break down full-text • Be ambitious and not be constrained by the Google ‘mentality’.
TRIP’s experiments • TILT – http://tilt.tripdatabase.com • Shared learning tool • Clinicians record their learning • Snippets of applicable evidence • Blitter – http://blitter.tripdatabase.com • Curated list of clinicians that blog or tweet • Index of content clinicians feel are noteworthy • Clinicians added based on speciality
Ultimately... • Need to give clinicians the: • The information they need • In the format they need • In the timeframe they need • To me answering their questions is vastly superior to what we currently have