Patient Viewpoint: A Website for Patient-Reported Outcomes Assessment

1. Patient Viewpoint:A Website for Patient-Reported Outcomes Assessment Claire Snyder, PhD Albert Wu, MD, MPH Johns Hopkins Schools of Medicine and Public Health

2. Presentation Objectives • To present the results from a literature review and qualitative research study supporting the development of a prototype website for patient-reported outcomes assessment, including: • aspects of patient reported health that are useful to assess • website features that patients and providers desire • barriers that need to be addressed • Prototype demonstration

3. Background (1) • 2004 NIH Roadmap RFA: Meetings and Networks for Methodological Development in Interdisciplinary Research • “applications for meetings or networks to develop methodologies that will facilitate interdisciplinary health research among behavioral or social scientists and investigators in the biomedical, mathematical/computational, physical sciences, and/or engineering”

4. Background (2) • Creation of the Quality of Life in Clinical PracticeInterest Group within the International Society for Quality of Life Research • Goals of using quality of life in clinical practice: • To improve clinician-patient communication • To facilitate individual patient management • To assist with medical decision-making

5. Research Proposal Creation of a network of internationally known researchers and practitioners to develop a centrally accessible website for patient-reported outcomes assessment that links to the electronic medical record

6. Goals of the Grant • Determine what aspects of patient reported health would be most useful for evaluating the effectiveness and quality of outpatient oncology practice • Explore desirable features for a web-based system to administer standardized questionnaires to patients • Explore potential technical, cultural, ethical, and legal barriers to the use of such a system • Develop a prototype website for use by outpatient oncology providers to collect patient-reported data in the clinical setting

7. Research Network Members • Albert W. Wu, MD, MPH (Johns Hopkins University) • Neil Aaronson, PhD (Vrije Universiteit, the Netherlands) • Michael Brundage, MD, MSc (Queens University, Canada) • Carolyn Gotay, PhD (University of Hawaii) • James Hodge, JD (Johns Hopkins University) • Denise Hynes, RN, MPH, PhD (Veterans Administration) • Claire Snyder, MHS, PhD (Johns Hopkins University) • John Wasson, MD (Dartmouth University) • Susan Yount, PhD (Evanston Northwestern Healthcare/Northwestern Univ.) • Brad Zebrack, PhD, MSW, MPH (University of Southern California)

8. Ably Assisted By: • J.B. Jones, MBA, PhD(c) • S. Orion Courtin, MHS • Roxanne Jensen • Web Development Team

9. Methods • Literature Review • First Research Network Meeting (Oct ’05) • Develop Website Specifications • Second Research Network Meeting (Apr ’06) • Website Design • Qualitative Research • Third Research Network Meeting (Feb ’07) • Website Development • Usability Testing

10. Literature Review • Rationale for web-based PRO assessment • Existing electronic and web-based platforms for collecting PROs • Desirable features from clinician’s and patient’s viewpoint • Potential ethical and legal issues Jones et al. Quality of Life Research 2007;16:1407-1417

11. Advantages of Web-Based PRO Assessment • Patient can complete the assessment anytime from anyplace with Internet access • Reminders and decision support can be built in • Scoring is automatic • Comparison scores immediately available • Data is easily accessible (privacy and security issues) • Summary reports can be generated at the patient level or practice level Jones et al. Quality of Life Research 2007;16:1407-1417

12. Desirable Features • Efficiency and Ease of Use • Will the website make efficiency better or worse? • Will the website fit into clinical workflow? • Will it add to my workload? • Will patients have an easy time using it? • Results Presentation and Interpretation • What do the results mean? • What should the clinician/patient do about them? Jones et al. Quality of Life Research 2007;16:1407-1417

13. Ethical and Legal Issues • Privacy Rule: • Protected health information can only be disclosed with written authorization unless certain criteria for exemption are met • Exemptions include public health research, minimal risk research that cannot be practicably done without a waiver • Security Rule: • Defines the technical, administrative, and physical safeguards that must be put in place to protect the confidentiality, integrity, and availability of protected health information Jones et al. Quality of Life Research 2007;16:1407-1417

14. Patient Viewpoint • A web-based system with a database back end for management and storage of patient responses • Developed using Microsoft ASP.Net and SQL database • Linked to the electronic medical record • Three main functions provided by the system: • Clinicians select questionnaires to administer to patients and how often • Patients enter responses, submit comments, and view the results of their response • Clinicians have a text and graphical view of patients’ responses and scores over time • Security • Database and programming code located on separate servers • Servers backed up regularly and backups stored at a remote location • Secure Sockets Layer secure the site as patients enter their responses and physicians view the results • Access to the system is controlled by requiring username and password

15. Qualitative Research • Interviews • 20 Cancer Patients • 3 Oncologists • 4 Research Nurses • Semi-structured in-depth interviews • Followed structure of the prototype website • Probed for opinions, possible improvements, and innovations

16. Findings-Patients (1) Keep surveys short and simple“I always have questions for him about my well-being, but if it becomes too long or too complicated, I’ll give up.” Response: Worked with physicians to choose a concise package of measures to lower respondent burden

17. Findings-Patients (2) Present multiple questions per survey page, but avoid long scrolling ““I would [prefer multiple questions], as long as there’s not too much.” Response: Will present questions with same response type together on a page

18. Findings-Patients (3) Some patients are interested in their survey results, others just want to comply with the request of their care provider “Somebody is going to find it interesting, but I’m not, you know personally, once I’ve done the survey and made my additional feedback, I’m basically done.” Response: Provide simple feedback for patients who are interested

19. Findings-Patients (4) Patients interested more in changes over time than survey values “It will be more meaningful just to see the individual change. I want to be able to look back six months and say, where was I six months ago.” Response: Will generate simple graphs showing change over time

20. Findings-Patients (5) Patients suggest making direction of improvement consistent across types of measures “I’d want them to run in the same direction, with high being bad and low being good. That’s like these tests they give us, and you need to be consistent.” Response: Explore options for making results easy to interpret while keeping in mind issues with violating conventions

21. Findings-Patients (6) Patients want direct communication, want their comments to inform the next visit, worried it will not be read “You need to go to your doctor with questions in writing, and it seems that this would be a vehicle to get those questions there.” “I am concerned where you say that you won’t read it until the next appointment,…, what am I chopped liver or something?” Response: Will keep the free text response and integrate it into the clinician results

22. Findings-Patients (7) Patients perceive the site’s purpose as valuable, concerned it will become burdensome “It would be useful from their point of view, so I’d be willing to take five minutes to fill out the thing.” Response: Will clearly state the purpose of the site, keep surveys short and continue to improve usability

23. Findings-Clinicians (1) Clinicians need guidance for interpreting the meaning of values “Of course I have no idea if this is a good score or a bad score.” Response: Will present results on commonly understood dimensions for their patients on standard scales

24. Findings-Clinicians (2) Clinicians are most interested in meaningful individual change over time “The individual patient values would be more useful, I want to know about this individual patient, some patients will always score low, this doesn’t represent change over time, you can’t always make a miserable person better, is there something that I can fix?” Response: Focusing results presentation on individual changes over time, exploring ways to determine what constitutes a significant change for each measure

25. Findings-Clinicians (3) Need ways to emphasize the key findings to focus on during the visit “We’re all taught to fix the one thing that the patient needs today, is there one parameter that is tipping the scale, or are they generally having trouble.” Response: Added a question after each survey asking what a patient would most like their care provider to address at their next visit

26. Findings-Clinicians (4) The website could be useful in clinical practice, but it needs to be easy to integrate into current routines with little administrative burden “I would want someone to print the information out to be reviewed with the chart, or linked directly in the EPR. I need something to hit me over the head to indicate that there’s something I need to look at.” Response: Adding a batch print feature to print results for patients based on scheduling Response: Working with Electronic Patient Record personnel to integrate with their system

27. Findings-Clinicians (5) Initially we envisioned clinicians choosing from a menu of surveys for each patient, but clinicians were concerned about administrative burden and lack of familiarity with different surveys Response: We conducted research to narrow the choice of measures, then met with physicians to choose the ones most appropriate to their practice

28. Summary of Qualitative Study Findings • Respondents optimistic about potential value but concerned about practicality and integration into current practice • Use short surveys and a straightforward interface for patients • Integrate results into the current clinical routine through the Electronic Patient Record or print for the paper chart • In results presentation, both patients and clinicians were most interested in individual changes over time

29. Demonstration of Patient Website • Patient visits site and logs in

30. Demonstration of Patient Website • Patient receives welcome screen

31. Demonstration of Patient Website • Patient is presented with disclaimer

32. Demonstration of Patient Website • Patient is presented with survey(s) to complete

33. Demonstration of Patient Website • Patient survey – sample question

34. Demonstration of Patient Website • Patient survey – sample question

35. Demonstration of Patient Website • Patient survey – sample question

36. Demonstration of Patient Website • Patient gets a last chance to answer skipped questions

37. Demonstration of Patient Website • Patient completes survey and can ask their Doctor a question or view results

38. Demonstration of Patient Website • Patient views results

39. Demonstration of Clinician Website • Clinician logs in and can search for patient

40. Demonstration of Clinician Website • Clinician can search for patient by survey

41. Demonstration of Clinician Website • Clinician can view patients results

42. Demonstration of Clinician Website • Clinician can search for another clinician

43. Demonstration of Clinician Website • Clinician can add or edit a patient

44. Demonstration of Clinician Website • Clinician can schedule a patient a survey

45. Demonstration of Clinician Website • Clinician can add/edit another care team clinician

46. Next Steps • Usability testing is ongoing • R21 submitted to conduct initial pilot test • Use • Usefulness • Acceptability

47. Questions?