1 / 24

Using Consumer Surveys for Quality Management: A Modified PES as One Option

Using Consumer Surveys for Quality Management: A Modified PES as One Option. Presentation to Cash & Counseling States. Sara Galantowicz July 11, 2006. Agenda . Consumer Surveys and Quality Management Modified Participant Experience Survey as one Survey Option PES Overview ME CD-PAS version

Download Presentation

Using Consumer Surveys for Quality Management: A Modified PES as One Option

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Using Consumer Surveys for Quality Management: A Modified PES as One Option Presentation to Cash & Counseling States Sara Galantowicz July 11, 2006

  2. Agenda • Consumer Surveys and Quality Management • Modified Participant Experience Survey as one Survey Option • PES Overview • ME CD-PAS version • WV hybrid • Additional Issues for Consideration Consumer Surveys for Self-Direction

  3. Consumer Surveys • Consumer surveys are one of many sources of quality data • Consumer experience • Knowledge, attitudes, and practices • Satisfaction • Where you want to end up is where you want to start • Research questions • Data gaps • Suspected problem areas • Mandates • Changes/policies requiring evidentiary support • CMS and other reporting • Who needs to know what Consumer Surveys for Self-Direction

  4. Consumer Surveys Options • Designing your own instrument • Flexible • Specific • Reflects stakeholder input • Using/modifying existing instruments • Stealing can be good! • Efficient • Testing • Comparability • Either approach requires involving the right players and technical expertise Consumer Surveys for Self-Direction

  5. Looking at Existing Tools • Multiple tools available • Vary on several dimensions • Target population • Domains or areas of emphasis • Method of administration • Role of proxy respondents • Length of survey • Resulting measures • Evaluation Criteria • Scope • Testing • Reliability and validity • Cost • Availability of benchmark data and support Consumer Surveys for Self-Direction

  6. Participant Experience Survey PES is a technical assistance tool for States • Solicits participant feedback on HCBS waiver services/supports • Generates indicators for QA/QI • Developed by The Medstat Group, Inc. for CMS/DHHS • Experience survey for participants only • Not a satisfaction survey • No proxy version • Designed to be administered in-person • Optional • Extensively tested before release • Cognitive testing of individual items • Field testing to evaluate administration and tool properties Consumer Surveys for Self-Direction

  7. PES Development • Combination of quantitative and qualitative methods • Work Group provided technical assistance and oversight, and specified priority areas • Review of related instruments • Draft items to operationalize domains • Cognitive testing (Phase I) • Field testing (Phase II) Consumer Surveys for Self-Direction

  8. PES: Current Versions PES E/D • Frail elderly and adults with physical disabilities • 33 indicators • Free automated survey software available • Responses entered directly into laptops • Pull-down menus for data analysis • PES MR/DD • Adults with mental retardation or developmental disabilities • 51 indicators • Includes 8 “core” questions for participants with severe cognitive impairments • Automated version • PES BI • Adults with acquired brain injuries • Up to 58 indicators (two optional modules) Consumer Surveys for Self-Direction

  9. PES: Domains • Access to Care • Are participants’ needs for personal assistance, adaptive equipment, and case manager access being met? • Choice and Control • Do program participants have input into the types of services they receive and who provides them? • Respect and Dignity • Are program participants treated with respect by providers? • Community Integration • Do program participants engage in activities and events of their choice outside their homes, when they want to? Consumer Surveys for Self-Direction

  10. Sample Items • Access to Care • Do you ever go without eating when you need to? • Is this because there is no one there to help you? • Have you ever talked with your case manager or support coordinator about any special equipment, or changes to your home, that might make your life easier? • Choice and Control • Do you help pick the people who are paid to help you? • Would you like to help pick the people who are paid to help you? • When you are at home, can you eat when you want to? (MR/DD) • Can you talk to your case manager or support coordinator when you need to? Consumer Surveys for Self-Direction

  11. Sample Items • Respect and Dignity • Do the people paid to help you treat you respectfully in your home? (E/D) • Do support staff in other places, such as at work, or at a day program, say “please” and “thank you” when they ask for something? (MR/DD) • Does anyone ever do mean things to you, such as yell at you? (MR/DD) • Community Integration • Do you like to go out to eat? (MR/DD) • Do you go out to eat? (MR/DD) • Do you get to help pick where you eat out? (MR/DD) • Is there anything you want to do outside your home that you don’t do now? Consumer Surveys for Self-Direction

  12. Maine’s CD-PAS Survey • State seeking feedback from CD-PAS participants • Non-elderly adults with physical disabilities • Reviewed PES E/D and identified gaps • Crafted new survey items • Stakeholder input • Operating agency • Medicaid agency • Provider (Alpha One) • Consumers • Testing • Cognitive • Field Consumer Surveys for Self-Direction

  13. Maine’s CD-PAS Survey • Domains • Program Referral and Support • Access to Services • Respect and Dignity • Technology and Community Integration • Interviewer comments and observations • Second year of use • Interviews conducted by staff from the Muskie School’s survey unit • Early results reflected political uncertainty • Modifications made based on findings Consumer Surveys for Self-Direction

  14. Modifying the PES in Maine • Added items specific to self-direction • How PAs located and hired • PAs’ relationship to waiver participant • PA retention and back-up • Support from Alpha One Independent Living Specialists • Experience with self-direction • Retained relevant items from PES • Respect and dignity • Unmet need for personal assistance • Adaptive equipment • Physical and verbal abuse, theft Consumer Surveys for Self-Direction

  15. Crafting the new items in Maine • General design principles and traps to avoid • Consistent response categories • Appropriate response categories • Adequate response categories • “Problem” words • Sound-alike words • Compound or double-barreled questions • Jargon and unfamiliar words • Complicated concepts • Relevance to consumers (screening questions) • Sensitive, appropriate language • Hypothetical questions • Scales • Specificity of times, dates, frequency, and amounts Consumer Surveys for Self-Direction

  16. Testing the new items in Maine • How people answer a survey question • Comprehension • Retrieval • Estimation • Response • Each area is a potential source of bias • Cognitive testing: determining if the question you think you are asking is the one waiver participants are hearing and answering • Iterative process of asking draft questions with probes • Small n, longer interviews • Discuss and act on results • Resulted in many changes Consumer Surveys for Self-Direction

  17. Field Testing in Maine • Larger n, shorter interviews • Evaluate survey logistics • Reaching consumers • Scheduling interviews • Interview length • Interview acceptability • Obtain baseline data • Evaluate appropriateness of tool • Examine utility of findings Consumer Surveys for Self-Direction

  18. West Virginia’s Combined PES Tool • State seeking feedback from both traditional E/D waiver participants and those enrolling in new Personal Options waiver • Developed hybrid tool with separate skip patterns for traditional and self-directing waiver participants • Multiple players • Medicaid agency • Operating agency • UCED staff • Stakeholder advisory group Consumer Surveys for Self-Direction

  19. WV’s Combined PES tool (con.) • Many common items • Access to care/unmet need • Respect and Dignity • Abuse/theft • Choice and control items for traditional waiver • Primarily same questions from PES E/D • Some new items • New Program Referral and Support items for Personal Options participants • Modified the Maine questions • Additional items on resource consultants and financial management services Consumer Surveys for Self-Direction

  20. Testing the WV Survey • Iterative development process • Reviewed PES E/D and Maine tool • Developed new questions • Lots of review and feedback • Consumer review • Expert review • Limited cognitive testing • Field testing to begin this summer Consumer Surveys for Self-Direction

  21. Issues for Consideration • What are your goals for the survey effort? • Whom do you want to survey and why? • Where will these data fit into your QM strategy? • What additional information would you like to capture? • With whom do you plan to share the data? • One-time vs. longitudinal use • What is your target population? • Sampling frame • Criteria for inclusion or exclusion • Sampling plan • Sample size • Stratification • Proxies? Consumer Surveys for Self-Direction

  22. Additional Issues for Consideration • Interviewer Selection and Training • Who will be conducting interviews? • Who will attend training? Who will conduct it? • Level of effort • Ongoing data collection vs. limited period • Contacting participants • How will sample respondents be contacted? • State staff, case managers, other • Scheduling and briefing respondents • Data collection on non-respondents Consumer Surveys for Self-Direction

  23. Additional Issues for Consideration • Pre-survey information • What kind of information can you access about participants before the interview? • How can you use this information to tailor the interview and provide appropriate supports • Assess ability to complete survey • Special issue for consideration! • Timeframes • When will interviews start • How long will the process last • When would you like to be able to have and share results Consumer Surveys for Self-Direction

  24. Other Issues for Consideration • Electronic data entry • Psychometrics • Inter-rater reliability studies • Validity concerns/cognitive screening • Data Analysis • Who will develop the analysis plan? • Who will conduct the analysis • Acting on and sharing your results • Who should see the findings and how best to share them? • Determining appropriate QI projects • Involving stakeholders • Need to consider this up front Consumer Surveys for Self-Direction

More Related