Measuring Client Satisfaction

1. Measuring Client Satisfaction Forsyth County Site NC ADRC Project Mary Anne P. Salmon ADRC Teleconference on Evaluation May 25, 2006 Posted 5/23/06

2. Context • The first 2 “stations” in the model • Established in the community • Showed high levels of satisfaction in their own periodic consumer satisfaction surveys • Surveys to demonstrate continued high quality during transition to ADRC model

3. Survey Purposes • Gauge maintenance of overall satisfaction • Would you tell a friend or relative to call [Agency name]? • Track quality of service (CQI) • Was the person at Helpline who spoke to you friendly and courteous? • Was the person you talked with knowledgeable? • Was the information you received from Helpline clear and understandable? • Were the steps to obtain those services clearly explained? • What could Helpline do differently to make the services they offer more useful? (open-ended)

4. Survey Purposes continued • Examine effects of streamlining • Were you told to go to, or to call, any other places for a service or for more information? • If you telephoned, were you able to talk to a(n) [agency name] representative on the telephone on your first call, or did you need to leave a message? • If you left a message, when did the person return your call? • If you visited the [agency name] office, how long did you wait to see someone? • If you contacted [agency name] for services, are you receiving the service that you were seeking? • Call outcome • Did the information you received from [agency name] help you make a decision or find the service you needed?

5. Additional Data Collected • Referral Source • Demographics • Ethnicity • Gender • Condition/Category • Age 60+ • Physical disability • Developmental disability/metal retardation • Mental disorder • Alzheimer’s Disease/dementia • Head injury/cognitive damage

6. Methodology • Written survey • Mail-out with return envelope to AAA • Online version available for those who contacted the agency by e-mail (not used) • Population: Consumers contacting agencies in these timeframes • Senior Services pre-test: May 16–30, 2005 • Adaptables pre-test: August 21–September 29, 2005 • First post-test: month of February 2006

7. Methodology continued • Sample • 1 in 6 Senior Services consumers in timeframe • Systematic sample from a random start • Pre-test N = 119 • First post-test N = 69 • All Adaptables consumers in timeframe • Pre-test N = 43 • First post-test N = 75 • Response Rates • Senior Services pre-test: 23% • Adaptables pre-test: 30% • First post-test: 23% • Adaptables: 29% • Senior Services: 16%

8. Low response rate + small N = Poor ability to detect significance Limitation of the Study In Fact Few respondent differences are even suggestive of real differences in the population. Unless otherwise stated, differences between agencies and differences between pre- and post-tests were neither suggestive nor statistically significant.

9. Findings • Overall satisfaction • 91% would recommend the service • 9% “unsure” • Quality of service • 100% said representative was friendly and courteous • 94% said representative was knowledgeable • 94% said the information they received was clear & understandable • 100% of those referred elsewhere said steps were clearly explained

10. Findings continued First Contact • Streamlining—phone process • Reached a representative • 68% reached a representative • 32% left message • Call returned • None within first hour • 25% within a day • 38% next day • 25% same week • 12% did not remember • Not improved from pre-test yet Bottom line: Problem with phone coverage “depth” when representatives are out of office

11. Findings continued • Streamlining—walk in visits • 100% were seen within 5 minutes • Streamlining—referred to other agency • 50% were referred • Of those referred and seeking services (N = 10) • 40% receiving them • 60% expecting services soon, on waiting list, or other • Increase from pre-test suggestive but not significant* *At pre-test 35% of Helpline and 0% of Adaptables respondents were receiving services.

12. Findings continued • Outcome • 83% said the information they received helped them find a service or make a decision • Another 10% were calling for future information (not seeking a service or decision) • Referral Source • 44% from friends or family members • 25% from other agencies (hope to see this ) • 15% from media (hope to see this )

13. Findings: Condition of Consumer

14. Findings continued • Diversity of Consumers Responding • Ethnicity • 72% White • 19% African American • 6% Asian or Asian American • 3% Multi-racial • Gender • 66% women

15. Findings continued • Do Ethnicity or Gender Affect Satisfaction? • Ethnicity—Both were largely very positive • African Americans slightly but significantly less likely to say they would recommend to a friend • African Americans give slightly lower (not significant) ratings to “representative was knowledgeable” and “clear and understandable” • No difference in “friendly and courteous” • Gender • No significant differences • Observed differences show men more positive

16. Questions????Comments?