1 / 64

Chapter 6. Making Success Measurable Through Quality Control

Chapter 6. Making Success Measurable Through Quality Control. Chapter 6. Section 1. Introduction to Quality Control. Introduction. How do you know the quality and accuracy of your rent calculations? Wait for HUD to tell you? Close your eyes and hope for the best?

koto
Download Presentation

Chapter 6. Making Success Measurable Through Quality Control

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Chapter 6.Making Success Measurable Through Quality Control

  2. Chapter 6. Section 1. Introduction to Quality Control

  3. Introduction • How do you know the quality and accuracy of your rent calculations? • Wait for HUD to tell you? • Close your eyes and hope for the best? • Assume that good people lead to good outcomes? • Wait until you tally SEMAP scores? • Or …..

  4. Introduction • Are you interested in a better way that could: • Enable you know when to take action on performance issues, • Help document the improvements in error reduction you are making, • Enhance the quality of the services you provide?

  5. Introduction • A good quality control program can be that better way that helps you: • Prevent errors • Documents compliance with program requirements • Demonstrate performance improvement over time

  6. Introduction • An effective quality control program: • Facilitates conversation about expectations and performance for individuals and the organization as a whole. • Enables program managers to be proactive, rather than reactive

  7. Introduction In this session, we will talk about: • Three types of quality control • Illustrate the thought process needed to develop a QC program • Provide sample QC tools • Provide guidance on communicating with your staff about quality issues.

  8. Introduction • We will focus on HUD’s expectations related to income and rent determination • BUT, the QC concepts we discuss apply equally well to other parts of a PHA operation.

  9. Why Quality Control? • Quality control can help to confirm what you already know. It can: • Provide documentation that supports recommendations you want to make, • Generate statistics that document staff performance, • Provide valuable information on a systematic basis to share within and outside your organization. .

  10. Why Quality Control? • QC can be even more helpful if the results are a surprise. Then the results may highlight: • Problems of which you were not aware • Good news about improved performance or excess capacity. • Were there aspects of the RIM review for your agency that were a surprise?

  11. Chapter 6. Section 2.Types of Quality Control

  12. Preventive Types of Quality Control Continuous Types of Quality Control Periodic

  13. Preventive QC • Preventive QC is checking on the quality before the transaction is completed. Use Preventive QC when: • The consequences of errors are high, or • The error is not recoverable. • Examples: • Supervisors sign termination letters even though specialists sign other correspondence • Occupancy supervisor reviews 100% of rent calculations for new staff (for some period of time).

  14. Preventive QC • More examples: • Intake specialist presents a “dry run” of the briefing to staff before doing the real thing in front of participants. • What examples of preventive QC occur at your PHA? • Try to identify at least one additional area (related to income and rent) where preventive QC might be helpful.

  15. Limits of Preventive QC • Preventive QC can be an important tool but what are its limitations? • PHA’s don’t have time or staffing to double and triple check every transaction –you need to prioritize. • Use Common Sense: Not everything requires this level of effort.

  16. Periodic Quality Control • In contrast, periodic quality control happens after the fact. Work has already been done and reported. • It’s purpose is to take a “picture-in-time” assessment of PHA operations that will help improve the program and prevent future errors. • Sound familiar? Yes – HUD’s RIM reviews and SEMAP audits are kinds of periodic QC.

  17. Periodic Quality Control • During a periodic review HUD systematically looks at: • Policies, procedures, and practices and • Individual files. • But you don’t have to wait for HUD. You can also do your own. You could: • Look at the same things HUD will be checking, or • Focus your periodic review differently to learn even more.

  18. Limits of Periodic QC • Periodic QC can provide an accurate point-in-time assessment, but: • It doesn’t help managers see trends • Results come infrequently so it doesn’t help managers take corrective action during the year. • The level of effort required may seem overwhelming (but HUD’s RIM Review guide provides a good model for the review).

  19. Continuous Quality Control • Continuous QC is used to help with day-to-day management. • It involves analyzing a few (but very important) factors on a regular, continuing basis. For example: • Tracking on-time reexaminations each month • Looking at accuracy rates by individual or transaction

  20. Continuous QC • Continuous QC helps managers: • Focus more quickly on vulnerable areas. • Look at trends, rather than point-in-time information. • HUD does some continuous QC of 50058 submissions and uses the data to: • Send error reports to PHAs, and • Assess in general terms how PHAs are doing. • But only the PHA can implement an effective continuous QC program.

  21. Limits of Continuous QC • Continuous QC is a powerful tool. • But, remember, the scope of the Continuous QC is limited. • This means: • Continuous QC provides only a limited picture (hopefully of the key areas) • Continuous QC should be supplemented by Periodic Reviews that take a broader view.

  22. Continuous QC Example Page 6-11 and 6-12 • Look at the “Error Analysis” chart • QC results are shown by quarter & individual • Caseload = annual caseload • Errors = things HUD would call findings • Comments = errors that would not be findings • Types of errors are defined on page 6-12

  23. Continuous QC Example • What issues does this report raise for you? • What are your concerns about: • Productivity? • Types of errors? • Error Rates? • What are the implications of the information you see for this PHA?

  24. Chapter 6. Section 3.Designing a QC Program

  25. Designing a QC Program • Now that we have discussed each type, it’s time to think about how to design your program. • Before you begin it is very important to think through: • What you want your QC program to look at. • What you will do with the information.

  26. Designing a QC ProgramWhere to Start • To focus your program ask three questions: • What are the outcomes you want to achieve? • What would success look like for each outcome? • What is the best way to confirm (track, measure) performance in each area? • Then decide: • Which measures are useful and feasible. • How you are going to record the information. • Who needs to know the results.

  27. A Simple Example- Sleep • What is my desired outcome? • A good night’s sleep • What would success look like? • Eight hours of uninterrupted sleep • What is the best way to measure? • An independent 3rd party could observe my sleep. • I could track myself the time I go to bed, time I wake up, and number of interruptions.

  28. Simple Example (Cont) • Decision: • Having someone watch me sleep could be expensive and disruptive • Instead, I will track sleep patterns myself • I’ll write this down in a notebook • No one needs to see it but me (unless I think maybe my doctor should know)

  29. A More Serious Example – Utility Allowances • What is my desired outcome? • Current and correct utility allowances will always be used in rent calculations. • What would success look like? • Utility allowances would be used correctly in 100% of the files. • Are there other ways you would describe success re: utility allowances?

  30. A More Serious Example – Utility Allowances • What are possible ways to measure the outcome? • 100% file review in advance (preventive) • 100% file review after the fact • Sample file reviews after the fact • What else could we do to see if the correct utility allowances are being used consistently?

  31. A More Serious Example • Decision: • 100% review would be too time consuming (before or after) • Therefore, review a sample of files (after the fact) to see how often staff have used the correct utility allowance • Generate a report that summarizes errors by employee, by program type, by transaction type.

  32. Thinking About Outcomes • Choosing the outcomes you want to use is an important part of the design process. • Outcomes can be expressed in very general or very specific terms. • Some “big picture” outcomes might include: • Consistency in data collection, documentation • Complete and accurate verifications • Accuracy in rent calculations • Timeliness • Complete, accurate reporting

  33. Thinking About Outcomes • Even broader outcomes might include: • No findings on the next RIM review • Informed decision making by applicants, participants and HCV landlords • Fair and equitable treatment • Equitable workload • Being a SEMAP high performer • Positive perceptions – we’re a good agency.

  34. Thinking About Outcomes • Caution: Generally more specific outcomes are better understood and easier to measure. For example: • Reduce departmental error rate to less than one percent • Complete 100% of annual reexaminations accurately and on-time • The more specific the outcome, the easier it is to track performance.

  35. Designing a Program to Support Outcomes • Page 6-20 through 6-21 show examples of turning desired outcomes into a QC plan. • Follow along on the chart • We will walk through Row 1 and Row 2 together

  36. The End Result • The end result of this thinking will be a Quality Control Plan. (see page 6-25). • The planning itself is more important that the format for the plan. • Think through the process before you begin data collection • Collect information about the items that have the most effect on error reduction • Be realistic about what data you can collect.

  37. What’s Next? • The balance of this chapter discusses 5 important “how to” aspects of QC: • Conducting file reviews • Conducting a “general” review of operations • Using reports to track quality • Addressing staff skills and skill gaps • Communicating about error reduction and quality control

  38. Chapter 6. Section 4.How to Aspects of Quality Control

  39. Conducting File Reviews • Files reviews are used: • To determine accuracy and completeness • To identify potential policy and management issues • “Files” for this purpose include the 50058 record as well as paper files. • Caution: A file review can tell you what has happened, but not necessarily why.

  40. Conducting Tenant File Reviews • Sample size and characteristics of files selected depend upon the purpose. • Generally PHAs use a “directed” sample. • Directed means selecting the files that contain the characteristics you want to monitor. • Page 6-28 shows characteristics that may be important to consider for the sample. • Not every characteristic must be represented each time you review files.

  41. Conducting File Reviews • For Continuous QC • Purpose is timely feedback on few issues. • File Selection Priorities would be: • Each specialist • Basic transaction types • Files completed over time

  42. Conducting File Reviews • For Periodic QC • Purpose is a more comprehensive review. • File Section Priorities: • Each specialist (but perhaps more files) • More transactions types • Larger cross section of income sources and household types

  43. Conducting File Reviews • Tools for Selecting Files: • PIC reports can generate lists for some transactions. (See Appendix A, A-25) • PHA vendor software may be able to generate reports that list files in categories. • How many files do I need to review? • QC sample should be sufficient to meet SEMAP requirements

  44. SEMAP File Sampling • Useful guideline for determining number of files to review

  45. Conducting File Reviews • Logistics • Use a file checklist to document your review. (RIM checklists are a good start.) • Complete 50058 calculations without looking at the completed calculations in the files. • Look for consistency between the paper and electronic files.

  46. What The Reviews May Show • Key RIM Review programmatic findings: • Earned income/pension/public assistance calculations • Incomplete/outdated PHA policies • Data not being transmitted to PIC • Lack of verification of income and deductions • File data doesn’t match data used to calculate rent • Improper calculation of utility allowances

  47. What the Reviews May Show • RIM reviews also found errors related to automation. • In general, these errors include: • Programming problems(software vendor) • Errors related to the way PHAs use automated systems such as: • Not attending to error reports and • Inadequate training for staff on the system • See page 6-32 for details

  48. Conducting the “General” Portion of a Periodic Review • Periodic Reviews enable PHAs to take the same broad look that HUD takes during a RIM Review. • The agenda: • Look at policy and procedural documents • Conduct an expanded file review • HUD’s RIM checklists are good models.

  49. Conducting the “General” Portion of the Review • You may need to add to HUD’s checklists: • Questions covering PHA-specific policies or performance standards • Protocols for checking how PHA-specific software handles income, rent and subsidy determinations.

  50. Using Reports to Track Quality • Perspective is important. • Identifying individual errors is important but it’s only one aspect of QC. • Looking at trends (performance over time) provides important insights that lead to error reduction. • Looking at trends can also help set performance standards.

More Related