Quality Improvement in Minnesota’s QRIS: Who and How?

1. Quality Improvement in Minnesota’s QRIS: Who and How? Kathryn Tout Presentation at the NACCRRA National Child Care Policy Symposium March 8, 2012

2. Acknowledgements • Parent Aware Evaluation Team • Rebecca Starr • Tabitha Isner • Jennifer Cleveland • Meg Soli • Katie Quinn • Ladia Albertson-Junkans • Minnesota Early Learning Foundation • Greater Twin Cities United Way • Office of Planning, Research and Evaluation in the Administration for Children and Families, U.S. Department of Health and Human Services • INQUIRE – Quality Initiatives Research and Evaluation Consortium

3. Improving program quality is a central goal of QRIS

4. What do we know about quality improvement approaches in QRIS? • QRIS Compendium – information from 26 QRIS collected in late 2009 from QRIS administrators and publicly available documents • The majority of QRIS provide TRAINING that is aligned with QRIS standards and ON-SITE TECHNICAL ASSISTANCE to support programs • These supports tend to focus primarily on “navigating” the QRIS and improving environment ratings • Training and TA address interactions with children to a lesser extent (see new report by NCCP)

5. What is the nature of on-site TA in QRIS? • Majority of QRIS in the Compendium reported that the frequency, length, and duration of onsite assistance varied by the needs of the program • Work usually is guided by a QI plan. Intensity and content of TA will vary also on the stage of the rating cycle (pre-entry or preparation phase; entry to the QRIS; re-rating and improvement phase) (see Isner et al, 2011)

6. Purpose of this Presentation • Review findings on quality improvement from one QRIS – Minnesota’s Parent Aware – to examine how well QI strategies are working • Analyze who improves and how they improve • Discuss implications of the findings for improving QI in QRIS

7. Overview of Parent Aware • Voluntary QRIS • Two rating pathways • Full rating: 1 to 4 stars • Accelerated 4-star rating for accredited programs, Head Start and school-based pre-kindergarten programs • Points are earned in four quality dimensions • Family Partnerships • Teaching Materials and Strategies • Tracking Learning • Teacher Training and Qualifications

8. Quality Improvement Supports in Parent Aware • Provider Resource Specialist – facilitates and oversees the process for programs • ERS Consultant • CLASS Coach • Financial assistance for QI expenditures

9. Who improves? Re-Ratings in Parent Aware

10. Ratings increase when programs are re-rated Initial Rating Second Rating N=97

11. Family child care providers make greater improvements than centers • Improvements by at least one star level at second rating: • 60% of centers • 70% of family child care providers • One-third of family child care providers increased by at least two star levels • 11% of centers decreased their rating by one star compared to 2% of centers

12. At re-rating, programs gain the most in Tracking Learning.

13. Programs make small gains on observed quality scores. • Points are earned in the Teaching Materials and Strategies category for ERS and CLASS scores • Programs gain about 1/3rd of a point on ERS scores. • Programs gain about 1/3rd of a point on Classroom Organization and nearly half a point on Emotional Support. • However, no gains are made on measures of Instructional Support.

14. How are improvements made? Quality improvement supports in Parent Aware

15. Provider Resource Specialists (PRS) • Seven PRSs reported on services to 83 programs (34 centers, 49 family child care) • Hours of direct contact • Range: 3 to 30 hours • Average: 8.2 hours • Average number of site visits per program – 3.6 • PRSs spend twice as many hours of direct contact with providers who are English Language Learners than English-speaking providers • 12.7 hours compared to 6.2 hours

16. Provider Resource Specialists (PRS) • PRSs spend more time with family child care providers than center-based programs • 9.9 hours compared to 6.3 hours • PRSs report that their most common activities with programs are: • Assembling the QRIS documentation packet • Preparing programs for their ERS observations • Picking out new materials or equipment for the program

17. Most common PRS activities

18. PRS activities vary by program need

19. ERS Consultants • Five ERS consultants reported on services to 77 programs (29 centers, 48 family child care) • Hours of consultation • Range: 2 to 40 hours • Average: 13.75 hours • Average number of site visits per program – 6.4 • ERS consultants spend more hours of consultation with family child care providers than with center-based programs • 15 hours compared to 12 hours • In centers, ERS consultants worked with 1, 2, or 3 classrooms

20. ERS Consultants • Hours of consultation differed for family child care providers who received different star levels • 2- or 3- stars – 16.9 hours • 4 stars – 10.6 hours • No difference in hours of consultation for centers that received different star levels • Most common activities that providers report doing with their ERS consultant • Understanding the ERS scoring system • Rearranging the physical space • Purchasing new materials • Learning hand washing and other sanitation procedures

21. ERS Consultants • ERS Consultants reported that programs were “very engaged and open to consultation” in 79% of the rating cycles • Family child care providers were slightly more engaged (85%) compared to center-based providers (68%) • Hours spent with “very engaged” programs – 15.6 hours • Hours spent with “not engaged” programs – 9.5 hours • No correlation between hours of consultation and ERS score for centers • Hours of consultation was negatively correlated with FCCERS-R scores for family child care providers

22. CLASS Coaches • Two CLASS coaches reported on services to 13 center-based programs • Hours of coaching • Range: 4 to 67 hours • Average: 23.2 hours • Average number of site visits per program – 8.8 • CLASS coaches worked with 1 or 2 classrooms per center

23. CLASS Coaches • Providers reported on the activities they did with their CLASS Coach • The most common activities were: • Observing me teach and giving me feedback – 76% • Helping me organize my classroom processes to aid children’s learning – 59% • Helping me understand the CLASS scales – 41% • Helping me understand the CLASS scoring – 35% • CLASS Coaches report than only 38% of providers are “very engaged”; 23% were “somewhat disengaged or hesitant”

24. QI Financial Supports • The average total expenditure per rating through June, 2011 was $2,791 • Average expenditure was significantly greater for family child care programs ($2,923) compared to centers ($2,604) • Expenditures per rating were classified into categories. • The largest portion of the expenditure was made on materials for the learning environment – books, toys, games (59%) • The second largest portion was on equipment for the learning environment – shelves, playground equipment (33%) • Training and consultation, teacher resource materials and assessment materials made up the remaining 8% of expenditures

25. What resources do providers say are “very helpful” to them? • Free training – 65% • Free curriculum materials -65% • Free assessment materials – 61% • Financial assistance – 61% • Provider Resource Specialist – 53% • ERS Consultant – 34% • Feedback reports from ERS – 36% • Publicity materials – 24%

26. Summary of Key Themes • Programs make significant improvements in their star level when they are re-rated • Programs make small but significant improvements on ECERS-R, ITERS-R and FCCERS-R • Programs make small improvements on the CLASS but not on the Instructional Support scale • Providers who are English Language Learners and family child care providers receive more hours of support from PRSs and ERS consultants • PRSs and ERS Consultants spend the majority of their time with providers on activities that do not involve interactions with children – paperwork, learning about the ERS, buying supplies and materials

27. Challenges for Improving QI • Developing models to work with providers who are not engaged – what works best? • Continuing to develop supports for high priority providers such as English Language Learners and family child care providers • Focusing site visits and consultation on activities that relate more directly to interactions with children • May require a change in the Quality Standards • Ensuring that financial supports are adequate to make “real” changes

28. Questions? • Please contact Kathryn Tout ktout@childtrends.org