1 / 38

Improving Program Quality in Central Texas

Improving Program Quality in Central Texas. Agenda. Overview Accomplishments Improvements Future Opportunities Break Assessors Methods Trainers . Community Action Framework for Positive Youth Development. You Are Here!. (Gambone et al., 2004). Overview.

evers
Download Presentation

Improving Program Quality in Central Texas

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Program Quality in Central Texas

  2. Agenda • Overview • Accomplishments • Improvements • Future Opportunities • Break • Assessors • Methods Trainers

  3. Community Action Framework for Positive Youth Development You Are Here! (Gambone et al., 2004)

  4. Overview PLCProfessional Learning Community SAESystem Accountability Environment STEP 2a Self-assessment POSPoint Of Service STEP 1 Decide to build system STEP 3 Plan for improvement STEP 4 Carry out plan STEP 5 Measure change STEP 2b External assessment Program Self-Assessment External Assessment Observation-Reflection A la carte Methods Workshops TA Quality Coaching PQA BasicsPQA IntermediateAnchoring 4 Quality Matters presentation Planning with Data

  5. Plan Make choices Engagement Reflect Lead and mentor Be in small groups Partner with adults Experience belonging Interaction Encouragement Encouragement Reframing conflict Reframing conflict Skill building Skill building Session flow Session flow Supportive Environment Active engagement Active engagement Welcoming atmosphere Welcoming atmosphere Psychological and emotional safety Psychological and emotional safety Program space and furniture Program space and furniture Emergency procedures Emergency procedures Safe Environment Healthy food and drinks Healthy food and drinks Physically safe environment Physically safe environment What is Assessed?

  6. Alignment of Professional Development Opportunities Aligned Professional Development Courses

  7. Cycle of Improvement Start Here Over the Span of a Year

  8. Highlights from Palm Beach County, FL: Prime Time Initiative • Goal was to develop a comprehensive, county-wide system of supports and resources for OST programs • Overall Reported Strengths • Teamwork • Support • Opportunities for networking, program enhancements and training

  9. Prime Time (continued) • Specific Professional Development Strengths • Refined offerings to meet shared learning needs • Connect training to program improvement plans • Variety of modules • Quality of workshop leadership • Tools and ideas for program improvement* • Real changes • Better communication • Better activities • More youth involvement • Impetus to make time for change

  10. Quality Improvement System • QIS Process • Baseline Assessment • Quality Advisor Assignment • Letter of Recommendation for Improvement • Training • Self-assessment* • Program Improvement Plan* • Implementation • Reassessment

  11. Prime Time (continued) • Pilot Quality Improvement System • 40 baseline programs; 37 follow-up • Concerns about managing baseline assessments and how assessment would be used • How were assessment handled better? Unclear. Just says “they learned to do it better” • Internal/external assessment value became clearer to providers as it was used for quality improvement • Satisfaction increased with use and understanding

  12. Prime Time Lessons: It Takes Time! To collect baseline data and develop a quality improvement plan To form meaningful relationships with people in the improvement process/resource providers It takes initiative, time and energy for staff to reflect on work and implement new practices Also, the culture of expectations are as diverse as the participants and affect the process

  13. Prime Time Recommendations: Ensure sufficient communication about the process, especially for new participants Consider presenting score reports in person for new participants (by a knowledgeable party) Bring agency/site directors together to discussstrategies to best support the improvement process (talk to each other about what helps) Strengthen rapport between sites and improvement facilitators (advisors/coaches)

  14. Accomplishments

  15. Accomplishments • Educating Leaders – Quality Matters • Building Local Capacity • 27 trained self assessors • 19 trained external assessors • 6 methods trainers • Improving Program Quality • 3 methods trainings • 22 attended Planning with Data

  16. Comparison ScoresNational Average and Austin External ScoresNational Average:N= 735 offerings in 180 organizations Austin External Assessments: N=12 offerings

  17. Comparison ScoresAustin External and Self Assessment ScoresAustin External Assessment: N=12offeringsAustin Self Assessment: N=18 assessments

  18. Additional Data Notes • Engagement was significantly related to both supportive environment and interaction, but the relationship was strongest with interaction • Thus, programs that were stronger in interaction were more likely than any others to be strong in engagement

  19. Key factors for high engagement Opportunity to develop a sense of belonging Opportunities to act as group facilitators* Opportunity to partner with adults Support youth with encouragement Support youth with building new skills Support active engagement* Provide welcoming atmosphere * Explains 56% of variance in engagement

  20. Improvements

  21. Score Improvement Suggestions • Our goal is for each person to measure similar actions the same way each time we go to a location, regardless of where it is (reliability) - and for this action to be measured correctly (validity)

  22. More Suggestions • Provide supporting evidence • Scores can only be: 1, 3, 5 or X • Small groups (III-M) • (2) If no small groups are used, then “1” not NR (X) • (3) Quality of small groups • Conflict/Supportive Environment (II-K) • If not observed ask the follow-up questions • Evidence: no conflict or no procedure for conflict

  23. What else can be improved? • Table discussions • Report out

  24. FutureOpportunities

  25. Future Opportunities • Quality Coach • Quality Advisor • Youth PQA Trainers • Quality Matters Presentation • More Methods Trainings & Assessments

  26. Quality Coach • Prepares youth program professional to help youth works improve their program or the way they work with youth. • Supervisors or Managers • Network Consultants • September 2009 *CYPQ Quality Coaching Endorsement

  27. Quality Advisors • Prepares individuals to be network consultants to work with programs and staff throughout the quality improvement process. • October/November 2009

  28. Youth PQA Trainers • Prepares participants to train others in Youth PQA related workshops including: • Youth PQA Basics • Youth PQA Intermediate • Planning with Data • June/July 2009 *Participants must have extensive experience collecting PQA data, participated in Basics, Intermediate and Planning with Data trainings. External Assessor

  29. Methods Trainings & Assessments • 12 - month calendar • Increase in cost per training • $20 for programs involved in assessment process • $25 for anyone interested in the modules • More baseline assessments coming in April/May

  30. Quality Matters Presentation • Present recent findings from the YPQI-Setting Change Study, highlight the data and success of local QIS and facilitate a discussion about embedding the improvement work. • Decision-makers, stakeholders, funders and programs.

  31. New Cycle of Improvement Quality Advisor Start Here Youth PQA Trainer

  32. Professional Development and Quality Assessment 2009 JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Outcomes and Indicators, Youth Service Mapping - Compact Process Begins Quality Matters Presentation Quality Matters Presentation Professional Development Professional Development Professional Development Quality Advising Workshop1 Quality Coaching Workshop QI Overview Planning w/data Youth PQA Training Of Trainers Annual External Assessments External Assessor External Assessments Baselines continue Self Assessor Annual - Self Assessments

  33. What is next? short- term • March 31st • Ask, Listen and Encourage Module • Reframing Conflict • April / May – External Assessments • Method Schedule of Modules

  34. Break

  35. Score Improvement Suggestions • Our goal is for each person to measure similar actions the same way each time we go to a location, regardless of where it is (reliability) - and for this action to be measured correctly (validity)

  36. More Suggestions • Provide supporting evidence • Scores can only be: 1, 3, 5 or X • Small groups (III-M) • (2) If no small groups are used, then “1” not NR (X) • (3) Quality of small groups • Conflict/Supportive Environment (II-K) • If not observed ask the follow-up questions • Evidence: no conflict or no procedure for conflict

  37. Example: Supportive Environment(II-J. Staff Support) “Teacher asked many open-ended questions: ‘What’s happening in this poem?’ ‘What does it mean to rearrange the earth?’ ‘What are some of the other things these poems have in common?’ ‘How do you feel about this?’ ‘Anything you didn’t like or a weird word you would change?’”

  38. Example: Supportive Environment (II-G. Session Flow) “Teacher thoroughly explains writing exercise twice. A student comes late. Teacher has a student explain it to the late student and then follows up with additional directions.”

More Related