Quality Framework Initiative for Youthreach and STTCs: Embedding a Culture of Evaluation

1. Quality Framework Initiativefor Youthreach and Senior Traveller Training Centres Embedding a Culture of Evaluation October 4th 2007 Ms. Shivaun O' Brien, QFI National Co-ordinator

2. Why a QFI for Youthreach and STTCs? • Demand from the centres (Youthreach 2000) • Education Act (1998) - Centres were designated as Centres for Education in 2004 • YR/ STTCs both work with children, ie. under 18, also sixteen year olds/ those not completed 3 yrs post primary school • DES responsible for the quality of education provided in Centres for Education as for schools • Schools have SDPI • YR/ STTC have QFI • SDPI and QFI similar in process and resources Ms. Shivaun O' Brien, QFI National Co-ordinator

3. Development of the Quality Framework Initiative- Timeframe Ms. Shivaun O' Brien, QFI National Co-ordinator

4. The Quality Framework QUALITY FRAMEWORK Ms. Shivaun O' Brien, QFI National Co-ordinator

5. Quality Standards – 27 Quality Areas

6. How ICE happens (practicalities) • ICE session takes place over two days • Centre closed • Some preparation required in advance • QFI Facilitator booked in advance/ TES rates • Structure of session prescribed/ guidelines • Important that each activity is useful/ time well spent • Facilitator types up outcomes of the session • Centre Co-ordinator/ Director prepares annual report for VEC following session • Check implementation of actions from year to year Ms. Shivaun O' Brien, QFI National Co-ordinator

7. What can be evaluated? Any of the 27 quality areas – select 9 areas to start Specific Programmes/ activities of the centre Implementation of the Centre Plan Combination of these Ms. Shivaun O' Brien, QFI National Co-ordinator

8. Who is involved? Core people(attend 2 days) Centre Co-Ordinator/ Director Centre Staff incl. admin Key - but involved to varying degrees Learners (evaluation workshops & questionnaires) Regional Co-Ordinator Adult Education Officer (AEO) Education Officer (EO) Chief Executive Officer (CEO) Board of Management Reps Involve where relevant(questionnaires/interview/ focus groups) Community Reps. e.g if evaluating links with community Parents/ Employers e.g. if contact with parents is being evaluated May attend part of 2 day session or complete questionnaire/ interview in advance

9. Internal Centre Evaluation Internal Centre Evaluation Ms. Shivaun O' Brien, QFI National Co-ordinator

10. Purpose of the ICE Primary purpose: • To examine centre practice, identify strengths and weaknesses, identify & implement actions to improve the service. • Accountability: annual reports to VEC, evidence for DES inspectors But is that all? Ms. Shivaun O' Brien, QFI National Co-ordinator

11. Evaluation can have process outcomes “Process refers to and is indicated by individual changes in thinking and behaviour and programme or organisational changes in procedures and culture, that occur among those involved in evaluation as a result of the learning that occurs during the evaluation process” Michael Quinn Patten (1993) Utilization Focused Evaluation UFE :Concern with the usefulness of the evaluation Ms. Shivaun O' Brien, QFI National Co-ordinator

12. Process Outcomes • Enhancing shared understanding ( NOT teachers teach and managers manage), staff concerned with overall service not just the subjects they teach • Teambuilding - regularity NB • Capacity Building – more capable more confident • Increasing staff engagement, sense of ownership and self determination ( participatory and empowerment evaluation) • Increased intentionality – end up with plan, a sense of direction, commitment to making progress Ms. Shivaun O' Brien, QFI National Co-ordinator

13. Building in process outcomes • Bring staff together • Clear communication – facilitator clarifies and challenges • Everyone talks/ listens – equally valued • See the service through each others’ eyes – opportunities for personal reflection • Safety – speaking openly, evaluation of systems not people, controlled & kept focused • Learning- new knowledge and skills: from gathering and discussing evidence, learning to solve problems, learning to evaluate (double loop learning), becomes integral to programme • Making decisions - deciding on actions that are actually carried out Ms. Shivaun O' Brien, QFI National Co-ordinator

14. Evidence Evidence can be Quantitative or Qualitative Lists of evidence pg 23-26 ICE guidelines Ms. Shivaun O' Brien, QFI National Co-ordinator

15. Evaluation by learners Resources See QFI Guidelines for Internal Centre Evaluation Chapter 3 ICE guidelines- reflect current good practice www.youthreach.ie www.sttc.ie Hearing Young Voices www.youth.ie Participation Spice it Up Dynamix UK Explain the reasons for evaluation to learners Approach depends on learners age, ability Decide the focus of the learner evaluation Plan session/s EVALUATION SHOULD BE A POSITIVE and USEFUL EXPERIENCE FOR LEARNERS Ms. Shivaun O' Brien, QFI National Co-ordinator

16. Evaluation by learners ctd. How will learners receive feedback Important to show learners that their opinion was considered and that appropriate action was taken Evaluation (reviews) of programmes by learners should happen every time a programme is delivered- feed into FETAC QA requirement SEE HANDOUTS Ms. Shivaun O' Brien, QFI National Co-ordinator

17. Follow-up to ICE Session Facilitator will type up outcomes of 2 day session and return it to centre Co-ordinator/Director Staff and VEC mgt get copy of evaluation report Ensure that action plan is implemented Keep action plan on agenda at staff meetings Co-ordinator/ Director responsible for ensuring that actions are implemented/ time allocated ICE following year – check progress Ms. Shivaun O' Brien, QFI National Co-ordinator

18. Contact: Shivaun O’Brien Quality Framework Co-Ordinator C/O Co Louth VEC King St Drogheda Co. Louth 041-9839930 shivaunobrien@eircom.net For QFI Resources www.youthreach.ie www.sttc.ie Ms. Shivaun O' Brien, QFI National Co-ordinator