Peer evaluation in further education colleges : shaping the quality agenda from within….

1. Peer evaluation in further education colleges : shaping the quality agenda from within…. Steve Cropper, Senior Librarian, Wirral Metropolitan College

2. What this session is about: • I am not here to sell you anything! • This is about an initiative which started in a very small way and has now grown beyond our expectations • I want to show you how it works at a practical level • The learning objective is that you go away from here sufficiently interested in peer evaluation to want to try it for yourselves

3. The Circle of Merseyside College Librarians: • Founded 27 years ago • Members from 6 major FE providers on Merseyside and two 6th form colleges • Strong links with North Wales and Cheshire • Lively, well-attended meetings each term • Annual benchmarking on key performance indicators

4. The Peer Evaluation Scheme • Has carried out 24 evaluations to date • In Lancashire: Blackpool and the Fylde College and Runshaw College • In Cheshire: Widnes 6th Form College and West Cheshire College • In North Wales: Denbigh and Rhyl Community Colleges and Coleg Llandrillo • Lancashire and North Wales have provided Evaluators for the scheme

5. The benefits of reciprocity: a broader perspective on quality • A knowledge of the Scottish FEFC standards and how they are applied. • Knowledge of the SLIC toolkit as used in Scotland. • Knowledge of the ESTYN standards as applied in Wales, and the fforwm toolkit.

6. How we began: A brief introduction to how peer evaluation began on Merseyside and why

7. The climate that gave rise to peer evaluation: • Many libraries struggling with inadequate resources and management indifference • An inspection regime for which we had high hopes delivered very little • Unqualified inspectors passing judgement on libraries • Inspection process too easy to subvert • The arrival of the dreaded SAR!

8. That was then, this is now: • The summer of 1999, “meltdown” arrives… • More businesslike, but how much has the underlying culture really changed? • Inspections in England and Scotland now consider libraries only in the context of the curriculum area inspections. • We are still being inspected by the unqualified. • Inspection process is still being stage-managed to death.

9. How do we respond and what we want to achieve? • To put libraries back on the inspection agenda, as central to the learning process • To counterbalance ill-informed judgements with qualified, professional assessment • To present inspectors and managers with thorough, detailed reports from well-qualified independent inspectors that are hard to dispute

10. Central to our thinking is: • That the professional knowledge and experience required to evaluate college libraries effectively, rests with those who are managing those services every day. • That keeping the process transparent gives it added legitimacy. • That keeping it free places it outside the realm of “consultancy” and available to all. • That it is not a resource-bidding exercise.

11. How we began to develop the scheme in practice: • Small working group set up consisting of representation from 4 member FE colleges and one 6th form college • Draft framework prepared • Emphasis on practical, real-world approach • Written up for CoFHE Bulletin • Moved forward quickly to first peer evaluation

12. The nitty gritty bit….. So much for the theory, how does peer evaluation actually work?

13. First: select your evaluators • Evaluations are initiated by a formal request to a Circle Meeting • Two volunteers are then requested who have no connection with the college to be inspected. • First-timers are always paired with an experienced evaluator • We also call upon colleagues from outside the area, with whom we have reciprocal links

14. Preparation for the visit: • Request advance documentation eg. SAR, Business plan • Make notes/draft questions arising out of the advance documentation • Set a realistic timetable for the day • Arrive at a division of labour • Make necessary travel arrangements

15. On the day: • Ensure we speak to a range of staff and to students • Strike a balance between interviews and legwork. Go and see for ourselves and be very hands-on • Stick to our own part of the brief and ask the right questions

16. Writing Up: • This is more time-consuming than you may think. Need to allow adequate time for drafting and re-drafting • Be prepared to re-check points of fact • Be helpful to the Librarian, but always tell the truth • Link to local and national standards

17. Peer evaluation and benchmarking : towards a synthesis • University sector has always had reliable benchmarks against which to assess services • FE sector until recently had only dated/over-ambitious guidelines • Three years ago, as a response to this CMCL began its own annual benchmarking round • Now augmented by realistic new CoFHE national guidelines

19. Example of Benchmarking Statistics On entering the library the overall impression on the ground floor is of a pleasant but rather Cramped space, dominated by a large counter area. The low ceiling height and requirement To keep access routes clear to fire exits reinforces this impression, and there is a need to Review the ration of staff to customer space, given that the library has, by the standard of Most colleges, generous back-of-house work space.

20. FEEDBACK: • Completed draft to the Librarian • Opportunity to correct matters of fact

21. Final Report: • Remember, it will be read by a wider audience • Remember to include an executive summary • Copies to: Librarian, Quality Manager, Principal, Circle archive

22. What those who have been through the process think: • “The peer evaluation provides an assessment of your service by fellow professionals greater in scope than other reviews or inspections, providing a substantial report useful for QA, external validations and service reviews” (Patrick Cox, Learning Resources Manager, Liverpool Community College) • “ A valuable and encouraging experience, with confidence that the assessors and the process are informed and supportive” (Catriona Martin, Librarian, West Cheshire College) • “The library has been inspected by those qualified to pass judgement and I welcomed the valuable report which has led to improvements” (Cathy Bennett, Librarian, St Helens College) • The Peer Evaluation report is a quality document produced by fellow professionals whose judgment I respect (Sue Haydock, Librarian, Southport College)

23. In conclusion, the peer evaluation process is: • A response to an inadequate inspection regime that did little to drive up quality • Appropriately rigorous, but also evaluative and supportive • Seeks to re-assert the knowledge and experience of library professionals in the inspection process • An achievable and practical means of assessing the strengths and weaknesses of college library services

24. What have we gained from all this? • We now have reliable, robust quality procedures that are standard across all Merseyside Colleges • Our activities in this area have given a focus to thinking about quality issues and raised our profile internally and externally • We have benefited from the process and we now know what standards we should aspire to.

25. Ten Years on, where next? • Evaluation framework continually revised to reflect changes in inspection regime • Evaluation is a continual quadrennial cycle, to show distance travelled • Towards a toolkit for English colleges?

26. email: steve.cropper@wmc.ac.ukFurther reading/References • Peer Evaluation : the Merseyside Experience. CoFHE Bulletin, Issue 99, Spring 2003 • Resources and Services Supporting Learning : a Service Quality Development Toolkit. SFEFC, November 2003, text available at: http://www.slainte.org.uk/Files/pdf/FEnet/toolkit03.pdf • EYNON, Andrew (ed.) Guidelines for Colleges : Recommendations for Learning Resources. London: Facet, 2005 • fforwm. Services Supporting Learning in Wales. fforwm, 2005.