1 / 7

A Level Qualifications

Setting and maintaining standards: criteria, compensation and statistics Ian Stockford Head of Research. A Level Qualifications. Compensatory v Mastery

dai
Download Presentation

A Level Qualifications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Setting and maintaining standards: criteria, compensation and statisticsIan StockfordHead of Research

  2. A Level Qualifications Compensatory v Mastery • No strong criteria candidates must meet. Candidates do not necessarily need to demonstrate mastery across all areas of the specification • Candidates perform in unpredictable and varied ways • Specific claims that “if a candidate got a grade A it means they can…” are problematic The purpose of A-levels In relation to university selection, A levels must constitute more than a simple means of ranking students to help university admissions authorities choose between applicants. They must also ensure that students have acquired any specific skills and knowledge that they need in order to embark on their chosen degree course or profession. Ofqual, A-level reform consultation, 2013

  3. The awarding process Qualitative examiner judgement • All judgements are made at grades A and E only • Examiners scrutinise scripts that fall on and around the judgemental grade boundaries suggested by the statistical evidence • Reference to archived scripts and grade descriptors Challenges to qualitative judgement • Imbalanced scripts • Variations in demand Good and CresswellBritish Educational Research Journal, 14(3), 263-28, 1988 • Benefit of the doubt to the candidates • Various other contextual/bias effects • Not a reflection of the capability/professionalism of examiners!

  4. Statistical Evidence Support provided to guide the judgements of examiners Predictive used: • Prior attainment of candidates – based on mean GCSE • Common centres analyses • Item Response Theory based test equating Alignment of standards: • Between awarding bodies (common basis of prediction) • Between specifications (intra-awarding body) • Over time (prediction generated from previous outcomes) • Regulated based primarily on statistical evidence Potentially vital in managing transitions between assessment structures

  5. Changing times… Removal of the January examinations series • Essentially removes the opportunity for resitting at A2 What does resitting provide? • Provides the opportunity for candidates who haven’t done themselves justice in their first sitting to better represent their true ability • Provides the opportunity for candidates to opportunistically sample assessment noise inherent when sampling a wide domain Wheadon and Stockford, Ofqual Reliability Programme, 2010 Candidates certificating in summer 2014 have had the advantage of being able to resit removed • Potentially disadvantaged by the happenstance of the timing of their birth Maintain performance standards or maintain outcomes…?

  6. Managing the risk Maintain Performance Standards Maintain (Contextualised) Outcomes • Seemingly fits with the purpose of ensuring that students have the appropriate skills • Retains the integrity of the A-level in terms of demonstrated performance • Impact on outcomes is unknown • Different impact across different subjects • Knock-on consequences for HE difficult to predict • Relies heavily on judgement • Protects candidates against disadvantage introduced by a change in the ‘rules’ • Realises a rank order of proficiency for selection purposes with a predictable number of candidates receiving each grade • Performance standards will likely need to be lowered to maintain outcomes • Different across subjects • Publically defensible?

  7. In summary… The grading of A-levels is not reliant of the ability of candidates to meet a set of performance criteria Grade boundary setting using a combination of judgement and statistics although the challenges facing the use of judgement are significant Whilst is has a role, the process of setting grade boundaries (and therefore controlling outcomes) is not an appropriate mechanism by which to drive increased competency/knowledge/skills of students …this must be addressed over a longer period through increased content standards driving teaching and learning Frequent reforms to the structure of qualifications drives in risk to the system. Comms between awarding organisations, the regulator, HE and the wider public are vital.

More Related