Assessment and Monitoring

1. Assessment and Monitoring Christine.Merrell@cem.dur.ac.uk www.cemcentre.org

2. Centre for Evaluation and Monitoring (CEM) • Part of Durham University • Monitoring Systems • 1.1 million assessments delivered each year • Pupils aged 3 – 18 years • CEM systems used in 44 countries

3. Why assess? • Newton (2007) 18 purposes for which educational judgements may be used • Not exhaustive • Expanding all the time • Interesting perspective on the historical debate about formative and summative assessment

4. Why assess? • From birth ~ weight, hearing etc. • Qualifications ~ driving test, GCSEs, degree • Profile of strengths and weaknesses for planning appropriate learning experiences • Indicator of special educational needs • Monitor progress and attitudes of pupils and cohorts over time

5. Comparisons • Children within a class • Groups such as boys/girls • Classes within a year-group • Current cohorts with previous ones • Other schools within a consortium and nationally • Progress over time • Research • Within school • Nationally and internationally

6. Layers of information (different levels of detail): • Diagnostic at pupil-level • Group and class trends • School-level information (including trends over time) • Consortium/Authority-level

7. Do you know the psychometric properties of the assessments that you use? Are these methods always reliable and valid? Early Years Foundation Stage Profile? Age 2 – 3 progress check Year 1 phonics check Ways to Assess and Cautions • Judgements • Observations • E.g. Behaviour • Objective tests • E.g. Attainment • Practical • Pencil and paper • Computer-delivered

8. Standardised Assessment • Administration procedure • Comparison against representative norms

9. Example of a high-level use of standardised assessment • Standards over time • Consistent content • Consistent sample of schools Merrell, C. and Tymms, P. (2011) Changes in Children’s Cognitive Development at the Start of School in England 2001 – 2008, Oxford Review of Education, Vol. 37 (3), June 2011, p333 – 345.

10. Initiatives in the Early Years • Foundation Stage • Age 3 – 5 years • Increased nursery provision • Curriculum • Assessment • Sure Start • Education Action Zones • Etc. Etc. Have these initiatives changed children’s cognitive development at the start of school?

11. PIPS On-entryBaseline Assessment • Computer delivered assessment of: • Vocabulary • Phonological awareness & early reading • Early mathematics • Additional data: • Date of birth • Sex • First Language

12. Test/Re-test Reliability = 0.98 • Internal Reliability (Cronbach’s alpha) = 0.94 • Correlation with attainment at age 11 = 0.68

13. 472 State Schools in England Sample

14. Background Variables

15. Mean PIPS T Scores

16. Mean PIPS Raw Scores Effect Sizes 0.11 -0.07

17. GLMs to analyse changes in BLA scores in relation to: • Year • EAL • Sex • Age at test was entered as covariate Main Effects & Interactions

18. Conclusions • Statistically significant decrease from 2001 to 2008 for early reading and picture vocabulary. However, the effect sizes of the differences were small. • Significant increase in early maths scores although again the effect size was small.

19. Why Should the BLA Scores Remain So Stable Over Time? • Success from early interventions aimed at influencing the development of young children is difficult to achieve • Reduction in efficacy of small-scale programmes when rolled out • Limitations with data analysed but nevertheless this is a large dataset that adds to current studies of trends over time

20. Thank you Christine.Merrell@cem.dur.ac.uk