Interpreting Feedback from Baseline Tests – Predictive Data

1. Peter Hendry: CEM Consultant Interpreting Feedback from Baseline Tests – Predictive Data Course: CEM Information Systems for Beginners and New Users Day 1 Session 3 Wednesday 17th October 2012 Peter.Hendry@cem.dur.ac.uk

2. The word ‘PREDICTION’: Quite probably the most contentious term that is used!! Concerns include: • A prediction for GCSE at start of year 7? • What has the baseline test got to do with my subject? • I know my pupils! • The predictions are too low: not valid!!! • And what about my professional judgement?

3. How is a ‘prediction’ generated? A* * ** *** ******************* A * ** *** ******************************** B * ** *** ********************************* *** ** * C * ** *** ************************************ *** ** * GRADE GRADE * ** *** ******************************** **** ** * * ** ***************************** *** ** * * ** *** ******************* ** * * * ** *********** ** * 50 100 150 BASELINE SCORE

4. 3 key points are: • The higher the baseline score the higher the final grade • Any one grade is achievable from a range of baseline scores • From any baseline score, a range of grades are possible

5. How is a ‘prediction’ generated? 50% on or above the trend line A* Subject National trend line (regression line) * ** *** ******************* A * ** *** ******************************** B * ** *** ********************************* *** ** * ‘PREDICTION’ (expected grade) C * ** *** ************************************ *** ** * GRADE GRADE * ** *** ******************************** **** ** * * ** ***************************** *** ** * * ** *** ******************* ** * * 50% on or below the trend line * ** *********** ** * BASELINE SCORE

7. ‘Predictions’…...are based on Average performance by similar pupils in past examinations The problem with the word ‘prediction’ is…? An alternative is ‘expected’ grade

8. Predictions 6.6 points = A/B Trend line 4 points = D

9. The graph below shows the middle 2/3 of some subject trend lines Comments?

10. Physics Maths Psychology Sociology Latin Photography English Lit Some Subjects are More Equal than Others…. A B >1 grade Grade C D E C B A A* Average GCSE

11. FACTORS THAT WILL INFLUENCE RELIABILITY OF PREDICTIONS: • Knowledge of student • Parental support/home life • Peer influences/social life • Student attitude, interest, language • Expectations of staff • Department/institution ethos • Resources • Quality of teaching and learning: pace of lessons • Understanding how children learn……… • And the reliability of the predictions......

12. Correlation = 1 Result

13. Correlation = 0 Correlation = 0.7

14. Point and grade ‘predictions’ to GCSE • Compare the predictions for English and Mathematics. • What pattern do you notice? • Look at Art and Design, Biology and French. Comments?

15. The graph below shows the middle 2/3 of some subject trend lines Comments?

16. Prediction/expected grade: 5.1 grade C Most likely grade

17. to place school at 75th percentile of VAD

19. Or insert own values and click adjust

22. Prediction/expected grade: 6.4 grade A/B Most likely grade

24. Independent Sector Prediction/expected grade: 5.9 grade C Most likely grade

26. Not a label for life...just another piece of information • The Chances graphs show that, from almost any baseline score, students come up with almost any grade - - -there are just different probabilities for each grade depending on the baseline score. • In working with students these graphs are more useful than a single predicted or target grade • Chances graphs show what can be achieved: • By students of similar ability • By students with lower baseline scores

27. Yellis predictive data: baseline score 103 (55%)

29. Student 1

30. Student 2

31. Why is the ‘predicted’ grade not always equal to the highest bar ? Predicted (‘expected’) grade Most likely grade AT WHICH POINT WILL THE SEE-SAW BE BALANCED? Predicted (‘expected’) grade i.e. the lower grades ‘pull’ the prediction to the left

32. Student 3

33. Student 4

34. Student 4 - IPR

36. Basing Targets on Prior VA – One Methodology from an Alis School • Discuss previous value added data with each HoD • Start with an agreed REALISTIC representative figure based, if available on previous (3 years ideally) of value added data • add to each pupil prediction, and convert to grade (i.e. in-built value added) • Discuss with students, using professional judgment and the chances graphs, adjust target grade • calculate the department’s target grades from the addition of individual pupil’s targets

38. Key Questions for Target Setting • What type of valid and reliable predictive data should be used to set the targets? • Should students be involved as part of the process (ownership, empowerment etc.)? • Should parents be informed of the process and outcome?

39. Key points to consider might include: • Where has the data come from? • What (reliable and relevant) data should we use? • Enabling colleagues to trust the data: Training (staff) • Communication with parents and students • Challenging, NOT Demoralising, students……. • Storage and retrieval of data • Consistency of understanding what the data means and does not mean