1 / 36

e-marking in large-scale, high stakes assessments conference themes :

e-marking in large-scale, high stakes assessments conference themes :. role of technology in assessments and teacher education use of assessments for continuous professional development assessments, teacher education, and school and systemic reform.

vanessalee
Download Presentation

e-marking in large-scale, high stakes assessments conference themes :

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. e-marking in large-scale, high stakes assessments conference themes: • role of technology in assessments and teacher education • use of assessments for continuous professional development • assessments, teacher education, and school and systemic reform

  2. large-scale, high stakes assessmentsthree questions: what do we mean by technology in this context how can we best assure assessment validity how can e-marking help get the best from current large-scale high-stakes assessment practices

  3. a. what do we mean by technology in this context?

  4. computer-based testing

  5. what do we mean by technology in this context innovativeitems … enabling students to show how they arrived at an answer and allowing them to score along a range of possible point values based on how well they understand a particular process.

  6. what do we mean by technology in this context Computer-adaptive tests …that better align with the performance levels of individual students by giving them more challenging or more manageable tasks according to how they perform in early questions.

  7. what do we mean by technology in this context intellectually ambitious assessments … through a combination of innovative items and computer adaptive testing to measure the learning that is difficult to assess in pencil-and-paper tests.

  8. in the context of large-scale, high-stakes assessments current issues are: hardware availability, compatibility and consistency high speed connectivity equality of access…

  9. in the context of large-scale, high-stakes assessments cost…

  10. how can we best assure assessment validity? b.

  11. how can we best assure assessment validity the greatest threat to validity for performance assessments may be construct under-representation — i.e. where the assessment is too narrow or fails to include important aspects of a construct.

  12. how can we best assure assessment validity often, the outcome of large-scale assessments is narrow – a pass or fail, a grade; a raw or standardized number; a percentage.

  13. how can we best assure assessment validity The greatest threat to validity for performance assessments may be construct under-representation — where the assessment is too narrow or fails to include important aspects of a construct. we need help to create good test designs we need pre-tested performance information

  14. how can we best assure assessment validity often, the outcome of large-scale assessments is narrow – a pass or fail, a grade; a raw or standardized number; a percentage we need detailed, more useful, information about performance on different aspects of the tests

  15. how can e-marking help? c.

  16. e-marking not like this!

  17. e-marking

  18. e-marking

  19. e-marking: efficient, secure, reliable…. images of student responses captured by scanning scripts are securely stored images are automatically distributed to appropriate markers markers log on to receive and score images on line, on screen, at item level. supervisors track marking progress and quality in real-time e-marking

  20. e-marking

  21. e-marking: security, accuracy, speed • Each year: • Pearson marks over 60 million exams in the USA and more than 7.2 million national tests and 9 million exams in the UK • Some exams taken in the England are marked in Australia Pearson designs and delivers national exam processing systems, including e-marking, for programmes as diverse as national testing in Abu Dhabi to State and national assessment programmes across Australia Results are delivered quicker and in more detail than ever before

  22. Key marking benefits… increased efficiency – removes majority of manual handling of scripts reduced marking time – concurrent marking, image distribution, no remarking, mark as scanned reduced risk, secure scripts, secure images increased control and flexibility increased marking quality fairer outcomes for students Item-level results data capture e-marking

  23. Item-level results analysis and reporting

  24. information to help desig good tests Automated analysis of e-marking pre-test results can inform test design and test validity: • item-level facility values • item performance • bias factors • overall test performance analysis

  25. Item performance characteristics A well performing item: The probability of a student selecting the correct answer increases as the student’s overall test performance increases Facility 0.76

  26. Item performance characteristics A poor performing item Success is not dependent on the students, overall test performance The item does not differentiate between a low and high ability student Facility 0.29

  27. Multiple choice question distractor analysis

  28. information to support school improvement and raise standards of student achievement • e-marking provides a level of accuracy and detail about student performance never before available • automated results analysis can directly benefit: ministry and departments of education to define policy schools and teachers to identify good practice and areas for development in teaching students and parents to take control of learning • …helping to raise standards

  29. information contributing to professional development • helping teachers to address such questions as: how are we doing as a school how are we doing compared with other schools like ours how are we doing compared with last year how are we doing each subject how are we doing in each class how are we doing on each paper; on each question and is each student doing.

  30. ResultsPlus | Analysis helping departments

  31. ResultsPlus | Analysis helping centres

  32. ResultsPlus | Analysis helping teachers

  33. ResultsPlus | Skills

  34. Summary role of technology in educational measurement and teacher education existing e-marking technology offers more from paper-based assessments to educate teachers about where their students are at.. use of educational measurement for continuous professional development ..leading to the sharing of professional expertise and shifting classroom expectations. educational measurement, teacher education, and school and systemic reform An emphasis on teacher education combined with quality assured assessments will profoundly affect the efficacy of classroom interactions and raise standards in education.

  35. Thankyou

More Related