1 / 13

REAQ

REAQ. Report on E-Assessment Quality A JISC project in eLearning. Lester Gilbert Gary Wills Bill Warburton Veronica Gale. March 2009. The big picture. Report process. The team. Management group Lester Gilbert, ECS, University of Southampton , PI

frisco
Download Presentation

REAQ

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. REAQ Report on E-Assessment Quality A JISC project in eLearning Lester Gilbert Gary Wills Bill Warburton Veronica Gale March 2009

  2. The big picture

  3. Report process

  4. The team • Management group • Lester Gilbert, ECS, University of Southampton , PI • Dr Gary Wills, ECS, University of Southampton • Expert consultants group • Cliff Beevers, Heriot-Watt • Paul Booth, Question Tools • John Kleeman & Greg Pope, Questionmark • Harvey Mellor, IoE • Chris Ricketts, Plymouth • Denise Whitelock, OU • The workers • Consultant researcher (Veronica Gale) • Bill Warburton, iSolutions, University of Southampton

  5. Interviewees • HEIs • Heriot-Watt • University of Southampton • Newcastle University • University of Plymouth • The Open University • Edinburgh University • Institute of Education • Cambridge Assessment • SQA • Question Tools (Network Rail) • VrijeUniversiteit Amsterdam

  6. Questions asked (1) • What denotes ‘high quality’ in summative e-assessment? • What steps do you follow to create and use summative e-assessment? • How do you ensure e-assessment: • reliability, • validity, • security, and • accessibility? • How does the process of creating good quality e-assessment differ from the process of creating traditional assessment?

  7. Questions asked (2) • Please give us examples of good e-assessment; why are these ‘good’? • When you have heard of poor e-assessment, what has made it ‘poor’? • What feedback have you received from students who have taken e-assessments? • What advice would you give to others using summative e-assessment? • What research or other work has informed your thinking about summative e-assessment? • What further research would you like to see conducted?

  8. Expectations… We thought the interviewees would talk about: … psychometric measures… … intended learning outcomes … … appropriate standards … … delivery issues… … and if we were lucky, capability maturity …

  9. We heard a lot about… Delivery issues: infrastructure, support, and how things go wrong …

  10. But not much about… Point biserials, Cronbach Alphas, Kuder-Richardsons Practice standards Content validity, Conformance to ILOs Metrics, Capability maturity

  11. And when we did hear… Difficulty coefficients / facility values were often inappropriately used…

  12. Possible recommendations • Toolkit for end-to-end e-assessment creation, delivery, and evaluation • Guidelines & workshops: • Psychometrics • Validity through ILOs • Capability maturity • Research on the psychometrics of item banks • Research on quality metrics for multi-staged and stepped constructed response questions • Exemplars repository

  13. Thanks! Questions, comments, discussion…

More Related