1 / 16

E-Assessment: Evaluation, vision and plan

E-Assessment: Evaluation, vision and plan. www.jisc.ac.uk John Winkley, e-assessment JISC E-learning Programme. Agenda. A vision for e-assessment Findings from our evaluation Recent e-assessment activity in the e-learning programme A shift of emphasis.

thane
Download Presentation

E-Assessment: Evaluation, vision and plan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. E-Assessment: Evaluation, vision and plan www.jisc.ac.uk John Winkley, e-assessment JISC E-learning Programme

  2. Agenda • A vision for e-assessment • Findings from our evaluation • Recent e-assessment activity in the e-learning programme • A shift of emphasis

  3. JISC – The structure of the e-learning programme

  4. A draft vision for e-assessment (by 2015) • E-assessment is used appropriately and with confidence … in a wide variety of HE curricula. • E-assessment … adds to traditional assessment processes with efficiency improvements, as well as improvements in assessment quality... • E-assessment is used at all stages of the learning programme… • Information from different types of assessment is interoperable... • The day-to-day use of e-assessment is encouraging change in the nature of assessment... E-assessment is embedded in institutional policy… • A wide range of technologies are used to support e-assessment in diverse forms… • Developments in JISC’s infrastructure projects… provide a development platform which allows new technical and pedagogical assessment opportunities to be researched, developed, piloted and exploited efficiently. • A vibrant world-class community of research underpins technical and pedagogical developments in e-assessment….

  5. e-learning programme – activity types • Investigates - Institutional-level demonstrators/pilots • Describes - Research, studies, mapping and modelling • Analyses - Review and Evaluate • Creates - Commission Development • Informs - Consult and Inform (Knowledge Exchange ) • Guides - Communication and Dissemination • Project and Programme Support, Programme Management • Recent move toward larger pedagogy-focused programmes with “cross-cutting themes”, eg OER, Curriculum Design and Delivery.

  6. eAssessment - Baselining • Programme Management • Underpinning Research • Technology • Content • Embedding and Rollout

  7. Programme Management • TODAY • Vibrant but small and specialised. • Too many technologists, not enough educationalists. • Recent efforts to get the two sides talking appear to be working. • Not enough vendors. Slight reluctance from academics. • Hard to maintain but important. Complex education sector, many agencies at the intersection of technology, pedagogy and exams. VISION • Established vibrant, open and mixed community of technical developers, educators and assessment specialists working to form, support and develop JISC’s vision for e-assessment. • Vigorous stakeholder engagement

  8. Underpinning Research • TODAY • Anecdotal evidence: • Much adoption, but limited evidence of widescale use of e-assessment in HE. Different elsewhere. • Relies on early adopters and enthusiasts. Poor scaling. • large gap behind the ‘bleeding edge’ and the day-to-day life of departments. • Policy and decision makers are critical. • Ok for Int’l HE, little wider view. VISION • Regular measurement of progress with embedding. • A clear understanding of institutional barriers to take-up and the best methods to fix. • Clear understanding of international best practice.

  9. eAssessment - Underpinning Research (2) • TODAY • Patchy. Importance very clear. • E-assessment generally not in HE policy and strategy. • Leading role in IMS. QTIv2 thoroughly explored. Killer business case not found yet. VISION • Technical research underpinned by clear pedagogy. • Alignment of best practice between e-assessment and assessment. • Successful tracking, influencing and adoption of interoperability standards.

  10. eAssessment - Technology • TODAY • Successful OSS projects: Reference player, item bank, item dev kit, peer review tools. But often poor takeup and sustainability. • Excellent proof of SOA and OSS in practice. • Needs extending for workflow and enterprise applications. • Human processes • “System” integration for information sharing • Struggling to make e-framework work for practical e-assessment applications. VISION • Successful group of OSS toolkit and demonstrator projects provide inspiration and seedcorn for larger developments. • Technical framework

  11. eAssessment - Technology (2) • TODAY • Formative e-assessment pedagogy not quite there, but big strides made recently toward a clear framework to support development. • Wide range of tools and processes under development in a wide variety of settings. VISION • E-assessment embedded in formative assessment, supported by range of OSS tools and components

  12. eAssessment - Content • TODAY • The QTI1.x and 2.x specifications are reasonably stable. Untried on large scale in UK. Attempts within FE and schools sector have had mixed success. • The two “killer applications” of content interoperability : avoidance of lock-in and content sharing both require a more widespread use of e-assessment than currently seen. • Some doubt about whether HEIs really wish to share content. (Also some examples of success where external QA is important). • It isn’t yet, in most HEIs. The tools are not yet ready to support it. VISION • Content Interoperability is supported, removing concerns about vendor lock-in and allowing sharing of content. • Content management of assessment content is well understood.

  13. eAssessment – Rollout, embedding • TODAY • Significant shortcomings and challenges faced by HEIs. E-assessment commonly run by small groups of “enthusiasts” who spend much of their effort on maintaining a reliable service rather than on managing assessment performance. • Yes, where its use is driven from pedagogy. • E-assessment appears to have a low profile within QAA and HE institutional policies (not fully investigated). • Communities of subject specialists appear to provide the richest community for successful takeup. VISION • HEIs have a clear understanding of how best to measure and manage quality in e-assessment systems including at policy level. • There is clear evidence of the benefits of e-assessment. • E-Assessment is widely used in HEIsand embedded in institutional and sectoral policy: “crossed the chasm”. • Content management of assessment content is well understood.

  14. eAssessment – Rollout, embedding (2) • TODAY • Technically possible but difficult to organise. • Empirical benefits very clear. Some examples of good practice. • Likely to be a major focus of next phase of JISC e-assessment work. VISION • Information from assessment including e-assessment is used effectively within learning programmes. Information from different types of assessment is interoperable, allowing it to be combined seamlessly, to provide a rich and dynamic picture of learning to date and planning for future learning, supporting personalised learning programmes.

  15. Focus of recent activities • Research to support development of a more diverse set of e-assessment approaches and technologies – “technology-enabled assessments” • Investment in formative e-assessment development • Research to investigate quality processes around e-assessment • Investment in e-assessment as a more integrated part of the ‘learning environment’ • Community development and engagement

  16. E-Assessment: Evaluation, vision and plan www.jisc.ac.uk John Winkley, e-assessment JISC E-learning Programme

More Related