1 / 24

A University Strategy for Delivering Online Assessment

a. A University Strategy for Delivering Online Assessment. Dr Richard Parsons Director of the Learning Centre University of Dundee r.parsons@dundee.ac.uk. University of Dundee. More than 10 years substantial experience of online assessments

bono
Download Presentation

A University Strategy for Delivering Online Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. a A University Strategy for Delivering Online Assessment Dr Richard Parsons Director of the Learning Centre University of Dundee r.parsons@dundee.ac.uk

  2. University of Dundee • More than 10 years substantial experience of online assessments • Blackboard clients – Academic Suite (virtually 100% modules) • Delivering up to 3500 formative assessment sessions a day with QM Perception • Delivering 50+ summative exams a year with QM Perception • Range of online assessment activities recognised

  3. Blackboard Server Configuration Summer 2006 – Bb upgrade 6.3 to 7.1 on Unix, QM application from 3.4 to 4.1, Bb - QM bridge software (twice), QM repositories for assessments, Blackboard building block deployments and upgrades, Solaris and Oracle database upgrades, Annual module rollovers, Enhanced PDP programmes, Medical School curriculum map integration, University restructuring configuration changes

  4. Online Assessment System Assessment hardware design & deployment Assessment software design & deployment Clear policy and procedures Central assessment support Learning systems integration Staff and student education

  5. The Hardest Task Remains writing quality questions • Staff time is precious • Students are more critical than ever before • Assessments to be aligned • Questions need to be prepared ahead of time • Quality runs to feedback, distracters, variety, depth, currency • Authoring Q within the software is secondary

  6. Staff and Student Education • For 4 years at Dundee we have run an Online Assessment course, online • Covers technical and pedagogical aspects • 5 weeks long, 3 to 6 hours a week. • Formally accredited by our Education Faculty (10 SC) • Building a bank of staff skills • Most students need little guidance

  7. Examples of Assessment Strategies • 15 Q from pool of 100, max mark taken (Genetics) • Take assessment first, then repeat in an open fashion. Mark taken from first run. (Medicine) • 10 numerical questions, each from a pool of 50 randomly generated Q (Life Sciences) • Student revision – students invited to compose 2 Q each with FB, all compiled automatically, checked & made available for open assessment (Env Sci) • Open assessment, one submission only (Law) • Summative (range of departments)

  8. Quality Hardware and Systems • Central servers built & maintained by ITS • LC involved in system design • Three servers, one formative, one summative, one shared repository • HP BL20p Blades Dual Pentium 3400 MHz 6GB RAM, dual NIC backplane. • Databases • Formative on Oracle Unix server • Summative on Oracle Unix server • Shared on Oracle Unix server

  9. University of Dundee Questionmark Perception V4 Systems Model Participant Participant Oracle 10G Oracle 10G QMP V4.2 CAA-02 Production Formative Repository Participant QMP V4.2 CAA-01 Production Exam Repository Participant LC Administrator QM / Bb integration QMP V4 Authoring Manager Oracle 10G Oracle 10G Participant QMP V4.2 CAA-03 DevelopmentRepository Blackboard 7.1 My Dundee Local Author (Staff) Participant QMP V4 Authoring Manager Participant Local Author (Staff) QMP V4 Authoring Manager Local Author (Staff) David Walker, University of Dundee

  10. Clear Policy and Procedures • Central online assessment system required clear guidance • Prepared draft Policy and Procedures • Generally follows BS7988:2002 • Developed for about a year, inc. consultation • Passed at University Senate • Excellent for ensuring quality • Runs to 17 pages • Additional documentation for Online Assessment system administrators

  11. Summative Procedures • Assessment must be tested • Assessment loaded to both servers, scheduled on one • Central staff attend examinations • Use special ”exam” accounts • Use Perception Secure Browser • Candidates complete Examination Attendance form • Immediately after the exam a backup of results is taken • Use two way radios or mobiles • No system is fullproof, 99% success target ?

  12. Self & Peer Assessment Bb Building Block • Assessments are added and edited by the instructors in any course • One or more questions, each with one or more marking criteria • Marking criteria as discrete values or range • Feedback provided as score and text if required • Students automatically allocated to mark peers (1 to many) • Peer assessments are assigned automatically • Self assessment is an option • Marks and reports are visible to the Instructor • All marks can be downloaded for analysis in excel • Moderated results automatically passed to Bb Gradebook • Exercise text is available for download and online plagiarism checks • Questions and criteria are stored for reuse and exchange

  13. The Dundee My PDP resources Online resource to support PDP; timetabled induction sessions; scope for faculty tailoring.Available for free under license. Each student can prepare an online portfolio based on templates, culminating in prioritised action points and an enhanced CV. Portfolios can be shared online.

  14. Mobile Campus Map • Java game-based map • Rich links with key campus information • Single download • Popular with students www.dundee.ac.uk/mobilemap

  15. Additional Online Assessment • Self & Peer • Framework for plagiarism advice and management • Procedures for online submission • Discussion board assessments • ePortfolio assessment rubrics and tools • Classroom/personal response systems • Business intelligence activities • Overall framework of OA support at Dundee

  16. Individual concerns within an online assessment world Traditional Lecturer Who can electronically author my questions?Can we use textbook questions?Is this assessment driving learning? College Officer Can we book an IT suite for 240 students on Wednesday the 25th?Two students have additional requirements.Will the results transfer directly to the SRS? eLearning Manager Are the staffing levels correct?Is our software choice appropriate?Are the server specifications correct to cope with the expected load?Are staff education procedures in place? Lecturer Are my questions too simple?Is this technology reliable?Do the students find it tedious?Where can we obtain challenging questions?Are these assessments aligned with our learning outcomes? IT Desktop Administrator Does the desktop software operate correctly?Are the “exam accounts” secure and available?Are the PCs well maintained and reliable? Timetabling Officer Can the proposed exams be fitted into the best rooms?Are assessments receiving appropriate prioritisation? Student This is nice and privateI am getting real feedback hereDo I know the answers?Can I learn from this?I am at home in this online environment Distance Student Is my technology compatible?Is my connection reliable?Is this a replacement for F2F? IT Server Administrator Is the server fully patched, reliable and secure?Is the OA software functioning correctly and reliablyDo we have monitoring and is it working?Are the logging and backup procedures working correctly? Online Assessment Officer Are all assessments up-to-date and correctly available?Is the server-client functionality all compatible?Will the next examination run smoothly?Are the staff education programmes ready? Examinations Officer Are these results 100% reliable?secure?Do our procedures represent the best possible practice? Richard Parsons, University of Dundee

  17. Elements Actors Procedural Each addressed by a tasked individual(s) within a supported team, and each element able to cope with growth and changes Revisit annually? The entire operation has limited inertia, limited history, sustainable staffing, space, budget. Represents a framework OA policy and procedures ZZ + AA Student and staff training XX + WW Examination practices Academic staff + XX + WW Room booking procedures College staff Student personal PCs Web guidance + VV University student desktop BB + CC Question creation Academic staff + XX + WW Question/assessment repositories XX + WW Network DD + EE OA application software & support XX + WW 13 specialist staff + academics & college staff Technical database considerations FF + GG Operating system & web server XX + HH Hardware & infrastructure HH + II Technical Richard Parsons, University of Dundee

  18. The Annual Cycle of Online Assessment Sept. Testing & tuning Modules begin Online resit exams Annual frameworkmeeting Summer upgrade window Online exams June Dec. Mid semester window Online exams • Growth may occur each year • Changes are made to the systems • A framework allows the process to be sustained OA staff education module Richard Parsons, University of Dundee March

  19. Summary • Dundee has an institutional online assessment & some elements of a strategic framework • Software, hardware, P&P, staffing, integration and education. 13 specialist staff. • Secure and reliable, but straightforward to use for students and staff • Let the academics create the Qs to stimulate learning • Costs are neither insignificant or astronomical

More Related