1 / 38

Alexander Schulz & Nicolas Apostolopoulos Freie Universität Berlin

Alexander Schulz & Nicolas Apostolopoulos Freie Universität Berlin. 10 Years E- Exams at Freie Universität Berlin An Overview. FU E-Examinations. Structure. Development of the E- Examination Unit The E- Examination Center Topics of the E- Examination Unit Further information.

dfaust
Download Presentation

Alexander Schulz & Nicolas Apostolopoulos Freie Universität Berlin

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Alexander Schulz & Nicolas ApostolopoulosFreie Universität Berlin 10 Years E-Exams at Freie Universität Berlin An Overview

  2. FU E-Examinations Structure • Development ofthe E-Examination Unit • The E-Examination Center • Topics ofthe E-Examination Unit • Further information

  3. Step I: Blended-Learning: The Statistical Lab (2003-2006) • Project New Statistics (Neue Statistik) • Improvingthestatisticaleducationbyusing a didacticaltool • Goal: makinge-learningwholisticby • usingthe Statistical Lab (Statistiklabor) not onlyduringthesemesteras a learningtool, but • as an examinationtoolaswellin an open-bookscenario

  4. Step I: Prototypal E-Examinations in Statistics (2003-2006) • Focus News Magazin 12/2005 of 03.21.2005, p. 98

  5. StepII: Extendingthe Topic „E-Examinations“ (2007 – 2010) • Project FU E-Examinations- „Digital LectureandExamination Hall“ • Evaluation ofe-examsoftwaresolutions • Renovation oflecturehallsintolaptopexaminationrooms • Identificationofthelegal topics • Goal: • Remedyformassivelyincreasedamountofexaminationsresultingfromtheeuropean Bologna Processfor all courseofstudies not onlystatistics • Wholistic e-learninguptotheexamination • Makeuseofstudents‘ laptops (BYOD) • Creatingthebasisfor a universitywideinstitutionalization

  6. Step II: Central PC-Pools and Laptop Examination Halls (2008) 1. Capacities (2008) Total:300 seatsCeDiSLecture Halls:221 seats ZEDAT Lecture Halls:79 PCs 2. Additional Capacities Total:117 PCs ZEDAT PC-Pools: 117 PCs

  7. Step II: Laptop ExaminationHalls (2008) JK 28/11260 seats JK 27/1472 seats JK 27/10346 seats JK 27/10643 seats

  8. Step II: Laptop Examination Halls in Use (2008) Examination in Statistics (Summer Semester 2008)Prof. U. Rendtel & Dr. A. Ghosh

  9. StepII: UsingStudent‘s Notebooks in a Nutshell (2010) • Advantages of BYOD: • High flexibility (whenandwhere) • Studentsusetheirdevices • University: • Reducedhardwarecosts • fewer PC-pools required • BYOD examinationhallscan also serve a normal lecture hall purpose • Disadvantagesof BYOD: • Hardware heterogenityofdevicesleadstohigherdemandfortechsupport • Technical configrequiredtoensuresafetyfromcheating on a BYOD • In Germany: legal concernsfromthe non-discriminationprecept

  10. FU E-Examinations Structure • Development ofthe E-Examination Unit • The E-Examination Center • Topics ofthe E-Examination Unit • Further information

  11. Step III: Implementation, institutionalizationandextension (since 2011) • Technical implementation: • 2011-2012: Conceptionandinstallationof a high performanceclusterfortheexamplattform (with LPLUS) • Constructionofthe E-Examination Center (EEC): • 2012-2013: Setting upthe E-Examination Center (EEC) • Bringingintoservice on Feb 2nd, 2013 with an examoftheformer vice presidentofFreie Universität Berlin: Prof. Dr. Klaus Beck • More than 150 computerstationsforexams • Most modern examinationcenter in Germany • Second largestcapacityof all examinationcenters in Germany

  12. Step III: The E-Examination Center - 3D-simulations in November 2011 Source: Gewers & Pudewill Architekten Berlin

  13. Step III: The EEC in January 2012

  14. Step III: The EEC in April 2012

  15. Step III: The EEC in October 2012

  16. Step III: The EEC in December 2012

  17. Step III: The firstE-Examinations in February2013

  18. Step III: Frame Conditions (2011-2014) • Setting a Support-Team: • Since 2011: 1,5 x scientificemployees • Since 2013: 1,5 x scientificemployees + 2 students • Legal Basis: • Since 2013: Basis for E-Examinations isregulated in theuniversitywide „Rahmenstudienprüfungsordnung (RSPO)“ (~ Legal Framework for Studies and Examinations at Freie Universität Berlin) • Operational Concept: • Fixed budgetfortheunit (firstuntil 2016)

  19. Step III: The E-Examination Center in a Nutshell (2011-2014) • Advantages: • Infrastructure entirely in ourhands • ConfigurabilityoftheExam-PCs • HomogenityofExam-PCs • Low personal costsforadministration • Disadvantages: • Initially high investment • Lowerflexibilityofthe hall (configurationforlegallysafeexaminations) • Follow-upinvestmentsinevitable

  20. Step III: Current Ratio ofthe EEC - Single Tests (ST) Sincetheinitiationofthe EEC in February 2013, 168 examinationswereconductedwithmorethan15.700 singletests (~ participantsexamined) .

  21. FU E-Examinations Structure • Development ofthe E-Examination Unit • The E-Examination Center • Topics ofthe E-Examination Unit • Further informations

  22. Topics ofthe E-Examination Unit • Logistics • Technology • Didactics • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  23. Topics ofthe E-ExaminationUnit: Logistics • Logistics • Original Question 2007: Howto deal withgroupsofparticipantsofupto600 ppl? • Educational Science, • Economic Science • Political Science • Psychology • AlteredQuestion 2012: Howto deal withgroupsofparticipantsofupto3,000 ppl? • Medical Science • Technology • Didactics • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  24. Topics ofthe E-Examination Unit: Didactics • Intention? • Howshouldthelearningcontentsbeexamined? • Testingfactualknowledgeorapplyknowledgeto a case? • TypesofExams(seeCrisp 2009) • Diagnosticalexams • Formative exams • Summativeexams • Logistics • Technology • Didactics • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  25. Topics ofthe E-Examination Unit: Psychology • Examnervs • Reducingthemediadiscontinuity • Environment optimizedforconcentratedattention • Fundamental fears • Prejudice: technologyis not suffientlyreliable • Proceduresto deal withtheparticipants • Standardizedrhetoricalworkflowsfor • PC crashs • Evacuationofthebuilding in caseofemergency • Server Crashs (neverhappened) • Logistics • Technology • Didactics • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  26. Topics ofthe E-Examination Unit: Legal Implications • Examinationregulations: • „The electronic examinationcanthus not beconsideredas a type of a writtenexamination (..)“ (seeNiehus/Fischer 2010)„Die elektronische Prüfung ist in diesem Zusammenhang nicht als eine Art schriftliche Prüfung zu verstehen (..)“ • Every facultyhasits individual examinationregulation • Every federallandhasspecificeducationregulations • Data protection • Minimizationofpersonal data • Storingofexaminationdatafor3 semestersonly • Logistics • Technology • Didactics • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  27. Topics ofthe E-ExaminationUnit: Quality and Efficiency (1) • Quality assurance: • Statistical evaluationofexaminationresults • Macroscopic: comparisonofstudents/cohorts • Microscopic: comparisonofquestions (item-total correlation) • Question log reports (whenandhowlongdid a studentrun a specificquestion?) • Improvingefficiency • Saving time (mostly) bybeingabletoreadthestudents‘ answers • Examsaremuchfasterevaluatedandthusmarked • Whicheffectscanbemeasuredwhenmigratingtraditional paper-pencilexamalmost 1:1 tocomputerbased e-examinations? • Logistics • Technology • Didactics • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  28. Topics ofthe E-ExaminationUnit: Quality and Efficiency (2) Jurisprudence: SR questions • Logistics • Technology • Didactics * Sample = 10 exams • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  29. Topics ofthe E-ExaminationUnit: Quality and Efficiency (3) Educational science: CR questions • Logistics • Technology • Didactics * Sample = 10 exams • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  30. Topics ofthe E-ExaminationUnit: Quality and Efficiency (4) • Logistics Amountof e-exams • Technology • Didactics • Psychology • Quality • and Efficiency Examinationduration in minutes • Senken der Einstiegshürden Research question (break even):Howmany e-examsarerequired per yeartoamortize a BATIIaposition? • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  31. SR Exams: Percentagedhigherrationalization Effectivelylowerrationalization Profitable onlybydoingmassesofexaminations CR Exams: Percentagedlowerrationalization Effectivelyhigherrationalization Profitable evenwithloweramountsofexaminations Topics ofthe E-Examination Unit: Quality and Efficiency in a Nutshell (5) • Logistics • Technology • Didactics • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  32. Topics ofthe E-Examination Unit: Technology (1) • PCs: • Homogenous end userdevices (silent AIO Dell-PCs) • Standardized Windows 7 image (distributedfromacentralimagingserver) • Extra silent Cherry keyboards • Infrastructure andbasicsecurity • LAN connectiononly • Web access limited totheexaminationplatform • Logistics • Technology • Didactics • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  33. Topics ofthe E-Examination Unit: Technology (2) • Examinationplatform (LPLUS) • Examinationplatform • 1 x productionsystem (dedicated) • 1 x testsystem (virtualized) DB Server 03 – Witness • Logistics DB Server 01 – Primary DB Server 02 - Mirror • Technology • Didactics • Psychology • Quality • and Efficiency NLB • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy App Server 02 App Server 01

  34. Topics ofthe E-Examination Unit: Technology – ExaminationPlatform (1) • Logistics • Technology • Didactics • Psychology • Quality • and Efficiency • Manypossiblequestiontypesintegrated • Freie Universität ispredominantly in thefieldofsocialsciences • Thus: 70% oftheexamsareconstructedresponse design • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  35. Topics ofthe E-Examination Unit: Technology – ExaminationPlatform (2) • ReduceRiskofCheating • Randomizationofquestionorder • Randomizationofanswerorder • Create different examsforanystudentfroma groupof „equal“ questions. • Secure Browser • Detectsbeingrun a VM • Availablefor Microsoft Windows only • Not in use at Freie Universität at themoment • Stability • Permanent connectiontotheexaminationserver • SR questionssavedimmediately on server • QR questionssavedevery 30secs on server (everyrevisionsavedseparately) • Logistics • Technology • Didactics • Psychology • Quality • and Efficiency • Senken der Einstiegshürden • Gewährleisten von Systemsicherheit • Legal QuestionsandData Privacy

  36. FU E-Examinations Structure • The e-examinationcentre • Development ofthedivision e-examinations • Topics ofthedevision e-examinations • Further information

  37. Further Information E-Examinations at Freie Universität Berlinhttp://www.e-examinations.fu-berlin.de Crisp, G. (2009): „Interactive e-Assessment: movingbeyond multiple choicequestions“, Online im Internet: http://eli.elc.edu.sa/2009/content/Crisp%5Bresearch%5D.pdf(last accessed 05/18/2012) Schulz, A. & Apostolopoulos, N. (2011): „E-Examinations PutTo Test – Potenziale Computergestützer Prüfungen“, in: „Hamburger eLearning Magazin - #07 eAssessment auf dem Prüfstand“, Online im Internet: http://www.uni-hamburg.de/eLearning/eCommunity/Hamburger_eLearning_Magazin/eLearningMagazin_07.pdf (last accessed 05/18/2012) Niehus, N. & Fischer, E. (2010): „Prüfungsrecht - 5. Auflage“, C.H. Beck, München, S.12ff. „Mehr Spaß beim Statistiklernen“, erschienen in: Focus 12/2005, vom 21.03.2005, S.98, Online im Internet: http://www.statistiklabor.de/de/news/Presseartikel/Focus-12-05/index.html (last accessed 05/18/2005) Schulz, A. & Apostolopoulos, N. (2014). E-Examinations at a Glance. Die Computerisierung des Prüfungswesens an der Freien Universität Berlin. In T. Škerlak, H. Kaufmann & G. Bachmann (Hrsg.), Lernumgebungen an der Hochschule. Auf dem Weg zum Campus von morgen (S. 283-298). Medien in der Wissenschaft, 66. Münster/New York: Waxmann. Resources, Literatur andSources

  38. Thankyouforyourinterest Alexander Schulz & Nicolas Apostolopoulos www.e-examinations.fu-berlin.de

More Related