1 / 9

D12.1 Evaluation Plan

D12.1 Evaluation Plan. Intro. Process. Questions. Formative & Summative. Simulation Data. D12.3 (internal) - appendices to D12.1 This plan will be refined as the project progresses. Swiss test case. Julie-Ann Sime, CSALT Janet McCracken, (Uni. of British Columbia).

jesse-sanford
Download Presentation

D12.1 Evaluation Plan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. D12.1 Evaluation Plan Intro Process Questions Formative & Summative Simulation Data • D12.3 (internal) - appendices to D12.1 • This plan will be refined as the project progresses. Swiss test case Julie-Ann Sime, CSALT Janet McCracken, (Uni. of British Columbia) Portuguese test case Method Summative Evaluation

  2. How does the design of the prototypeshelp or hinder learning? Evaluation workshop - March 2001. All partner discussion. Evaluation plan v1. June 2001, (including partner input), v2. January 2002. Evaluation Process Intro Process Questions Formative & Summative Simulation Data Swiss test case Portuguese test case Method Summative Evaluation

  3. Examine Learners’ experiences. Identify problems with content and interface. To inform instructional design. Main Questions: Do the learners understand the visualrepresentations of complex concepts…? Are learning outcomes improved…? Evaluation Questions Intro Process Questions Formative & Summative Simulation Data Swiss test case Portuguese test case Method Summative Evaluation

  4. “Do the learners understand the visual representations ?” Evaluation of prototypes with real users. To assess learners’ experiences. “Are learner outcomes improved…?” Final evaluation at the end of the project. To assess effect on performance. To consider delivery e.g. if instructors satisfied with integration in curriculum. Formative & Summative Intro Process Questions Formative & Summative Simulation Data Swiss test case Portuguese test case Method Summative Evaluation

  5. An innovative feature of A-TEAM = dynamic modelling of emergency scenarios. Decision making involves use of complex data from multiple sources, interpretation of data is crucial. Do the learners interpret the data as we expect? Visual Representations Intro Process Questions Formative & Summative Simulation Data Swiss test case Portuguese test case Method Summative Evaluation

  6. “Commanders/emergency planners shall be able to take decisions about operational and tactical actions and planning and will use simple models (i.e. fire, explosion, toxic gas dispersion, smoke density in tunnels, etc.) to get important information i.e. about heat radiation distances, explosion damage zones, visibilitiy in tunnels, toxic gas concentrations, and so on, for scenarios.” Excerpt from D1.1 (Requirements Analysis) E.g.1. Swiss test case Intro Process Questions Formative & Summative Simulation Data Swiss test case Portuguese test case Method Summative Evaluation

  7. “..(some)..potential trainees…do not understand the hydrodynamic and environmental phenomena that occur during the spill and dispersion of contaminant. They are therefore unable to make any kind of forecast… the student will be required to interact with a simulation model evaluating the model results for different input data..”(Excerpt from D1.1 Requirements Analysis) E.g.2. Portuguese test case Intro Process Questions Formative & Summative Simulation Data Swiss test case Portuguese test case Method Summative Evaluation

  8. Small sample (4-6 learners/ test case). Pairs - for 2 hours each, tape recorded. Log of actions. Observations of behaviour. Pre and Post test. Evaluators - emergency management experts trained in evaluation. Method Intro Process Questions Formative & Summative Simulation Data Swiss test case Portuguese test case Method Summative Evaluation

  9. Focus on learner performance Pre and Post tests, Log of simulated scenario, Questionnaire - views on usability, acceptance, utility of course etc. Approx. 20 learners for 1 hour. Log examines decisions made and not deviation from a single correct path. Summative Evaluation Intro Process Questions Formative & Summative Simulation Data Swiss test case Portuguese test case Method Summative Evaluation

More Related