Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
What Is Evaluation? Per Michael Scriven (1991): • “Process of determining the merit, worth, or value of something” • “(Process that) leads to evaluative conclusions” • good/bad • works/ doesn’t work • high/low quality
Course Evaluation • Is the course working as intended? • Does the course meet its objectives? • Is the online learning environment as effective as it can be? • Is the course manageable? • Are learners satisfied?
Why Evaluate? • To improve instructional quality • To motivate learners • To motivate instructors *Scriven, M. (1991). Evaluation Thesaurus, 4th Ed. Sage Publications: London
Conducted by Administration vs. Conducted by Instructor • Administration-Led • Focuses on learner satisfaction & broader issues of program • Access, Course Management System • University service – registration, instructor • Minimal instructor feedback • Instructor-Led • Focuses on course components/design • Student feedback & performance • Instructor reflection
Needs. Anal. Revise Per Obj. Inst. Strat. Inst. Mtls. Form Eval. Assmt. Instr. Goals Lnr. & Context Summ Eval. Dick and Carey Design Model
Evaluate Perform. Assess Entry Behavior Analyze Feedback Gerlach and Ely Design Model Determine Strategies Specify Content Org. Groups Allocate Time Specify Object. Allocate Space Select Resources
In Reality, Evaluation is…. Done continuously once the course begins.
Recommendation Formativecourse evaluation conducted by the Instructor.
It needs to be… Informal and Reflective, as opposed toReactive.
Interim vs. Terminal • Interim • Evaluating at various points in the course • Not TOO many points • 2-3 times per semester, mid-term at the minimum • May affect future weeks of course • Terminal • Conducted at the end of the course • Affects next course offering • May get more input on interim evaluations than terminal evaluation
AEIOU – Course Evaluation(Based on AEIOU Program Evaluation Method) • A – Access – Was the course easily and consistently accessible? • E – Effectiveness - How well was the course designed/organized? Objectives & instructions clear? Assessment matched objectives? Level of student performance? • I – Interaction – Was there a good balance of interaction? Level of learner participation in discussions? Level of peer support, instructor support? • O – Obstacles – What hindered the potential effectiveness of the course? • U – Unanticipated Consequences – What happened that was not expected --- both from learner and instructor perspective?
Evaluation Considerations • Environment • Course structure • Access reliability • Instructional Strategy • Objectives • Activities • Interaction/Communication • Assessment
Learner Performance Motivation Satisfaction Instructor Performance Motivation Satisfaction Evaluation Considerations
Better to Design the Quality In Than to evaluate the problems out
Communication Tools for Evaluation • Email • Survey attached • Quick/one question surveys • Online surveys • Quick feedback on a new technique – “What did you think? Check the box..” • Discussion board (evaluation summaries) • Fax • For ‘handwriters’
Encouraging Completion – Interim • Make it as short/easy as possible • Students need to trust interim won’t affect their grade – stay away from instructor components (i.e. feedback) • Evals could be submitted to peer/TA who summarizes and forwards to instructor or posts on course site • Could be incremental questioning • Every student to respond at least once
Encouraging Completion - Final • One peer ‘holds’ evals until after final grades are submitted • Could be team leaders • Make it the final assignment and student would have to turn it in (to peer/TA) before final grade was submitted
Summary • Clear, easy, non-threatening – to maximize feedback • Focus on key components of your course and key participants in the course • Include YOUR thoughts as well as your students