Grading smarter: How are you going the distance?

1. Grading smarter:How are you going the distance?

2. Grading, Test Banks & Assessment Objectives Gregory Benton Health, Human Performance & Recreation (EDHP) gbenton@uark.edu & Linda Jones World Languages, Literatures & Cultures (ARSC) lcjones@uark.edu

3. Issues to consider • Pros/cons of Online instruct & assess for student v. for instructor • Pros/cons of Test Banks (online) for student v. for instructor 3. Does the test bank Publisher matter? 4. The mechanics of multiple choice Qs 5. Hybrid or blended assessment 6. Rubrics: type & form 7. Blackboard mechanics of rubric creation

4. A place to start Given the transformation in delivery of instruction & assessment … What do pre-service teachers (Grad) think of online assessment in general? • 95% std preferred some online assessment (b/c immediate feedback) • 91% std said it helped their learning (b/c discussions, forums, e-mail, boards, chats) (Boyles, 2011)

5. “No significant difference phenomenon” Russell, T. L. (1999) website & book (2001) http://www.nosignificantdifference.org/ Catalogs 355 research reports & articles on distance education technology with p values greater than .05 Distance education technology versus face-to-face instruction (accessed 07/02/12)

6. Whose brain works better… under what conditions?

7. Carolyn Speer-Schmidt Article in Online Classroom (April, 2006) offers perspective on complete switch in instructional delivery & assessment delivery • thought she would lose the human touch but instead “I have expanded my definition of it” (p.1) • fear of losing connection to students & academic freedom • old ideas: “teaching is bounded by a schedule slot” (p.2); leave teaching when you leave the classroom. • new attitude: her students have broken out of the classroom, come into her head wherever she goes

8. Follow-up with Speer-Schmidt In a follow-up e-mail, author clarified points: • favors formative evaluation – wants all experiences to go towards learning (Nursing would be appropriate summative evaluation) • allows multiple test attempts, scores highest • favors publisher-generated test banks and has students blog regarding flawed Q’s • notes that Blackboard (Bb) doesn’t allow changes in tests once they are deployed*

9. Talking points Whenyou convert a traditional in-class course to online, in what ways do you feel you: a) lost effectiveness in learning b) gained authentic assessment c) struggled to make adjustments d) succeeded in improving learning

10. How to do web-based assessment An experimental evaluation of web-based tutorial quizzes (Klass & Crothers, 2000) • American govt. online students could take optional tutorial quizzes = no sig diff posttest • Mallard software (Blackboard, TopClass) • Almost all Am Govt textbooks provide banks • Standards for writing multiple choice tests • Publishers of test banks frequently violate test guidelines (results in “bad” questions)

11. Klass & Crothers cont. Advantages • immediate feedback • can retake until satisfied • takes almost no class time • quiz auto graded & entered by computer Disadvantages • initial investment of time to format quizzes (Mallard software, not Respondus)

12. Interview with Elaine Terrell of Bb About students online: • Std open up more b/c more individualized • In-class std can sit back & let others answer • Std have to work harder to respond online but is better (esp if they can’t think on their feet in public); lower vulnerability=courage • Don’t allow std to post to blogs anonymously

13. More Elaine Terrell thoughts Online assessment cons: • If instructor can access bank answers, so can students (esp for large survey courses) • Instructor perception of organized cheating • Instructor still needs to trouble-shoot questions (but quicker process)

14. E.T. continued Online assessment pros: • Previous experience with large class paper & scantrons: lots of materials/print/scan but time to distribute & collect tests • Can give online tests during scheduled class meeting time to avoid other time conflicts • Meta study at Uark: std learned more online (only) than in-class but even more in hybrid or “blended course” b/c multi learn styles, learning mediums, attention span, lecture prep

15. Talking points • Do students prefer online assessment? • “Extra” quiz practice Y/N? • Trustworthy textbooks/test banks publishers? • Perceptions of student learning/cheating • Instructor facilitation versus student learning

16. References • Boyles, P. C. (2011). Maximizing learning using online student assessment. Online Journal of Distance Learning Administration, 14, 3, 1-7. • Klass, G. & Crothers, L. (2000). An experimental evaluation of web-based tutorial quizzes. Social Science Computer Review, 18, 580-516. • Shank, P. (2006). Multiple choice question types (Assessments, Part 4). Online Classroom, April, 4-5. • Speer-Schmidt, C. (2006). Online teaching got me out of the box. Online Classroom, April, 1-3. • Terrell, E. (2012). Blackboard perspectives on online assessment and test banks. (personal communication, April 19, 2012).

17. Grading Options in Blackboard • Teacher-graded, sans rubriques! • Blogs, wikis, discussion boards, and so on…risqué! • Teacher-graded, avec rubriques! • Rubrics are everyone’s friend! • Computer-graded • As smart as the programmer/teacher!

18. Grading Smart with Blackboard“Teacher-Graded” Rubrics A set of scoring guidelines for evaluating student work… Rubrics show learners what good performance looks like even before they complete their assignment.

19. Questions to Ponder… • By what criteria should performance be judged? • What should we look for to judge performance success? • What does the range in the quality of performance look like? • How do we determine scores and their meaning? • How should the different levels of quality be described and distinguished from one another?* *Shrum, J. & Glisan, E. (2005). Teacher’s Handbook: Contextualized Language Instruction, 3rd Edition, p. 372.

20. Recipe for a Good Rubric… • Assignment Directions – when to complete it; what to focus on; format for the assignment • Categories – organization, comprehensibility, grammar, and so on… • Levels of Achievement – Excellent, good, fair… • Descriptors for each level of achievement – what makes performance excellent, good, fair… • Point Scale

21. Creating a Rubric in Blackboard(Will demonstrate with Blackboard if possible!)

22. Create Rubric and Name it…

25. Connect Rubric to Assignment…

28. Create Blackboard Rubric file for “unattached” assignment

29. Grading in Blackboard“Computer Driven” • Respondus, desktop assessment design, Windows platform • Internal tests and assessments (will demonstrate).

31. Issues to consider • Pros/cons of Online instruct & assess for student v. for instructor • Pros/cons of Test Banks (online) for student v. for instructor 3. Does the test bank Publisher matter? 4. The mechanics of multiple choice Qs 5. Hybrid or blended assessment 6. Rubrics: type & form 7. Blackboard mechanics of rubric creation

32. Now you can better go the distance! Thank You – Gregory & Linda