Using Rubrics to Evaluate Student Writing

1. Using Rubrics to Evaluate Student Writing Center for Teaching and Learning February 21, 2012

2. Why Rubrics? • No itemized grading option in Blackboard 8 • Encourage grading uniformity across large classes • Reduce real or perceived grading “bias” • Concretize subjective evaluations • Peg student achievement to predetermined waypoints • Faster than relying entirely on long-form evaluations.

3. Why Rubrics? (Cont) • Clarify instructor expectations • Preserve detailed, itemized grading records in electronic format. • Avoid student questions about “what counts”. • Allow students to measure progress during semester, clearly indicate areas for improvement. • Institutions can require universal adoption and collect aggregate data

4. Drawbacks • Reduces qualitative evaluations to quantitative measures • Discourages personalized evaluations • May discourage inclusion of non-technical criteria (word choice, textual flow, “lyricism”) • May create impression that quality writing is strictly a technical matter • May encourage “formulaic” writing, discourage risk-taking.

5. Drawbacks (Cont) • Depersonalizes feedback when not used in tandem with long-form evaluations • 3rd party webware may present security concerns • Advanced software products charge exorbitant fees for institution-wide use • Not efficient for “low stakes” assignment where little or no feedback is provided.

6. Best Practices • Distribute rubric form in advance, alongside assignment instructions • Explain rubric at time of distribution, detailing each criterion • Supplement rubric with long-form evaluations OR Supplement each criterion with long-form comments

7. Suggested Uses • Writing Intensive courses • Large and unwieldy class sizes • Major writing assignments • Final Papers / Final Projects • Student Presentations • Lab Reports (?) • Peer Evaluations (?)

8. How to Design a Rubric • Criteria can be designed from scratch or borrowed from a freely available template. • Criteria are assigned points values and descriptive language (Ex: always [fulfills criteria], often, sometimes, rarely, never….) • Criteria can be weighted to indicate relative importance • Some rubric systems allow for “negative” criteria (deductions)

9. Technical Issues • Some products are Blackboard compatible, but CUNY’s existing license does not allow Blackboard to share information directly with 3rd party webware/software • Most products allow student rosters to be imported • To download students rosters from Blackboard, go to control panel  grade center  manage  download  select “comma” delimiter

10. Technical Issues (Cont) • Most products allow assignment grades to be exported, then imported back into Blackboard, but compatibility issues are likely to arise • Suggested workaround: Export grades from software  export complete Blackboard grade center  copy and paste relevant grading column into offline grade center  import revised grade center to Blackboard • Less technically capable instructors may prefer to input grades longhand in Grade Center

11. Case Study Introduction to Labor Studies Final Paper Instructions Due December 23 Your task is to analyze a key debate, conflict, or struggle in the contemporary US labor movement, taking into account multiple competing perspectives.  The bulk of your research should be drawn from primary source materials (eg. newspaper articles, websites, flyers, pamphlets, press releases).  You may also choose to consult secondary sources relevant to your topic (eg. scholarly articles, books), however this is not required.  In addition, your research must include an element of fieldwork.  This can take a number of forms -- we’re defining fieldwork very broadly here.  A partial list of acceptable fieldwork activities would include attending a meeting, participating in a demonstration, conducting an interview, walking a picket line, or visiting a worksite. When conducting your research you should pay particular attention to the following questions.  These are only guidelines; you are not expected to answer every one. Who are the major players on each side? What organization(s) do they represent? How powerful is each side? What position has each side taken? How has each side attempted to counter its opponent’s position? Are the rival organizations sticking together or splintering? Are the rival organizations heading towards compromise or confrontation? Which side appears to be winning? Why has the winning side been more effective? What are the prospects for the future?

12. Case Study (Cont) Your paper should be analytical, not merely descriptive.  In other words, don’t just summarize recent events -- explain why they happened.  Although you may probably find it useful to refer to recent events, your focus should be on the issues and ideas surrounding these events.   Before beginning work on this project, you must submit a proposal briefly describing your topic along with a preliminary list of sources.  This should be submitted via email (introlaborstudies@gmail.com) no later than November 16.   I’ll respond to your proposal either by giving you a go-ahead, by asking you to re-tweak your proposal.  In very rare cases, proposals are rejected entirely, but if you follow this guideline sheet closely you can avoid that unfortunate outcome.  Paper proposals will not be graded but they are required.

13. Sample Criteria • Strength of Argument (15pts) • Is the argument clear and well-defined? • Quality of Research (10pts) • Did the student gather information from a variety of quality sources? Are the sources are relevant and balanced? • Organization (10pts) • Is the essay logic and methodical?

14. Sample Criteria (Cont) • Fieldwork (5pts) • Is the fieldwork well-documented? Does it make a significant contribution to the author’s argument? • Depth of Analysis (5pts) • Is the essay intellectually sophisticated? Is there strong evidence of analytical thinking? • References (5pts) • Does the student consistently supports his/her argument by drawing upon specific and appropriate examples from the text?

15. Sample Criteria (Cont) • Improper Quotations Penalty (up to -10) • Quotations are not properly cited or supported by relevant commentary. • Grammar and Spelling Penalty (up to -10) • The student does not adhere to basic conventions of sentence structure, grammar, spelling, and punctuation • Lateness Penalty (up to -20)

16. Waypoint Outcomes • Elaborate, user-friendly interface • Full Blackboard integration available • Integrated in-text commenting tool • Data aggregation and reporting capability • Premium product • Free, limited-feature edition available

17. Waypoint Outcomes

18. Waypoint Outcomes

19. Waypoint Outcomes

20. Waypoint Overview

21. Rubistar • http://rubistar.4teachers.org/ • Basic product, not as slick and technically sophisticated as Waypoint • Includes database of sample rubrics • Pre-fab rubric templates for various subjects • No Blackboard integration, but possible to import student roster • Free

22. RubiStar

23. iRubric • http://www.rcampus.com/indexrubric.cfm • One component of rCampus course management system (Blackboard alternative) • “Freemium” business model • Works best alongside rCampusgradebookand other rCampus tools • Could also be used as a standalone product • Possible to import student roster from Blackboard • iPad friendly

24. iRubric

25. Google Docs (DIY Rubric) • http://docs.google.com • Google Spreadsheet allows users to simulate the basic features of other rubric systems • Can be shared with multiple instructors • Requires saving multiple copies of each rubric as individual documents • Potentially time-consuming

26. Google Docs (DIY Rubric)

27. Proprietary Software Options • Generally offers few advantages over free / freemiumwebware • Offers security to instructors who prefer to keep student records away from 3rd parties • Pricing ranges up to $100 for individual usage • Examples: Rubrixhttp://rubrix.com/, iSocrateshttp://www.isocrates.org/

28. Future of Rubrics • Next edition of Blackboard may include a built-in rubric component • Freemiumwebware is becoming more sophisticated, increasingly appealing option • CUNY may choose to purchase a license for premium webware if there is enough instructor demand • Some departments may wish to create standardized rubrics for particular assignments