Download
1 / 31
310 likes | 314 Views
Learn about the assessment process, rubric development, reader training, and presentation of results for the First-Year Writing Program at the Department of English.
E N D
English 1110 Program Assessment 2017-18 <http://go.osu.edu/1110Assessment>
First-Year Writing Program Department of English • Agenda • Introductions • Background and context • The overall assessment plan • Developing the rubric • Recruiting and training raters • Processing direct and indirect results • Considering program changes
First-Year Writing Program • Department of English The FYWP Assessment Process • Assessment Plan Developed and Approved, SU17 • Entrance and Exit Surveys Developed and Added to Carmen, AU17 • Essays Submitted from All Campuses, AU17 • Readers Recruited, AU17 • Reading and Scoring, SP18 • Data Analyzed and Report Created, SU18 • Data Collection and Student Surveys Continue in Anticipation of Further Assessment
First-Year Writing Program • Department of English • Developing the Survey and Rubric
First-Year Writing Program • Department of English GE Learning Outcomes • LEVEL ONE (English 1110) • ELO 1: Students communicate using the conventions of academic discourse. • ELO 2: Students can read critically and analytically.
First-Year Writing Program • Department of English Survey Development • Goal of learning students’ self-reported abilities in terms of confidence and familiarity with outcomes • Goal of learning whether students have a sense of improvement in their abilities over the term • Interest in comparing students’ self-reported abilities with direct assessment measures.
First-Year Writing Program • Department of English Disciplinary Assumptions Feeding into Rubric Development • Writing and reading are integrated discursive processes, not isolated activities. • Writing and reading are bound by contextual constraints that affect production and reception, some of which are conventional, some not. • Writing is complex and creative and there are many ways to achieve acceptable or exceptional outcomes.
First-Year Writing Program • Department of English The Assessment Rubric • Sense of Purpose and Central Idea • Reading and Integration of Sources and Evidence • Awareness of Conventions of Academic Discourse • Mechanics and Usage of Academic Writing • Format and Source Documentation • Overall
First-Year Writing Program • Department of English • Assessment Rubric 1-3
First-Year Writing Program • Department of English • Assessment Rubric 4-6
First-Year Writing Program • Department of English • Criteria for Performance Levels Criteria Differences: E.g., 2 vs. 3
First-Year Writing Program • Department of English • Reader • Training
First-Year Writing Program • Department of English Reader Training Process • Three months prior to session: Call made for volunteers to serve as raters ($100 gift cards) • Week prior to session: Raters given the evaluation rubric and asked to score three “calibration” essays • Day prior to session: Scores transferred to training presentation as anonymous ratings • At the session: Scores and rubric discussed • After session: Recalibration scores requested
First-Year Writing Program Department of English E-mail before Calibration Session
First-Year Writing Program Department of English Slides from Reader Training Scores for 997
First-Year Writing Program Department of English Slides from Reader Training Scores for 998
First-Year Writing Program Department of English Slides from Reader Training Scores for 999
First-Year Writing Program • Department of English • Slides from Reader Training After Today… • You will rescore the calibration essays. • You will receive an invitation to a box.osu folder with approximately 15 essays. • You will rate the essays using the rubric and post those ratings by April 15. • The FYWP team will use the data you provide to create a report to the ASC Curriculum Committee in the summer. • The report will be shared with all.
First-Year Writing Program Department of English Recalibration—997 after Discussion
First-Year Writing Program Department of English Recalibration—998 after Discussion
First-Year Writing Program Department of English Recalibration—999 after Discussion
First-Year Writing Program • Department of English • Processing and Presenting • Results
First-Year Writing Program Department of English Presenting Results: Birdseye View
First-Year Writing Program Department of English Presenting Results: Direct Measures 1 The presentation of results in the final report focuses on showing the percentage of the sample achieving particular performance levels for each evaluative category.
First-Year Writing Program Department of English Presenting Results: Direct Measures 2 In addition to presenting performance percentages for each category of evaluation, the report also cross-references the evaluative categories with learning outcomes.
First-Year Writing Program Department of English Presenting Results: Direct Measures 3 The “Overall” evaluative category provides a holistic measure of performance that can be reconciled with narrower categories to isolate areas for improvement.
First-Year Writing Program Department of English Presenting Results: Indirect 1 The indirect measures allow for a consideration of student sentiments about their own writing and critical reading performance.
First-Year Writing Program Department of English Presenting Results: Indirect 2 We compared entrance and exit results for the same survey questions to assess change in student sentiments about their own writing and critical reading performance.
First-Year Writing Program • Department of English • Considering Program Changes Based on Results
First-Year Writing Program Department of English Takeaways and Adjustments • The students are generally meeting both outcomes and prepared for future coursework. • The current curriculum doesn’t need substantial adjustment, but the results will feed into teacher training in different ways. • Variations in performance among the categories suggest areas for review and reflection: • Category 3 and the question of why comparatively more Basic, rather than Intermediate or Advanced • Category 2 and the question of student ownership of their own writing and learning
First-Year Writing Program • Department of English • Questions?
