Assessment that Matters: Creating Authentic Academic Cultures of Evidence - PowerPoint PPT Presentation

daniel_millan
slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Assessment that Matters: Creating Authentic Academic Cultures of Evidence PowerPoint Presentation
Download Presentation
Assessment that Matters: Creating Authentic Academic Cultures of Evidence

play fullscreen
1 / 20
Download Presentation
Assessment that Matters: Creating Authentic Academic Cultures of Evidence
163 Views
Download Presentation

Assessment that Matters: Creating Authentic Academic Cultures of Evidence

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

    1. Assessment that Matters: Creating Authentic Academic Cultures of Evidence Peter T. Ewell National Center for Higher Education Management Systems

    2. Original Roots of Assessment Re-Capturing Coherence in the Curriculum From Core Curriculum to Coursework Bazaar Faculty Isolation in the Classroom Growing Demands for Learning Productivity Public Accountability for Results TQM/CQI Parallels in Business and Industry Resulting Tensions in Both Motive and Method Ever Since.

    3. Why Not Grades? Cant Communicate About the Outcomes of Groups of Courses Lack of Agreed-Upon Content/Outcome Referents [Standards I] Judgements are Inter-subjectively Unreliable [Standards II]

    4. Three Enduring Characteristics of Assessment Assessment is About Learning Focus on Cognitive Abilities Behaviors Useful, but Secondary Assessment is About Performance Focus on Concrete Abilities--What Students Can Do Requires Clear Definitions of Performance [Appropriate to Context in Which Ability Will be Used] Assessment Goes Beyond Individual Students Focus on Curriculum and Group Performance Requires More than Summative Judgements

    5. More Recent Influences on Assessment Growing Knowledge About How Teaching and Learning Really Works Cognitive Science Instructional Good Practice New Approaches to Instruction Technology and New Media Learner-Centric Approaches Increased Emphasis on Quality Assurance K-12 Standards Movement Competency-Based Certification

    6. Assessment Approaches: A Resulting Shift in Emphasis Accountability-Based: Assessments Added onto Instruction to Check Up on the System in the Aggregate Scholarship and Continuous Improvement: Assessments Built Into the System to Simultaneously Assure Standards and Provide Feedback on Collective Performance

    7. An Expanded Vision of Assessment Seamless Assessments are an Integral Part of Curriculum and Pedagogy Assessments Emphasize Connections and Longitudinal Development, not Just Attainment Assessments Look at Processes and Experiences as well as Outcomes

    8. Examples of Assessment Technique Consistent with this View Capstone Courses and Experiences Student Portfolios [Increasingly Electronic] Course-Embedded Assessments and Dimensional Grading of Selected Class Assignments Longitudinal Studies and Analyses of the Effectiveness of Pre-Requisite Sequences

    9. Commonly-Encountered Issues Creating Appropriate Assignments Judging Student Work Consistently and Usefully Keeping the Process Manageable Creating and Maintaining a Culture of Evidence

    10. Using Class Assignments: A Basic Distinction Direct Assessment: Assignment is Constructed to Explicitly Address the Ability in Question Indirect Assessment: Work Samples are Selected that [Hopefully] Reveal the Ability in Question

    11. Types of Assignments Suitable as Vehicles for Assessment Common Course Examinations Research/Topical Summaries Short Papers Presenting or Critiquing an Argument or Position Journals or Reflective Commentaries Individual or Group Oral Presentations Portfolios

    12. Looking at Student Work: Some Basic Principles Ensure that the Assignment Actually Elicits the Ability in Question Think About Separating Official Grades from Assessment Evidence Incorporate Multiple Judgement Incorporate Multiple Dimensions of the Ability in Question

    13. Looking at Student Work: Some Tools Ability Checklists Holistic Scoring Rubrics Dimensional Scoring Rubrics

    14. Looking at Student Work: Some Operational Guidelines Start with Real Student Work Rate Samples Using an Overall Binary Judgement (e.g. Adequate/Inadequate) Work Outward Toward Additional Dimensions of Performance Work Outward Toward Additional Levels of Performance Anchor Each Level with Actual Examples of Student Work

    15. Some Guidelines for Keeping Assessment Manageable Inventory What You are Already Doing Utilize Existing Points of Contact with Students Select and Sample Drop What You Dont Use

    16. Some Prominent Pitfalls Linear Thinking Goals Dont Always Come First Application and Action Needed at All Stages The Perfect Data Fallacy Recognize that All Data are Bad How Good is Good Enough to Inform Action? Fixed Targets and Standards Look at Strengths and Weaknesses Disaggregate Emphasize Continuous Improvement Either Excessive or Non-Existent Consequences The Paranoia Shift Emphasize Collective Benefits

    17. Cultures of Evidence: Success Factors Visible Metaphor of Scholarship Emphasizing Evidence Over Measurement Beginning with Real Problems and Processes, not with Method Consistent Messages [and Incentives] from Leadership Periodically Re-Socializing the Community Closing the Loop with Action Turn now to what we think we have learned about some of the practicalities of making assessment processes worktactics of assessment in any setting.. - Begin with factors inducing and inhibiting success.. - Then move to the particularly delicate topic of what the relationship ought to be between quality assurance agencies and institutions (or internally between assessment offices and units/faculties) in order to get real engagement and improvement.. [SLIDE] Turn now to what we think we have learned about some of the practicalities of making assessment processes worktactics of assessment in any setting.. - Begin with factors inducing and inhibiting success.. - Then move to the particularly delicate topic of what the relationship ought to be between quality assurance agencies and institutions (or internally between assessment offices and units/faculties) in order to get real engagement and improvement.. [SLIDE]

    18. Some Final Reminders About Assessment: Assessment is.. Not Measuring Everything that Moves Not Just Checking up After the Fact Not Searching for Final Answers Not Always Being as Precise as Possible Not Ever Expecting to be Done

    20. Four Design Principles for Constructing Program-Level Assessments Comprehensiveness: Assessment procedures should cover knowledge and skills taught throughout the program Multiple Judgments: Assessment procedures should involve more than one source of evidence or involve multiple judgments of student performance Multiple Dimensions: Assessment procedures should provide information on multiple dimensions of student performance, i.e., they should yield more than a single summative grade Directness of Assessment: Assessment procedures should involve at least one type of evidence based on direct observation or demonstration of student capacities; i.e., they should involve more than simply a self-report