Improving undergraduate education through the assessment of student learning
Sponsored Links
This presentation is the property of its rightful owner.
1 / 30

Improving Undergraduate Education through the Assessment of Student Learning PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Improving Undergraduate Education through the Assessment of Student Learning. SACS-COC Institute on Quality Enhancement and Accreditation July 30, 2006 Peter T. Ewell National Center for Higher Education Management Systems. Looking Back: Origins of the Assessment Movement Two Decades Ago.

Download Presentation

Improving Undergraduate Education through the Assessment of Student Learning

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Improving undergraduate education through the assessment of student learning

Improving Undergraduate Education through the Assessment of Student Learning

SACS-COC Institute on Quality Enhancement and Accreditation

July 30, 2006

Peter T. Ewell

National Center for Higher Education Management Systems

Looking back origins of the assessment movement two decades ago

Looking Back: Origins of the Assessment Movement Two Decades Ago

  • Undergraduate Reform Reports of 1985-86

  • Internal Stimulus: Call for More Coherent Teaching/Learning Approaches and Information for Improvement

  • External Stimulus: Stakeholder Demands for Information on “Return on Investment”

  • Tensions in Motive and Message Ever Since

Why didn t assessment go away

Why Didn’t Assessment Go Away?

  • Pressure to Produce Evidence of Student Learning Outcomes Never Let Up

  • By Early 1990s, Accreditors Replace States as Primary External Stimulus to Get Started

  • Intermittent Federal Interest in Assessment as an Element of National Accountability

  • But Resulting Faculty Ambivalence About a Process Seen as “External” and “Administrative”

Looking back what s been accomplished

Looking Back: What’s Been Accomplished?

  • Assessment Is for the Most Part Perceived as Inevitable and Legitimate

  • Vast Majority of Institutions Have Statements of Learning Outcomes (General and Programmatic)

  • A “Semi-Profession” of Folks Involved in Assessment

  • Steadily Growing Sophistication with Respect to Methods of Gathering Evidence

Looking back what hasn t happened

Looking Back: What Hasn’t Happened?

  • Authentic Integration of Assessment into Faculty Cultures and Behaviors

  • Assessment Activities Still Largely “Added On” to the Curriculum Instead of Being Embedded In It

  • Systematic and Widespread Use of Assessment Results for Institutional and Curricular Improvement

  • Proactive and Sincere Institutional Engagement with Accrediting Organizations Around Topics of Assessment

Assessment as a perpetual movement

Assessment as a “Perpetual Movement?”

  • Most Social Movements…

    • Fade as Fads, or

    • Go Away Because Core Ideas are Mainstreamed

  • The “Assessment Movement” Has Done Neither...

    • External Requirements Keep it Alive

    • But External Pressures also Constrain Faculty Buy-In and Meaningful Institutional Use

  • Moving Beyond a “Perpetual Movement” Will Demand...

    • Making Assessment Real to Faculty by Connecting It to the Actual Practice of Teaching and Learning

    • Re-Focusing Accountability on the Authentic Student Abilities that Society Says it Needs

Some prominent changes in higher education s operating environment

Some Prominent Changes in Higher Education’s Operating Environment

  • Doing More with Less

    • Need for Curricula that are Effective and Efficient

    • Need Information About Curricular Functioning to Enable Effective Action

  • Changes in Instructional Delivery

    • Competencies and “Deep Learning”

    • Student Engagement and Role of Technology

  • New Views of Accountability

    • “Quality”— From Inputs/Processes to Results

    • Stakeholder Voices — Students and Employers

Challenges to the academy the internal dimension

Challenges to the Academy: The Internal Dimension

  • Changing Paradigm of Teaching and Learning

  • Resulting Changes in Academic Roles, Behaviors and Structures

    • Nature and Role of “Faculty” in Instruction

    • Patterns of Student Enrollment

  • [Technology as a “Wild Card”]

A changing paradigm of teaching and learning

A Changing Paradigm of Teaching and Learning

  • From “Faculty Teaching” to “Students Learning”

  • Students “Make Their Own Paths” through Multiple Learning Opportunities

  • Explicit Designs for Learning Based on Research

Changing paradigm possible lines of response

Changing Paradigm: Possible Lines of Response

  • Demonstrated Achievement Becomes Paramount, not “Seat Time”

  • Students “Teach” One Another

  • Individual (and Asynchronous) Paths

  • Technology Seen as Opportunity to Re-Think [not as a “Solution”]

Some implications for student assessment processes

Some Implications for Student Assessment Processes

  • Assessments Reinforce Common Standards for Learning Across Curricula and Classes

  • “Seamless” Assessments Become an Integral Part of Curriculum and Pedagogy

  • Assessments Emphasize Connections and Longitudinal Development, not Just Attainment

Assessment approaches a resulting shift in emphasis

Assessment Approaches: A Resulting Shift in Emphasis

  • Accountability-Based: Assessments Added onto Instruction to “Check Up” on the System in the Aggregate

  • Scholarship and Improvement: Assessments Built into the System to Simultaneously Assure Standards and Provide Feedback on Collective Performance

Kinds of information needed

Kinds of Information Needed

  • Alignment of Key Learning Outcomes Across Units, Sequences, and Courses

  • Match Between Curricular Design, Delivery, and Student Experience

  • Match Between Instruction and Needs of Diverse “Student Bodies”

  • Effectiveness of Particular Innovations and Interventions

Changing nature of the faculty role

Changing Nature of the Faculty Role

  • Dis-aggregation of Instructional Roles

  • More Things for Faculty to Do

  • New Potential Career Patterns and Paths

New faculty roles possible lines of response

New Faculty Roles: Possible Lines of Response

  • Emphasize Peer Support and Collaboration in Faculty Development

  • Recognize and Regularize Alternative Career Paths

  • Recognize “Mentorship” as the One Thing You Can’t Responsibly Outsource

Kinds of information needed1

Kinds of Information Needed

  • Capturing and Re-Aggregating Data About Discrete Instructional Functions

  • Accounting Technology-Based Costs

  • Tracking Faculty “Assets”

Changing patterns of student attendance

Changing Patterns of Student Attendance

  • Increased Levels of Multi-Institutional Attendance

  • Increased Complexity in Course-Taking Behavior within Institutions

  • Greater (and Unpredictable) Time Lapses Between Instructional Encounters

Changing patterns of attendance possible lines of response

Changing Patterns of Attendance: Possible Lines of Response

  • Coherence Based on Common Practices [and Outcomes], not Common Content

  • Stress Ways for Students to Exploit and Reflect on Their Own Experiences

  • Establish Clear Transition Points at which to Assess Student Mastery of Key Concepts

Kinds of information needed2

Kinds of Information Needed

  • Relationships Between Particular Institutional Experiences and Particular Outcomes

  • Tracking Student Learning Styles and Individual Paths of Development

  • “Episode-based” [as opposed to time-based] Data Structures

Challenges to the academy the external dimension

Challenges to the Academy: The External Dimension

  • Increased Accountability

  • Changing Expectations Regarding What Students Know and Can Do

  • The “New Competition”

What are states doing

What Are States Doing?

  • Forces Influencing State Approaches

    • Decreased Agency Capacity Due to Funding Cuts

    • Momentum of K-12 Reform (NCLB)

    • Political Uncertainty and Instability

  • Types of State Policy Responses

    • P-16 Alignment

    • Performance Measures and “Report Cards”

    • Achievement Testing (Driven by K-12)

What s shaping accreditation

What’s Shaping Accreditation?

  • Forces Influencing Accreditors

    • Pressure for Specific Performance from Federal Government

    • Demands from Institutions to “Add Value”

    • New Models from Other Sectors and Abroad

  • “New Looks” in Accreditation

    • Focus on Outcomes and Effectiveness

    • Presenting Evidence [e.g. “Portfolios”]

    • Review Approaches [e.g. “Academic Audits”]

    • Connection to Institutional Planning [e.g. QEP]

Learning outcomes what employers expect

Learning Outcomes: What Employers Expect

  • Higher-Order “Literacies” as Well as Specific Skills

  • Framed in Terms of “Practice” [not “Knowledge”]

  • As Much About Attitudes as Academics [“Soft Skills”]

External forces some resulting influences on the academy

External Forces: Some Resulting Influences on the Academy

  • Increased Emphasis on Credentialing

  • Modularity and Acceleration to Increase Accessibility

  • Pressure to Respond to “Students as Customers”

  • Accountability “Superstructures” that Divert Attention and Information Resources

Kinds of information needed3

Kinds of Information Needed

  • Outcomes and Performance Measures

  • Data on What Experiences/Services Students Can Expect

  • Peer Comparisons [Increasingly Outside the Academy] and Comparative Performance

  • Needs and Satisfaction of External Stakeholders

Attributes of a meaningful culture of evidence

Attributes of a Meaningful “Culture of Evidence”

  • Shared Recognition That Many (But Not All) Things Are Knowable

  • An Accessible Store of Information About Organizational Condition and Performance

  • An Attitude Toward Problem-Solving that Minimizes “Finger-Pointing”

  • Clear Follow-Through On Decisions Made and Why They Were Taken

Cultures of evidence success factors

Cultures of Evidence: Success Factors

  • Visible Metaphor of Scholarship

  • Beginning with Real Problems and Processes, not with “Method”

  • Consistent Messages from Leadership

  • Periodically “Re-Socializing” the Community

  • “Closing the Loop” with Action

Cultures of evidence inhibiting factors

Cultures of Evidence: Inhibiting Factors

  • Either Excessive or Non-Existent Consequences

  • Alien Language and “Management Culture”

  • Excessive Complexity

  • Burnout and “Committee Fatigue”

  • Changing the Rules

Meaningful assessment is more about mindset than method

Meaningful Assessment is More About Mindset than Method

  • Questions About Learning are not Just Matters of Opinion

    • What Information Might We Collect?

    • What Might We Expect to Find?

    • What Difference Would Finding Out Make?

  • Assessment is About Improving Practice

    • What are We Trying to Fix?

    • How Good is Good Enough?

    • What Changes are Implied?

The bottom line

The Bottom Line

  • For Internal Management, “Seat of the Pants” Decision-making is no Longer Sufficient

    • Information Used Openly, Consistently, and Continuously to Inform Academic Decisions

  • For External Constituencies, “Trust Me” is no Longer Sufficient

    • Need Clear, Understandable Evidence of Student Academic Attainment

  • Be Vigilant about the Information You Choose and the Signals it Sends

    • Make Sure that What You Measure is What You Value

    • Harness the Accreditation Process to Make it Happen!

  • Login