Improving undergraduate education through the assessment of student learning
1 / 30

Improving Undergraduate Education through the Assessment of Student Learning - PowerPoint PPT Presentation

  • Uploaded on

Improving Undergraduate Education through the Assessment of Student Learning. SACS-COC Institute on Quality Enhancement and Accreditation July 30, 2006 Peter T. Ewell National Center for Higher Education Management Systems. Looking Back: Origins of the Assessment Movement Two Decades Ago.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Improving Undergraduate Education through the Assessment of Student Learning' - kyne

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Improving undergraduate education through the assessment of student learning

Improving Undergraduate Education through the Assessment of Student Learning

SACS-COC Institute on Quality Enhancement and Accreditation

July 30, 2006

Peter T. Ewell

National Center for Higher Education Management Systems

Looking back origins of the assessment movement two decades ago
Looking Back: Origins of the Assessment Movement Two Decades Ago

  • Undergraduate Reform Reports of 1985-86

  • Internal Stimulus: Call for More Coherent Teaching/Learning Approaches and Information for Improvement

  • External Stimulus: Stakeholder Demands for Information on “Return on Investment”

  • Tensions in Motive and Message Ever Since

Why didn t assessment go away
Why Didn’t Assessment Go Away? Decades Ago

  • Pressure to Produce Evidence of Student Learning Outcomes Never Let Up

  • By Early 1990s, Accreditors Replace States as Primary External Stimulus to Get Started

  • Intermittent Federal Interest in Assessment as an Element of National Accountability

  • But Resulting Faculty Ambivalence About a Process Seen as “External” and “Administrative”

Looking back what s been accomplished
Looking Back: What’s Been Accomplished Decades Ago?

  • Assessment Is for the Most Part Perceived as Inevitable and Legitimate

  • Vast Majority of Institutions Have Statements of Learning Outcomes (General and Programmatic)

  • A “Semi-Profession” of Folks Involved in Assessment

  • Steadily Growing Sophistication with Respect to Methods of Gathering Evidence

Looking back what hasn t happened
Looking Back: What Hasn’t Happened? Decades Ago

  • Authentic Integration of Assessment into Faculty Cultures and Behaviors

  • Assessment Activities Still Largely “Added On” to the Curriculum Instead of Being Embedded In It

  • Systematic and Widespread Use of Assessment Results for Institutional and Curricular Improvement

  • Proactive and Sincere Institutional Engagement with Accrediting Organizations Around Topics of Assessment

Assessment as a perpetual movement
Assessment as a “Perpetual Movement?” Decades Ago

  • Most Social Movements…

    • Fade as Fads, or

    • Go Away Because Core Ideas are Mainstreamed

  • The “Assessment Movement” Has Done Neither...

    • External Requirements Keep it Alive

    • But External Pressures also Constrain Faculty Buy-In and Meaningful Institutional Use

  • Moving Beyond a “Perpetual Movement” Will Demand...

    • Making Assessment Real to Faculty by Connecting It to the Actual Practice of Teaching and Learning

    • Re-Focusing Accountability on the Authentic Student Abilities that Society Says it Needs

Some prominent changes in higher education s operating environment
Some Prominent Changes in Higher Education’s Operating Environment

  • Doing More with Less

    • Need for Curricula that are Effective and Efficient

    • Need Information About Curricular Functioning to Enable Effective Action

  • Changes in Instructional Delivery

    • Competencies and “Deep Learning”

    • Student Engagement and Role of Technology

  • New Views of Accountability

    • “Quality”— From Inputs/Processes to Results

    • Stakeholder Voices — Students and Employers

Challenges to the academy the internal dimension
Challenges to the Academy: EnvironmentThe Internal Dimension

  • Changing Paradigm of Teaching and Learning

  • Resulting Changes in Academic Roles, Behaviors and Structures

    • Nature and Role of “Faculty” in Instruction

    • Patterns of Student Enrollment

  • [Technology as a “Wild Card”]

A changing paradigm of teaching and learning
A Changing Paradigm of EnvironmentTeaching and Learning

  • From “Faculty Teaching” to “Students Learning”

  • Students “Make Their Own Paths” through Multiple Learning Opportunities

  • Explicit Designs for Learning Based on Research

Changing paradigm possible lines of response
Changing Paradigm: EnvironmentPossible Lines of Response

  • Demonstrated Achievement Becomes Paramount, not “Seat Time”

  • Students “Teach” One Another

  • Individual (and Asynchronous) Paths

  • Technology Seen as Opportunity to Re-Think [not as a “Solution”]

Some implications for student assessment processes
Some Implications for Student Assessment Processes Environment

  • Assessments Reinforce Common Standards for Learning Across Curricula and Classes

  • “Seamless” Assessments Become an Integral Part of Curriculum and Pedagogy

  • Assessments Emphasize Connections and Longitudinal Development, not Just Attainment

Assessment approaches a resulting shift in emphasis
Assessment Approaches: EnvironmentA Resulting Shift in Emphasis

  • Accountability-Based: Assessments Added onto Instruction to “Check Up” on the System in the Aggregate

  • Scholarship and Improvement: Assessments Built into the System to Simultaneously Assure Standards and Provide Feedback on Collective Performance

Kinds of information needed
Kinds of Information Needed Environment

  • Alignment of Key Learning Outcomes Across Units, Sequences, and Courses

  • Match Between Curricular Design, Delivery, and Student Experience

  • Match Between Instruction and Needs of Diverse “Student Bodies”

  • Effectiveness of Particular Innovations and Interventions

Changing nature of the faculty role
Changing Nature of the Faculty Role Environment

  • Dis-aggregation of Instructional Roles

  • More Things for Faculty to Do

  • New Potential Career Patterns and Paths

New faculty roles possible lines of response
New Faculty Roles: EnvironmentPossible Lines of Response

  • Emphasize Peer Support and Collaboration in Faculty Development

  • Recognize and Regularize Alternative Career Paths

  • Recognize “Mentorship” as the One Thing You Can’t Responsibly Outsource

Kinds of information needed1
Kinds of Information Needed Environment

  • Capturing and Re-Aggregating Data About Discrete Instructional Functions

  • Accounting Technology-Based Costs

  • Tracking Faculty “Assets”

Changing patterns of student attendance
Changing Patterns of Student Attendance Environment

  • Increased Levels of Multi-Institutional Attendance

  • Increased Complexity in Course-Taking Behavior within Institutions

  • Greater (and Unpredictable) Time Lapses Between Instructional Encounters

Changing patterns of attendance possible lines of response
Changing Patterns of Attendance: Possible Lines of Response Environment

  • Coherence Based on Common Practices [and Outcomes], not Common Content

  • Stress Ways for Students to Exploit and Reflect on Their Own Experiences

  • Establish Clear Transition Points at which to Assess Student Mastery of Key Concepts

Kinds of information needed2
Kinds of Information Needed Environment

  • Relationships Between Particular Institutional Experiences and Particular Outcomes

  • Tracking Student Learning Styles and Individual Paths of Development

  • “Episode-based” [as opposed to time-based] Data Structures

Challenges to the academy the external dimension
Challenges to the Academy: EnvironmentThe External Dimension

  • Increased Accountability

  • Changing Expectations Regarding What Students Know and Can Do

  • The “New Competition”

What are states doing
What Are States Doing? Environment

  • Forces Influencing State Approaches

    • Decreased Agency Capacity Due to Funding Cuts

    • Momentum of K-12 Reform (NCLB)

    • Political Uncertainty and Instability

  • Types of State Policy Responses

    • P-16 Alignment

    • Performance Measures and “Report Cards”

    • Achievement Testing (Driven by K-12)

What s shaping accreditation
What’s Shaping Accreditation? Environment

  • Forces Influencing Accreditors

    • Pressure for Specific Performance from Federal Government

    • Demands from Institutions to “Add Value”

    • New Models from Other Sectors and Abroad

  • “New Looks” in Accreditation

    • Focus on Outcomes and Effectiveness

    • Presenting Evidence [e.g. “Portfolios”]

    • Review Approaches [e.g. “Academic Audits”]

    • Connection to Institutional Planning [e.g. QEP]

Learning outcomes what employers expect
Learning Outcomes: EnvironmentWhat Employers Expect

  • Higher-Order “Literacies” as Well as Specific Skills

  • Framed in Terms of “Practice” [not “Knowledge”]

  • As Much About Attitudes as Academics [“Soft Skills”]

External forces some resulting influences on the academy
External Forces: EnvironmentSome Resulting Influences on the Academy

  • Increased Emphasis on Credentialing

  • Modularity and Acceleration to Increase Accessibility

  • Pressure to Respond to “Students as Customers”

  • Accountability “Superstructures” that Divert Attention and Information Resources

Kinds of information needed3
Kinds of Information Needed Environment

  • Outcomes and Performance Measures

  • Data on What Experiences/Services Students Can Expect

  • Peer Comparisons [Increasingly Outside the Academy] and Comparative Performance

  • Needs and Satisfaction of External Stakeholders

Attributes of a meaningful culture of evidence
Attributes of a Meaningful Environment“Culture of Evidence”

  • Shared Recognition That Many (But Not All) Things Are Knowable

  • An Accessible Store of Information About Organizational Condition and Performance

  • An Attitude Toward Problem-Solving that Minimizes “Finger-Pointing”

  • Clear Follow-Through On Decisions Made and Why They Were Taken

Cultures of evidence success factors
Cultures of Evidence: Success Factors Environment

  • Visible Metaphor of Scholarship

  • Beginning with Real Problems and Processes, not with “Method”

  • Consistent Messages from Leadership

  • Periodically “Re-Socializing” the Community

  • “Closing the Loop” with Action

Cultures of evidence inhibiting factors
Cultures of Evidence: Inhibiting Factors Environment

  • Either Excessive or Non-Existent Consequences

  • Alien Language and “Management Culture”

  • Excessive Complexity

  • Burnout and “Committee Fatigue”

  • Changing the Rules

Meaningful assessment is more about mindset than method
Meaningful Assessment is More About Mindset than Method Environment

  • Questions About Learning are not Just Matters of Opinion

    • What Information Might We Collect?

    • What Might We Expect to Find?

    • What Difference Would Finding Out Make?

  • Assessment is About Improving Practice

    • What are We Trying to Fix?

    • How Good is Good Enough?

    • What Changes are Implied?

The bottom line
The Bottom Line Environment

  • For Internal Management, “Seat of the Pants” Decision-making is no Longer Sufficient

    • Information Used Openly, Consistently, and Continuously to Inform Academic Decisions

  • For External Constituencies, “Trust Me” is no Longer Sufficient

    • Need Clear, Understandable Evidence of Student Academic Attainment

  • Be Vigilant about the Information You Choose and the Signals it Sends

    • Make Sure that What You Measure is What You Value

    • Harness the Accreditation Process to Make it Happen!