slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Effective Methods for Assessing the Impact of Service-Learning on Students, Institutions, and Communities PowerPoint Presentation
Download Presentation
Effective Methods for Assessing the Impact of Service-Learning on Students, Institutions, and Communities

Loading in 2 Seconds...

play fullscreen
1 / 35

Effective Methods for Assessing the Impact of Service-Learning on Students, Institutions, and Communities - PowerPoint PPT Presentation


  • 248 Views
  • Uploaded on

Effective Methods for Assessing the Impact of Service-Learning on Students, Institutions, and Communities. ANDREW FURC0 University of Minnesota February 27, 2009. Problems with Service-Learning Evaluation. Program goals are unrealistic. Expected outcomes cannot be easily measured.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Effective Methods for Assessing the Impact of Service-Learning on Students, Institutions, and Communities' - denali


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Effective Methods for Assessing the Impact of Service-Learning on Students, Institutions, and Communities

ANDREW FURC0

University of Minnesota

February 27, 2009

problems with service learning evaluation
Problems with Service-Learning Evaluation
  • Program goals are unrealistic.
  • Expected outcomes cannot be easily measured.
  • Evaluation is an after thought.
  • Evaluation is not aligned to program goals
  • We claim impacts when we only measured outcomes.
  • The time frame is not commensurate for impacts to manifest.
  • We view all service-learning experiences to be the same .
  • We make bold claims after one study.
general tips for successful evaluation
General Tips for Successful Evaluation
  • Set measurable, realistic goals
  • Focus the evaluation
  • Have your evaluation design match evaluation question(s)
  • Assess the quality of the service-learning practice
  • Secure an appropriate sample
  • Have a realistic timeline for data collection
  • Apply appropriate instruments & data collection techniques
  • Use a systematic approach for data collection
  • Have a plan for how the data will be analyzed systematically
example
Example

DIFFICULT TO EVALUATE:

To what extent did the service-learning experience reduce homelessness?

BETTER:

In what ways did the homeless situation change during the s-l experience?

example5
Example

DIFFICULT TO EVALUATE:

What are the effects of service-learning on students’ development of civic capacities?

BETTER:

Which civic capacities do students report having gained from their engagement in service-learning?

service learning quality matters
Service-Learning Quality Matters
  • Principles (e.g., Wingspread Principles)
  • Balance Beam
  • Quadrant
  • Essential Elements
service learning a balanced approach to experiential education
Service-Learning: A Balanced Approach to Experiential Education

FOCUS

Service

Learning

PRIMARY INTENDED BENEFICIARY

Recipient

Provider

Service-Learning

Community Service

Field Education

Volunteerism

Internships

slide8

Assessing Service-Learning Quality

High Service

II

I

Unrelated

Learning

Integrated

Learning

III

IV

Adapted from the Service Learning Quadrant, Service Learning 2000 Center

Low Service

slide9

Students in a Biology course are studying the brain and the nervous system. Students in the class are required to spend one afternoon during the month observing doctors at a neurology center. Students observe the kinds of treatments doctors offer patients with neurological disorders. At the end of the course, students write a brief paper in which theydiscuss their observations.

High Service

II I

Unrelated Integrated Learning Learning

III IV

Low Service

slide10

Students in a Scandinavian Languages course spend three to five hours each week visiting with elderly individuals of Scandinavian descent. The students assist the seniors with daily tasks, such as reading the mail or newspaper, shopping, and exercising. The students also provide the seniors with an opportunity to share stories about the seniors’ life in Scandinavia. From this information, the students develop oral histories of Scandinavian life in the mid-Twentieth Century and preserve the histories in a monograph.

High Service

II I

Unrelated Integrated Learning Learning

III IV

Low Service

essential elements of service learning
ESSENTIAL ELEMENTS OF SERVICE-LEARNING

CLUSTER 1: LEARNING

1) CLEAR EDUCATIONAL GOALS

2) CHALLENGING EDUCATIONAL TASKS:

3) ONGOING ACADEMIC ASSESSMENT

CLUSTER 2: SERVICE

4) MEANINGFUL SERVICE TASKS

5) EVALUATION OF SERVICE OUTCOMES

From Essential Elements of Service-Learning. National Youth Leadership Council. Minneapolis, MN.

essential elements of service learning12
Essential Elements of Service-Learning

CLUSTER 3: CRITICAL COMPONENTS THAT SUPPORT SERVICE AND LEARNING

6) STUDENT VOICE

7) DIVERSITY OF PERSPECTIVES

8) OPEN AND EFFECTIVE COMMUNICATION

9) STUDENT PREPARATION

10) ONGOING AND CRITICAL REFLECTION

11) CELEBRATION

.

service learning and academic outcomes
Service-Learning and Academic Outcomes

Mediating Factors

Self-esteem

Empowerment

Prosocial behaviors

Motivation

Engagement

Academic

Outcomes

Service-

Learning

Clearly defined

programmatic features

measurements for student outcomes
MEASUREMENTS FOR STUDENT OUTCOMES
  • CART (http://cart.rmcdenver.com/)
slide16
Bringle, Phillips, and Hudson (2004):

Contains a series of validated scales used in service-learning studies.

measurements for student outcomes17
MEASUREMENTS FOR STUDENT OUTCOMES
  • CART (http://cart.rmcdenver.com/)
  • Bringle, Phillips, and Hudson (2004)
  • Higher Education S-L Survey (Civic Responsibility, Academic Attitude, Career Development, and Empowerment) (Furco, 2000)
  • CASQ (Civic Action, Social Justice, Appreciation of Diversity) (Moely et. al., 2002)
issues in s l student assessment
ISSUES IN S-L STUDENT ASSESSMENT
  • Student service preferences matter
issues in s l student assessment20
ISSUES IN S-L STUDENT ASSESSMENT
  • Student service preferences matter
  • Service-learning extends student learning
aspects of student learning in service learning
Aspects of Student Learning in Service-Learning

Civic Responsibility

Academic Achievement

Community-based

Learning

Experiences

LEARNING ABOUT

SERVICE

Connecting Learning

and Service

through Reflection

LEARNING ABOUT

THE SOCIAL ISSUE

Classroom-based

Learning

Experiences

LEARNING THE

COURSE CONTENT

…Service

…Learning

learning aspects of service learning
Students learn the process of photosynthesis, the development of nutrients in plants, and the life cycles of various types of plants.

.

Learning Aspects of Service-Learning

Plant Biology Edible Garden for the Homeless

Learning the Course Content

Students learn: to cultivate various edible plants; the nurturing that different types of plants require; assessment of plant quality; and garden care.

Learning about Service

Students learn: why and how individuals become homeless; the ways in which the homeless can benefit from being served the food from the gardent; the nature of homeless services in the local area, etc.

Learning about the Social Issue

assessing institutional outcomes
Assessing Institutional Outcomes
  • Institutionalization
  • Institutional climate (e.g. sense of community)
  • Learning communities
  • Faculty engagement in service-learning
  • Other
instruments to measure institutionalization engagement and service learning
Instruments to Measure Institutionalization(Engagement and Service-Learning)
  • Kellogg Forum Checklist
  • Committee on Institutional Collaboration Benchmarks
  • Furco Rubric for Institutionalizing Service-Learning
  • Holland Matrix on Relevance to Mission
  • Campus Compact Indicators of Engagement

7) Bringle et al.’s Comprehensive Assessment for the Scholarship of Engagement (CASE)

8) Carnegie Elective Classification for Community Engagement

kellogg forum dimensions of engagement
KELLOGG FORUM:DIMENSIONS OF ENGAGEMENT

A CHECKLIST of ten principles:

  • Access to Learning
  • Enhanced Diversity
  • Civic Leadership
  • Public Scholarship
  • Social Well-Being
  • Trusted Voice
  • Public Spaces
  • Community Partnerships
  • Self Governance
  • Public Accountability
cic benchmarks of engagement
CIC BENCHMARKS OF ENGAGEMENT

7 BENCHMARKS that show evidence of:

  • Institutional commitment to engagement
  • Institutional resource commitments
  • Student Involvement in engagement activities
  • Faculty and staff partnerships with community
  • Institutional engagement with community
  • Assessing impact & outcomes
  • Resource/Revenue opportunities generated

Committee on Institutional Cooperation

cic benchmarks of engagement27
CIC BENCHMARKS OF ENGAGEMENT

1. Evidence of Institutional Commitment to Engagement

1.1. The institution’s commitment is reflected throughout its administrative structure.

1.2. The institution’s commitment is reflected in its reward structure for faculty and staff.

1.3. The institution’s commitment is reflected in its policies and procedures designed to facilitate outreach and engagement activities.

1.4. The institution’s commitment is reflected in its policies and procedures that are responsive to students.

2. Evidence of Institutional Resource Commitments to Engagement

2.1. The institution shows evidence of leadership for engagement and outreach activities.

2.2. The institution shows evidence of financial support for engagement through its budgetary process.

2.3. The institution shows evidence that faculty and staff time is devoted to outreach and engagement activities.

2.4. The institution includes engagement activities as part of its programs for faculty, student and staff development.

campus compact indicators of engagement
CAMPUS COMPACT INDICATORS OF ENGAGEMENT

INDICATORS that suggest the presence of engagement at the institution:

  • Mission and Purpose
  • Academic and Administrative Leadership
  • Disciplines, Departments, and Interdisciplinary work
  • Teaching and Learning
  • Faculty Development
  • Faculty Roles and Rewards
  • Support Structures and Resources
  • Internal Budget & Resource Allocations
  • Community Voice External
  • Resource Allocation
  • Coordination of Community-Based Activities
  • Forums for Fostering Public Dialogue
  • Student Voice
operationalization of indicators of engagement
OPERATIONALIZATION OF INDICATORS OF ENGAGEMENT

A. Mission and Purpose

• The institution’s mission statement explicitly articulates its commitment to the public purposes of higher education and is deliberate about educating students for lifelong participation in their communities.

• This aspect of the mission is openly valued and is explicitly used to promote and to explain the civic engagement and community building activities on and off campus.

• The institution demonstrates a genuine willingness to review, discuss, and strengthen its commitment to civic engagement and community building.

• All members of the campus community demonstrate their familiarity with and ownership of the institution’s mission.

From Burack and Saltmarsh (2007), Advancing Civic Engagement through Strategic Assessment. NERCHE

furco self assessment rubric for service learning institutionalization
FURCO SELF-ASSESSMENT RUBRIC FOR SERVICE-LEARNING INSTITUTIONALIZATION

A RUBRIC built on 22 components that are organized within five dimensions:

  • Philosophy and mission of service-learning
  • Faculty support and involvement in service-learning
  • Student support for and involvement in service-learning
  • Community participation and partnerships
  • Institutional support for service-learning

Three stages: Critical mass building; Quality building; Sustained institutionalization

findings and lessons learned
FINDINGS AND LESSONS LEARNED

Based on data from 209 Colleges and Universities:

  • Overall Yr. 1 institutionalization level:  =1.6, s.d. = .21

Overall Yr. 3 institutionalization level:  = 2.1, s.d. = .31

Year 1-3 change:  = .50, p = 0.018

  • Estimated 3-5 years to move from one stage to next
  • Internal assessments are more positive than external assessments 3 out of 4 cases (n = 43) in Year 1.

Internal assessments are more negative than external assessments 3 out of 4 cases (n = 42) in Year 3.

findings and lessons learned33
FINDINGS AND LESSONS LEARNED
  • The self-assessment process is more important than the rubric score
  • The purposeful, ambiguous wording of the rubric encouraged discussion and dialogue

6) The components of the rubric are interdependent

7) All components do not have to be at Stage Three to achieve institutionalization

8) As you progress you may regress

strategies for evaluating community outcomes
STRATEGIES FOR EVALUATING COMMUNITY OUTCOMES
  • Involve community partners/members in the design of the evaluation
  • Narrow the impact area to isolate service-learning’s effect
  • Secure pre-post measures; benchmark change
  • Support personal stories with data
  • Use pictures to support the data