social work education assessment project sweap responding to the challenges of assessment n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Social Work Education Assessment Project ( SWEAP): Responding to the Challenges of Assessment PowerPoint Presentation
Download Presentation
Social Work Education Assessment Project ( SWEAP): Responding to the Challenges of Assessment

Loading in 2 Seconds...

play fullscreen
1 / 69

Social Work Education Assessment Project ( SWEAP): Responding to the Challenges of Assessment - PowerPoint PPT Presentation


  • 202 Views
  • Uploaded on

Social Work Education Assessment Project ( SWEAP): Responding to the Challenges of Assessment. Pre-Conference Workshop BPD Annual Conference Louisville, KY March 19, 2014. Workshop Outline. Introductions Introducing SWEAP /Goodbye BEAP The SWEAP Team Workshop Participants

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Social Work Education Assessment Project ( SWEAP): Responding to the Challenges of Assessment' - liz


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
social work education assessment project sweap responding to the challenges of assessment

Social Work Education Assessment Project (SWEAP): Responding to the Challenges of Assessment

Pre-Conference Workshop

BPD Annual Conference

Louisville, KY

March 19, 2014

workshop outline
Workshop Outline
  • Introductions
    • Introducing SWEAP /Goodbye BEAP
    • The SWEAP Team
    • Workshop Participants
  • Overview of Accreditation
    • History of Accreditation
    • Direct vs. Indirect Measures
    • Implicit Curriculum
    • Multiple Measures
  • Organizational Framework for Program Assessment
    • EPAS 2008 (&2015)
    • Competencies
    • Characteristic Knowledge, Values & Skills
    • Practice Behaviors
  • SWEAP Instruments
    • Entrance
    • Exit
    • Alumni/Graduate
    • Employer
    • Curriculum Instrument (FCAI)
    • Field Instrument (FPPAI)
  • BREAK
workshop outline continued
Workshop Outline(continued)
  • Unique Benefits of SWEAP
  • Tying SWEAP to Program Assessment
    • Linking SWEAP to EPAS
    • Matrix
    • Program Integration Example
    • Using SWEAP beyond EPAS
  • Sharing Assessment Ideas
    • Group Work
  • Navigating SWEAP
    • Website
    • Ordering
    • Raw Data Policy
    • Processing
    • Reports
  • Questions
introducing sweap
Introducing SWEAP
  • Was BEAP
  • Now SWEAP
    • Not just for undergraduate programs anymore
    • Can assess foundation year for graduate programs as well
    • Can be modified for program specific advanced year assessment
the sweap team
The SWEAP Team

Kathryn Krase

LIU Brooklyn

kathryn.krase@liu.edu

Phil Ng

BEAP@PHILNG.NET

Patrick Panos

University of Utah

patrickpanos@gmail.com

Roy (Butch) RodenhiserBoise State UniversityRoyRodenhiser@boisestate.edu

Vicky BuchanColorado State Universityvictoria.buchan@colostate.edu

Brian ChristensonLewis-Clark State Collegeblchristenson@lcsc.eduTobi DeLong HamiltonLewis-Clark State College, CDA

tadelong-hamilton@lcsc.edu

Ruth Gerritsen-McKaneUniversity of Utahruth.gerritsen-mckane@socwk.utah.edu

Sarah Jackman

Sarah.jackman@socwk.utah.edu

introducing workshop participants
Introducing Workshop Participants
  • Your Name
  • Your Program
  • Your Role
  • Have you used SWEAP?
    • If so, which instrument(s)?
overview of accreditation history of cswe accreditation
Overview of Accreditation:History of CSWE Accreditation
  • What was the process like before BEAP?
  • How has it changed over time?
  • How do changes to EPAS impact it?
  • Move to competency based educational standards in EPAS 2008
overview of accreditation direct vs indirect measures
Overview of Accreditation:Direct vs. Indirect Measures
  • Direct Measures
    • “Student products or performances that demonstrate that specific learning has taken place.”
    • Examples (SWEAP and Non-SWEAP)
  • Indirect Measures
    • “May imply that learning has taken place (e.g., student perceptions of learning) but do not specifically demonstrate that learning or skill.”
    • Examples (SWEAP and Non-SWEAP)
  • Which do you need and why?
overview of accreditation implicit curriculum
Overview of Accreditation:Implicit Curriculum
  • What is Implicit Curriculum?
    • “Educational environment in which the explicit curriculum is presented. It is composed of the following elements: the program’s commitment to diversity; admissions policies and procedures; advisement, retention, and termination policies; student participation in governance; faculty; administrative structure; and resources.” (EPAS, 2008)
    • How do you measure it?
    • Why should you measure it?
slide10

Overview of Accreditation:Multiple Measures

  • The Importance of Multiple Measures
  • SWEAP alone is not enough
organizational framework for program assessment epas 2008 2015
Organizational Framework for Program AssessmentEPAS 2008 (& 2015)
  • Working under EPAS 2008
  • EPAS 2015 underway… expected changes
    • Fewer competencies
    • Fewer practice behaviors
organizational framework for program assessment competencies
Organizational Framework for Program AssessmentCompetencies
  • EPAS 2.1—Core Competencies
    • Competency-based education
    • Outcome performance approach to curriculum design.
    • Measurable practice behaviors comprised of knowledge, values, & skills.
    • Need to demonstrate integration & application competencies in practice with individuals, families, groups, organizations, and communities.
    • 10 competencies listed along with description of characteristic knowledge, values, skills,& practice behaviors that may be used to operationalize the curriculum and assessment methods.
    • Programs may add competencies consistent with their missions and goals.
organizational framework for program assessment competencies1
Organizational Framework for Program AssessmentCompetencies
  • 2.1.1—Identify as a professional social worker and conduct oneself accordingly.
  • 2.1.2—Apply social work ethical principles to guide professional practice.
  • 2.1.3—Apply critical thinking to inform and communicate professional judgments.
  • 2.1.4—Engage diversity and difference in practice.
  • 2.1.5—Advance human rights and social and economic justice.
  • 2.1.6—Engage in research-informed practice and practice-informed research.
  • 2.1.7—Apply knowledge of human behavior and the social environment.
  • 2.1.8—Engage in policy practice to advance social and economic well-being and to deliver effective social work services.
  • 2.1.9—Respond to contexts that shape practice.
  • 2.1.10— Engage, assess, intervene, and evaluate with individuals, families, groups, organizations, and communities.
organizational framework for program assessment characteristic knowledge values skills
Organizational Framework for Program Assessment: Characteristic Knowledge, Values & Skills
  • Current focus on measuring practice behaviors.
  • Don’t forget about knowledge, values & skills.
organizational framework for program assessment practice behaviors
Organizational Framework for Program AssessmentPractice Behaviors
  • Multiple practice behaviors per competency
  • Each practice behavior MUST be measured for self-study/reaccreditation
  • TWO measures required for each practice behavior
    • At least one measure must be DIRECT
sweap instruments what are they and how do i use them
SWEAP InstrumentsWhat are they and how do I use them?
  • Entrance
  • Exit
  • Alumni/Graduate & Employer
  • Curriculum Instrument (FCAI)
  • Field Instrument (FPPAI)
entrance purpose
EntrancePurpose
  • Provides demographic profile of entering students.
  • Completed at time of entrance into the program (Program Defined).
  • Provides overview of financial resources students are using or plan to utilize.
  • Provides employment status & background information regarding both volunteer and paid human service experience.
  • Helps track planned or unplanned changes in the profile of students in the program.
  • Evaluate impact of policy changes, such as admissions procedures, over time.
slide18

EntranceQuestions

  • Student tracking: ID number & Survey completion date
  • Gender
  • Year in school
  • Overall GPA & GPA in major & Highest possible GPA at school
  • Length of current social work-related work experience (volunteer & paid)
  • Citizenship/ length of residence in USA
  • Employment plans during social work education
  • Hours per week expected to work during education
  • Sources of financial aid expected
  • Language fluency
  • Expected date of graduation
  • Race/ Ethnicity
  • Disabilities/Accommodation
exit purpose
ExitPurpose
  • Completed by students just prior to graduation.
    • Often administered in field seminar or capstone seminar.
  • Feedback from students about their experiences while in the program.
  • Addresses:
    • Evaluation of curriculum objectives based on EPAS.
    • Post-graduate plans, related to both employment and graduate education, are addressed.
  • Collects demographic information to compare with the entrance profile.
exit questions
ExitQuestions
  • Educational Experience
    • Including implicit curriculum assessment
  • Current Employment
  • Employment-seeking activities
  • Current & anticipated Social Work Employment
    • Primary function & major roles
  • Post Graduate Educational Plans
exit questions1
ExitQuestions
  • Students Evaluate how well program prepared them to perform practice behaviors
  • Professional Activities
    • Use of research techniques to evaluate client progress & Use of program evaluation methodology
  • Personal Demographic Information
    • Gender, Citizenship, Language fluency, & Disabilities
alumni graduate purpose
Alumni/GraduatePurpose
  • Intended for completion two years after graduation
    • Standardized timing for administration is essential to create a reliable dataset for comparison over time.
  • Alumni evaluate how well program prepared them for professional practice.
  • Alumni employed in social work and those not employed in social work are surveyed.
  • Also gathers information on current employment, professional development activities, and plans/ accomplishments related to further education.
alumni graduate questions
Alumni/GraduateQuestions
  • Current Employment
  • Current Social Work Employment
  • Evaluation of preparation by the Program in the 10 EPAS competency areas (using Likert-type scale)
  • Educational Activities
  • Professional Activities
  • Demographics
employer purpose
EmployerPurpose
  • Intended for completion two years after graduation
  • Addresses both accreditation and university concern for feedback from the practice community.
  • Measures graduate’s preparation for practice based on supervisor’s assessment.
  • Alumni/aerequest employer complete the survey:
    • Addresses primary concern of confidentiality.
    • Use of student identified allows connection to other instruments.
employer questions
EmployerQuestions
  • Educational background of supervisor/employer
  • Twelve (12) items which evaluate alumni/ae proficiency in all EPAS competencies.
curriculum instrument fcai purpose
Curriculum Instrument (FCAI)Purpose
  • Provides Pre/Post test in seven major curricular areas of the foundation year.
  • Provides a direct measure to assist programs with evaluation of their curriculum.
  • Assists with identification of curricular areas that may need attention.
  • Provides national comparative data.
sample hbse question
Sample HBSE Question
  • The concept “person-in-environment” includes which of the following:
    • Clients are influenced by their environment
    • Clients influence their environment
    • Behavior is understood in the context of one’s environment
    • All of the above
sample practice question
Sample Practice Question
  • Determining progress toward goal achievement is one facet of the _____ stage.
    • a. Engagement
    • b. Evaluation
    • c. Assessment
    • d. Planning
reliability testing
Reliability Testing
  • Version 9
    • Tested in two junior practice classes
    • Students tested twice, 2 weeks apart
    • Pearson’s correlation coefficient
      • r = .86
item difficulty index
Item difficulty index
  • Overall difficulty or average should be around .5 (Cohen & Swerdlik, 2005)
  • FCAI = .523 (n=415)
  • “This is a very good difficulty level for the test.

Not likely to misrepresent the knowledge level of test takers”.

reliability effect size
Reliability & Effect Size
  • Cronbach’s alpha = .784
  • Effect Size d = 6.87
current data 1 13 to 9 13
Current Data1/13 to 9/13
  • Number of Schools using the FCAI in 2013

110

  • Number of Respondents in 2013:
    • Pre-test: 8432
    • Post-test: 5369
expansion beyond bsw
Expansion beyond BSW
  • Based upon CSWE assertions related to educational levels in social work education, we expanded testing to three additional groups:
    • MSW foundation students:
      • entering
      • exiting
    • Advanced standing students:
      • entering
points to keep in mind about the faci
Points to keep in Mind about the FACI

1. Purpose of this instrument: to review and improve curriculum

2. Program will want to “monitor” scores over several years (or several cohorts) for trends.

3. The FCAI can be considered a measure of “value added” from program entry to exit.

4. Benchmarks: can be set two ways,

a. by competency

b. overall score

slide43

Field Practicum/Placement Assessment Instrument

(FPPAI)

  • Responds to need for a standardized field/practicum assessment instrument that measures student achievement of practice behaviors.
slide44

Field Instrument (FPPAI)

Piloting Phase

  • Initial Piloting for BSW in May 2008
  • Second Pilot in Fall 2008/Spring 2009
  • Third Pilot in Fall 2009
  • Reliability Analysis
    • Chronbach’s Alpha of 0.91 or higher in each practice behavior
  • Full implementation in BSW: Fall 2010
  • Piloting in MSW: Spring-Fall 2012
  • Full implementation in MSW (Foundation): Fall 2013
slide45

Field Instrument (FPPAI)

Methodology

  • 58 Likert Scale questions measuring practice behaviors linked to the EPAS 2008 competencies.
  • Qualitative feedback form for each domain available for program use.
  • Available online and in print format.
  • Individual program outcomes report with national comparisons available.
  • Individual program outcomes report with national comparisons for EPAS 2008 Competencies & Practice Behaviors including CSWE benchmark reporting.
  • Can be used as a final field assessment and mid-test/post test design.
slide47

Field Instrument (FPPAI)

Current Status

  • 84 programs currently use this instrument
  • More than 3,132 administrations to date
  • National data comparisons are available
  • Cronbach’s Alpha reliability test of internal consistency at midpoint (Pilot and first two years Testing): 0.969
  • Cronbach’s Alpha reliability test of internal consistency at final (Pilot and first two years testing): 0.975
slide49

SWEAP

Unique Benefits

  • Student demographics
  • Numerous data points for comparison
  • Explicit and Implicit curriculum assessment
  • Doesn’t end at graduation
  • Peer comparison by region, program type, auspice & nationally
  • …and more
slide50

Tying SWEAP

to Program Assessment

Linking SWEAP to EPAS

  • All instruments updated to reflect 2008 EPAS
  • All instruments will be updated and available as soon as 2015 EPAS goes into effect
  • Competency Matrix for 2008 EPAS(Handout)
slide51

Tying SWEAP

to Program Assessment

slide53

Tying SWEAP

to Program Assessment

Program Integration Example

  • Program Integration Example (Handout)
slide54

Tying SWEAP

to Program Assessment

Using SWEAP Beyond EPAS

  • Curricular Development
  • Addressing Implicit Curriculum Concerns
  • Understanding student body
    • Changes in student body over time
slide55

Sharing Assessment Ideas

Break into small groups, tackle the assigned competency discussing the following:

How can/do you measure competencies in your programs?

Offer SWEAP & Non-SWEAP options

slide56

Sharing Assessment Ideas

Break into small groups, tackle the assigned competency discussing the following:

How can/do we measure

Implicit Curriculum?

Offer SWEAP &

Non-SWEAP Measures

slide57

Navigating SWEAP

The Website

http://sweap.utah.edu

  • Navigating the site
  • Locating the results of your instruments
navigating sweap ordering
Navigating SWEAPOrdering
  • Online through website.
  • Pay by credit-card, or send check.
  • Receive email with .pdfs to be printed, or links to online instruments.
  • Printing .pdfs
    • Can be double-sided, BUT DO NOT STAPLE

For questions about yourorder or status:

sarah.jackman@socwk.utah.edu

online instruments
Online Instruments
  • All SWEAP instruments are available in electronic format.
  • eTickets (electronic tickets) are simple to use. You order them via the SWEAP order page on our website, just like you would order instruments to print on paper.
  • Each eTicket is an individual instrument and cannot be copied.
  • The simplest way to administer instruments with eTickets is to send an email to each student, field educator or alumnus/a who is expected to complete the instrument. In that email, you provide an individual eTicket link.
  • The respondent enters the letters and numbers that follow the equal sign (=) on the eTicket link. For instance, to access the email example, the code to enter in the pink box is RITNFN26197 (see next slide).
slide64

Navigating SWEAP

Raw Data Policy

  • We offer return of de-identified data to programs with the following data removed:
    • Year of Birth
    • Day of Birth
    • Parts of Social Security Numbers
    • ZIP code of Employment or Residence.
    • Sex / Gender
    • Ethnic Identity
  • In addition, specific answers to the FCAI instruments will not be returned, nor specific answers to the Values at Exit or Entrance. These will be reported in an aggregate only.
  • FPPAI instruments, due to their nature, will return all identifying information.
slide65

Navigating SWEAP

Raw Data Policy

  • There will be a surcharge to utilize this service, it will be a one time charge of $25.00 to have all permissible data returned to the program.
slide66

Navigating SWEAP

Processing

  • Paper instruments need to be mailed to Utah.
    • Processing involves scanning forms into system and then calculating data.
    • Can take up 3-6 weeks, depending on time of year.
  • Online instruments
    • Processing is very quick.
    • Can be days, instead of weeks.
    • No need to be scanned at our end.
slide67

Navigating SWEAP

Reports

  • Receive email when reports are ready.
  • Retrieve reports through website.
  • Can request reports with tailored comparisons.
coming in 2015
Coming in 2015!
  • Bridging the gap for direct measures: New direct measures in-line with 2015 EPAS
  • Outcome Reports on your webpage as required by CSWE
  • All current instruments updated for EPAS 2015