Developing an outcomes assessment program
This presentation is the property of its rightful owner.
Sponsored Links
1 / 50

Developing an Outcomes Assessment Program: PowerPoint PPT Presentation


  • 33 Views
  • Uploaded on
  • Presentation posted in: General

Developing an Outcomes Assessment Program:. The CCBC Approach. Dr. Irving Pressley McPhail, Chancellor The Community College of Baltimore County Delgado Community College – February 18, 2005. LearningFIRST 2.0. The Strategic Plan For CCBC FY 2004 to FY 2008.

Download Presentation

Developing an Outcomes Assessment Program:

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Developing an outcomes assessment program

Developing an Outcomes Assessment Program:

The CCBC Approach

Dr. Irving Pressley McPhail, Chancellor

The Community College of Baltimore County

Delgado Community College – February 18, 2005


Learningfirst 2 0

LearningFIRST 2.0

The Strategic Plan For CCBC

FY 2004 to FY 2008


Learning first 2 0

Learning First 2.0

Learning

Support

Learning College

Effective

Communication

Infusing

Technology

Student

Learning

Enrollment

Management

Organizational Excellence

Community

& Institutional

Advancement

Valuing

Diversity


Ccbc a vanguard learning college

CCBC: A Vanguard Learning College

  • One of twelve community colleges from across the U.S. and Canada chosen to participate in the League’s Vanguard Learning College Project

  • Five areas of concentration:

    • Organizational Culture

    • Staff Recruitment and Development

    • Learning Outcomes Assessment

    • Technology

    • Underprepared Students


Middle states standard 14 assessment of student learning

Middle States Standard 14: Assessment of Student Learning

Assessment of student learning

demonstrates that the institution’s

students have knowledge, skills, and

competencies consistent with

institutional goals and that students at

graduation have achieved appropriate

higher education goals.


Middle states standard 14 assessment of student learning1

Middle States Standard 14: Assessment of Student Learning

In order to carry out meaningful assessment activities, institutions must articulate statements of expected student learning at the institutional, program, and individual course levels, although the level of specificity will be greater at the course level. Course syllabi or guidelines should include expected learning outcomes.


Sacs principles of accreditation

SACS – Principles of Accreditation

  • 3.3Institutional Effectiveness

    • 3.3.1 The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.


Council on innovation and student learning cisl

Council on Innovation and Student Learning(CISL)

How it all began!!


Chancellor s charge to cisl

Chancellor’s Charge to CISL

  • Serve as a college-wide think tank

  • Lead the transformation of the CCBC into a premier, learning centered college

  • Serve as change agents

  • Help to frame policies, procedures and infrastructure needed to become a learning college


Getting started fall 1998

Getting Started - Fall 1998

  • Council on Innovation and Student Learning

    • Membership included trustees, chancellor, faculty, professional staff and classified staff

      • (35-40 members)

    • Assessment subcommittee created the Learning Outcomes Assessment Plan for the College

      • (8 members)

    • Guide to Learning Outcomes Assessment and Classroom Assessment (updated Spring 2003)

      • www.ccbcmd.edu/loa/index.html


Measuring student learning

Measuring Student Learning


Measuring student learning1

Non-measures

Student satisfaction surveys

Program evaluation

MHEC reporting

Student grades

Retention rates

Graduation rates

Transfer rates

Indirect measures

Exit interviews of graduates

Employer surveys

Transfer studies

Measuring Student Learning


Measuring student learning2

Measuring Student Learning

  • Direct measures include:

    • Standardized tests

    • Portfolio assessment

    • Capstone experience

    • Locally developed tests

    • Externally reviewed exhibitions and performances


Ccbc s learning outcomes assessment program

CCBC’s Learning Outcomes Assessment Program


Ccbc s assessment plan principles

CCBC’s Assessment Plan Principles

  • Primary reason for assessment is to improve and expand student learning

  • Development of an effective program is a long-term, dynamic process

  • Must involve a multi-method approach

  • Must include training and support for faculty and staff

  • Results are not used punitively for students or faculty


Ccbc s assessment plan principles1

CCBC’s Assessment Plan Principles

  • Seek to use the most reliable, valid methods and instruments available

  • Never an end in itself, only a means to an end – the improvement of student learning

  • Continuous Quality Improvement

Design

Implement

Assess


Ccbc s outcomes assessment philosophy

CCBC’s Outcomes AssessmentPhilosophy

Assessment is:

  • A natural and on-going part of instruction

  • Consistent with traditional instructional practices

  • Designed to meet specific objectives

  • Conducted in a risk-free environment


Loa project collaboration

LOA Project Collaboration

  • Learning Outcomes Associate

  • Planning, Research and Evaluation Liaison

  • Learning Outcomes Assessment Advisory Board

  • Deans’ Council

  • General Education Review Board

  • Developmental Education Advisory Committee


Loa project collaboration1

LOA Project Collaboration

  • Learning Outcomes Associate

    • Development of design

    • Survey/tool development

    • Staff development

    • Logistical plan

  • Institutional Research

    • Data entry

    • Data analysis

    • Data interpretation


Types of designs

Types of Designs

  • Portfolio assessment

  • Standardized tests

  • External graders/experts

  • Pre- and post-tests

  • Cooperation with other schools

  • Creativity Abounds!


Project elements

Project Elements

  • Stage 1:

    • Design Course Project

      • Determine Measurable Objectives

      • Select Assessment Instrument

      • Include External Validation

      • Control Important Variables

    • Develop Full Proposal


Project elements1

Project Elements

  • Stage 2:

    • Implement Design

    • Administer Assessment Instrument(s)

    • Collect Data

    • Analyze Data


Project elements2

Project Elements

  • Stage 3:

    • Test Objectives

      Met? > Write Final Report

      Not Met?>Design Course Improvements Write Interim Report


Project elements3

Project Elements

  • Stage 4:

    • Implement Course Improvements

    • Collect Data

    • Analyze Data

  • Stage 5:

    • Final Report

      • Discipline/Campus

      • College


In summary

In Summary

  • Establish Climate of Continuous Improvement

  • Empower Faculty

  • Provide Substantial Administrative and Fiscal Support

  • Share Results - Celebrate Successes


Course level assessment

Course Level Assessment


High impact loas

CINS 101

HLTH 101

PEFT 101

SDEV 101

ENGL 101

MGMT 101

SOCL 101

PHIL 101

BIOL 110

SPCM 101

High Impact LOAs

FY ‘03

FY ‘04

FY ‘05

  • ARTS 104

  • ENVS 101

  • PSYC 105

  • MATH 082


Hlth 101 health and wellness

HLTH 101: Health and Wellness

  • High impact project

  • Pre-test/post-test design

  • 100 item, four option multiple choice format exam

  • Topics included subjects commonly covered in introductory health textbooks

  • Draft instrument was reviewed by a health/curriculum expert at a four-year university

  • Two and four-year colleges and universities have been invited to share our instrument so that we might develop external comparisons


Health 101 loa results

Health 101 LOAResults

  • Matched pair t-test analysis

  • Students’ scores improved significantly from pre- to post-test

  • At least 75% of students scored a 75% or higher on the post-test

  • Scores varied by campus; small difference in scores based on race

  • Reassessment will be conducted after recommendations have been implemented


Health 101 loa recommendations

Health 101 LOA Recommendations

  • Three topic areas were identified as needing improvement: nutrition, heart disease, and human sexuality

  • The campus with students who consistently scored higher on the pre-test will offer more honors’ sections

  • A faculty guide, “Strategies for Teaching Health Education” has been developed and shared with full-and part-time faculty


Rdng 052 college reading

RDNG 052: College Reading

  • Instruments

    • Used the Nelson-Denny Reading Test (Forms G and H) pre and post for the 2001 and the 2003 assessments

    • Used the Learning Attitudes Study Strategies Inventory (LASSI) for the two semesters of pilot projects and the 2001 assessment

  • Design

    • Pre-post project design

    • Matched pair data analysis


Rdng 052 college reading fall 2001 results

RDNG 052: College Reading - Fall 2001 Results

  • Mean pre-post difference of 3.5 (raw points) was statistically significant at the .001 level

  • Post-test score was greater than pre-test score in 74% of cases

  • Resulting grade level increase of .7 grade level (in a 15 week semester)


Rdng 052 college reading fall 2001 results1

RDNG 052: College Reading - Fall 2001 Results

  • Exiting students read at 9.6 mean grade level

  • Differences among campuses—learned from best practices on each campus

  • LASSI results showed significant improvement in 10 of 12 scales from pre to post


Rdng 052 college reading fall 2001 interventions

RDNG 052: College Reading – Fall 2001 Interventions

  • Interventions included:

    • Using more Internet-based learning opportunities

    • Enhancing instructional activities that focused on literal and inferential comprehension

    • Greater focus on working with adjunct faculty


Rdng 052 college reading fall 2003 results

RDNG 052: College Reading – Fall 2003 Results

  • Mean pre-post difference of 3.7 was statistically significant at the .001 level

  • Post-test score was greater than pre-test score in 75% of cases (larger sample)

  • Resulting grade level increase of 2.0! (in a 15 week semester)

  • Exiting students read at 10.1 mean grade level

  • Differences among campuses; every campus improved


Rdng 052 2001 to 2003 improvements in student learning

RDNG 052 - 2001 to 2003: Improvements in Student Learning

  • Significant improvements in students’ reading levels from 2001 to 2003 indicate increased learning in RDNG 052.


Rdng 052 college reading final recommendations

RDNG 052 College Reading - Final Recommendations

  • The Cycle of Continuous Improvement

    • Research Accuplacer cut scores between RDNG 051 and RDNG 052

    • Continue to close the performance gap

    • Continue to share best practices

    • Continue to study post-Reading success rates (success in subsequent courses)


Making the transition from course to program level assessment

Making the Transition from Course to Program Level Assessment

  • Program Review

  • General Education

  • Developmental Education


General education

General Education

The Academic Profile

And

The “GREATs”


The academic profile

The Academic Profile

  • Norm-referenced, externally developed test of general knowledge

  • Widely used

  • Sample size: 1,017 Students

  • Purpose - to establish baseline data upon implementation of new General Education Program in Fall 2001


Academic profile findings

Academic Profile - Findings

  • The Academic Profile provides CCBC with a baseline measure of how our students are acquiring academic skills developed through General Education courses compared to national norms.

    • CCBC students scored at or close to the national norms on the total mean score and all sub-scores the first time the test was administered.

    • The Academic Profile will be administered again in Fall 2004 (sample size 2,000+).


Academic profile data implications

Academic Profile –Data Implications

  • Work on Critical Thinking skills

  • Reinforce skills learned in one class in other classes

  • Provide Culturally Mediated Instruction

  • Possible statewide initiatives


Great projects

GREAT Projects

  • General Education Assessment Teams (GREAT) Projects for all Maryland Higher Education Commission general education categories

  • Common Graded Assignments (CGA) and accompanying Scoring Rubrics

    • Created by faculty teams

  • External Consultant

  • Faculty Training

  • Three semesters of Pilot Projects

  • Full implementation began in Fall 2003

  • Every course assessed once every three years


Great data fall 2003 implications

GREAT Data (Fall, 2003) Implications

  • Increased awareness of faculty regarding how General Education courses are defined by the six criteria

  • Midrange to higher scores in Content may indicate a traditional approach in how “Content” is conveyed with little expectation for higher level use of content in application, critical thinking, analysis, and synthesis activities

  • Workshops on how to use the GREAT data to improve student learning


Loa successes

LOA Successes

  • Creating a “culture of assessment” with increased faculty participation and buy-in

  • Using outcomes assessment for Program-level assessment

  • Forging new partnerships between faculty teams, institutional research staff, the Vice Chancellor for Learning and Student Development’s office, and the Outcomes Associate in analyzing data and making curricular and pedagogical recommendations for change


Loa successes1

LOA Successes

  • Creating a newly updated Guide for Learning Outcomes Assessment and Classroom Learning Assessment, a model guide for assessment

    • www.ccbcmd.edu/loa/index.html

  • Establishing the Learning Outcomes Assessment Advisory Board

  • Recognition by the League for Innovation in the Community College and Middle States Commission on Higher Education as a national leader in Learning Outcomes Assessment


  • For further information

    For Further Information:

    Dr. Irving Pressley McPhail

    Chancellor

    The Community College of Baltimore County

    800 South Rolling Road

    Baltimore, Maryland 21228-5317

    Telephone: 410-869-1220

    Fax: 410-869-1224

    Email:[email protected]

    Website:www.ccbcmd.edu


    For further information1

    For Further Information:

    Dr. Alvin Starr, Acting Vice Chancellor for Learning and Student Development

    Email: [email protected]

    Dr. Rose Mince, Assistant to the VCLSD

    Email:[email protected]

    Professor Tara Ebersole, Outcomes Associate

    Email:[email protected]


    Questions comments

    Questions /Comments


  • Login