Association of Institutional Research
This presentation is the property of its rightful owner.
Sponsored Links
1 / 46

May 2010 PowerPoint PPT Presentation


  • 68 Views
  • Uploaded on
  • Presentation posted in: General

Association of Institutional Research 50 th Annual Forum. FAIR Best Paper: Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers. May 2010. Alumni and Employer Surveys. Presenters

Download Presentation

May 2010

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


May 2010

Association of Institutional Research

50th Annual Forum

FAIR Best Paper: Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers

May 2010


Alumni and employer surveys

Alumni and Employer Surveys

Presenters

  • Dr. James Coraggio, Director of Academic Effectiveness and Assessment

    Other Authors Not Present

  • Maggie Tymms, Assessment Coordinator for Academic Programs

  • Nicole Carr, Former Assessment Coordinator for Academic Programs

Association of Institutional Research


Spc background

SPC Background

  • SPC, established in 1927, is the oldest

    2-year college in Florida

  • First Community College in Florida to offer 4 year degrees (2002)

  • 9 Campuses throughout the county

  • 2008-09 FTE: 16,966.8 (LD), 1,676.7 (UD)

  • Opening Fall 2009 credit enrollment: 29,560

  • Annual 2008-09 headcount: 61,414

Association of Institutional Research


Institutional effectiveness

Institutional Effectiveness

“Closing the Loop”

“Institutional Effectiveness is the integrated, systematic, explicit, and documented processes of measuringperformanceagainst the SPC mission for purposes of continuousimprovement of academic programs, administrative services, and educational services offered by the college.”

Association of Institutional Research


Performance improvement

Performance Improvement

  • From Compliance to Performance Improvement

Association of Institutional Research


Agenda

Agenda

  • Student Assessment Points

  • Purpose of Follow-up Surveys

  • Alumni/Employer Survey Process

  • Dissemination of Survey Results and Reports

Association of Institutional Research


Student assessment points

Student Assessment Points

Students surveyed multiple times

Association of Institutional Research


Purpose of follow up surveys

Purpose of Follow-up Surveys

Recent Alumni Survey was designed to:

  • Gain insight into alumni’s perceptions of the quality of general education and career preparation

  • Determine the current activities of former students such as employment information and educational status

Association of Institutional Research


Purpose of follow up surveys1

Purpose of Follow-up Surveys

Employer Survey was designed to:

  • Gain insight into an employer’s perceptions of hired SPC graduates in the areas of key general education competencies and foundational skills

  • Provide the employee’s major job responsibilities and wage information, as well as employer’s willingness to hire another SPC graduate

  • Identify employer’s willingness to support various college activities

Association of Institutional Research


Purpose of follow up surveys2

Purpose of Follow-up Surveys

State Requirements

  • Survey data are also collected for state required annual Placement & Follow-up Report (FETPIP: Florida Education Training Placement Information Program) and supplemental information to meet workforce criteria

Association of Institutional Research


Alumni employer survey process

Alumni/Employer Survey Process

Previous model (Before 2005)

  • Labor intensive

  • Manually tabulated paper surveys

  • Telephone interviews conducted using overtime hours

Association of Institutional Research


Alumni employer survey process1

Alumni/Employer Survey Process

Issues with previous model (Before 2005)

  • Costly survey process

  • Declining college budget

  • Increasing demand for accountability

  • Outdated contact information

  • Students over-surveyed

  • Declining response rates

  • Conducted in isolation

Association of Institutional Research


Alumni employer survey process2

Alumni/Employer Survey Process

Goals of new survey process

  • Improve data collection process

  • Enhance quality of survey forms

  • Employ industry best practices

  • Include discipline specific information

  • Increase response rates

Association of Institutional Research


A improve data collection process

A. Improve Data Collection Process

Data Collection Process

  • Eliminated telephone calls

  • Automated the process (Remark Office OMR and email merge)

  • Employed online survey submission (ZoomerangTM)

  • Obtained accurate email addresses

  • Used multiple points of contact

Association of Institutional Research


A improve data collection process1

A. Improve Data Collection Process

Survey Schedule

  • Recent Alumni Survey

    • Graduates surveyed six months after the end of their last term (Three times per year)

  • Employer Survey

    • Employer of graduates surveyed at the end of academic year (Once per year)

Association of Institutional Research


A improve data collection process2

A. Improve Data Collection Process

First email Second emailThird email

2 weeks

2 weeks

2 weeks

Recent Alumni Survey Multiple Points of Contact

2 weeks

First mailing

Second mailing

Association of Institutional Research


B enhance quality of survey forms

B. Enhance Quality Of Survey Forms

Improved Survey Forms

  • Better ‘look and feel’ of survey format

  • Developed scannable paper forms

  • Used online branching models to reduce survey length

  • Designed items to gather only pertinent data

Association of Institutional Research


C employ industry best practices

C. Employ Industry Best Practices

Don Dillman’s Five Elements

  • Respondent-friendly questionnaire

  • Up to five contacts with the respondent

  • Inclusion of stamped envelope

  • Personalized correspondence

  • A token financial incentive that is sent with the survey request

Association of Institutional Research


Online survey email and form

Online Survey: Email and Form

Association of Institutional Research


Paper survey letter and form

Paper Survey: Letter and Form

Association of Institutional Research


C include program specific information

C. Include Program Specific Information

Program specific information

  • Met with Deans/Program Directors

  • Discussed requirements for program accreditation and timelines

  • Included their necessary program specific items

Association of Institutional Research


D increase response rates

D. Increase Response Rates

Create awareness and convey importance

  • Involved program faculty in the process

  • Requested faculty assistance to inform students

  • Included a flyer with graduation materials

  • Coordinated with programs that hand-carry surveys to local employers

Association of Institutional Research


Alumni survey response rates

Alumni Survey Response Rates

  • Summer 2007

Association of Institutional Research


Alumni survey response rates1

Alumni Survey Response Rates

  • Fall 2007

Association of Institutional Research


Alumni survey response rates2

Alumni Survey Response Rates

  • Spring 2008

Association of Institutional Research


Alumni survey response rates3

Alumni Survey Response Rates

  • Summer 2008

Association of Institutional Research


Alumni survey response rates4

Alumni Survey Response Rates

  • Fall 2008

Association of Institutional Research


Alumni survey response rates5

Alumni Survey Response Rates

  • Spring 2009

Association of Institutional Research


Alumni survey response rates6

Alumni Survey Response Rates

  • Summer 2009*

*Summer 2009 in progress

Association of Institutional Research


Alumni survey response rates7

Alumni Survey Response Rates

*Summer 2009 in progress

Association of Institutional Research


Alumni survey response rates8

Alumni Survey Response Rates

Source of Responses 2008-09

Association of Institutional Research


Alumni survey response rates9

Alumni Survey Response Rates

*Summer 2009 in progress

Association of Institutional Research


Alumni survey response rates10

Alumni Survey Response Rates

Overall Response Rate

*Summer 2009 in progress

Association of Institutional Research


Alumni survey response rates11

Alumni Survey Response Rates

Note: For 2009-10, only partial Summer 2009 results have been received.

Association of Institutional Research


Response rates employer

Response Rates - Employer

Recent Alumni Surveys Sent

Is a 44% Employer Response Rate Good?

Employer Surveys Sent

COE

Responses

Permission Not Required

Other

Responses

Responses

Permission Required

Association of Institutional Research


Disseminating information

Disseminating Information

  • College wide access to reports

    • http://www.spcollege.edu/Central/Surveys/SurveyList.htm

  • Program Specific Reports

    • Published on the Educational Outcomes website

    • http://it.spcollege.edu/edoutcomes

Association of Institutional Research


Disseminating information1

Disseminating Information

Association of Institutional Research


Disseminating information2

Disseminating Information

Association of Institutional Research


Disseminating information3

Disseminating Information

Association of Institutional Research


Program specific reports

Program Specific Reports

Association of Institutional Research


Recent improvements

Recent Improvements

  • Survey invitations from program administrators (signatures)

  • Additional program specific reports (i.e., baccalaureate programs and PSAV)

  • Improved oversight (and QC) of printing functions

Association of Institutional Research


Lessons learned

Lessons Learned

  • Do not expect that students will remember their ID#s

  • Administer the survey soon after the end of the session (six months)

  • Use multiple points of contact (two weeks)

  • Find the most recent contact information

  • Partner with programs to acquire up-to-date information

Association of Institutional Research


Lessons learned1

Lessons Learned

  • Make students aware of the importance of the survey process (survey announcements)

  • Provide contact updates to college data systems (PeopleSoft, Alumni Foundation)

  • Monitor results and make changes as necessary

  • Use automated processes for a quicker turn around of results and more accurate data

Association of Institutional Research


Future direction

Future Direction

  • Our ultimate goal is to provide Program Administrators ‘timely’, ‘relevant’ ‘accurate’, and ‘interpretable’ information regarding their programs and their performance outcomes

    • Multi-measure ‘Dashboard’ style reports (Business Intelligence/Performance Measurement)

Association of Institutional Research


Questions next steps

Questions/Next Steps

Association of Institutional Research


May 2010

Association of Institutional Research

50th Annual Forum

FAIR Best Paper: Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers

May 2010


  • Login