slide1 n.
Download
Skip this Video
Download Presentation
How to Assess Student Learning in Arts Partnerships Part II: Survey Research

Loading in 2 Seconds...

play fullscreen
1 / 25

How to Assess Student Learning in Arts Partnerships Part II: Survey Research - PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on

How to Assess Student Learning in Arts Partnerships Part II: Survey Research Revised April 2, 2012 Mary Campbell-Zopf, Ohio Arts Council Mary.Campbell-Zopf@oac.state.oh.us Craig Dreeszen Ph.D., Dreeszen & Associates craig@dreeszen.com. Your Instructors. Our intended learning outcomes.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'How to Assess Student Learning in Arts Partnerships Part II: Survey Research' - marny-morse


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

How to Assess Student Learning

in Arts Partnerships

Part II: Survey Research

Revised April 2, 2012

Mary Campbell-Zopf, Ohio Arts Council

Mary.Campbell-Zopf@oac.state.oh.us

Craig Dreeszen Ph.D., Dreeszen & Associates

craig@dreeszen.com

our intended learning outcomes
Our intended learning outcomes
  • Understand the use and limitations of written assessment instruments
  • Write effective survey questions
  • Understand survey distribution options
  • Interpret survey data
    • Qualitative data analysis
    • Quantitative data analysis
  • Report survey findings
written assessment instruments
Written Assessment Instruments
  • Evaluation forms
  • Tests
    • Pre- and post-tests
    • Standardized tests
  • Surveys
using written assessments
Using Written Assessments

Good reasons to use:

  • Efficient
  • May reach large samples
  • Test qualitative conclusions from small samples
  • Provide quantitative data
  • Enables reticent people to participate

May not be appropriate if:

  • Uncertain what to ask
  • Questions are ambiguous
  • Questions are complex
  • Respondents not literate
two sequencing options
Two Sequencing Options

Start with interviews or focus groups:

  • Test questions
  • Learn likely responses
  • Explore conclusions
  • Create fixed-response questions
  • Distribute to larger sample

Start with survey:

  • Explore ambiguous survey findings in interviews or focus groups
  • Investigate reasons for survey findings
survey steps
Survey Steps
  • Decide what data is needed
  • Design questionnaire
  • Select sample
  • Acquire lists
  • Pretest
  • Distribute and remind
  • Analyze and report
determine needed data
Determine Needed Data
  • Learning outcomes
    • To what extent were did participants learn what was intended
    • Unintended outcomes
  • Program satisfaction
    • What liked best
    • What needs improvement
  • Recommendations
  • Marketing advice
  • Advocacy evidence
write questions
Write Questions

Open-ended questions

  • Short answer
  • Long answer
  • Prompted

Fixed-response questions

  • Yes/No
  • Rating Scale
  • Choice (one or multiple)
  • Matrix rated
  • Ranked

Ask basic demographic questions too

review sampling
Review Sampling
  • Select representative sample of larger population
  • Or survey entire population
  • Acquire mailing or email lists
fixed response questions
Fixed-Response Questions

[Replace sample with OAC AP grantee examples]

practice writing questions
Practice Writing Questions
  • Try writing an open-ended question
  • Then, write a fixed-response questionand response choices
paper and online surveys
Paper and Online Surveys
  • Mailed surveys
    • Envelopes
    • Address labels
    • Postage
    • Cover letter
    • Survey
    • Return envelope and postage
  • Handout surveys
  • Online surveys
      • SurveyMonkey.com
      • Zoomerang.com
    • Email invitation
    • Survey
    • May provide links to survey results and organization website
    • Survey closed notice
online survey features
Online Survey Features
  • Screening questions
  • Skip logic
  • Quick response
  • Inexpensive
  • Track responses
  • Send reminders to unresponsive email addresses
test survey
Test Survey
  • Send early draft survey to partners
  • Revise
  • Send next draft to small sample of your population
  • Ask for feedback
  • Look for ambiguity or misunderstanding
  • Revise final survey
distribute surveys
Distribute Surveys

Paper surveys

  • Address and mail
  • Remind (newsletter, announcements…)

Handout surveys

  • Distribute and collect

Online surveys

  • Time right
  • Email invitation with link to survey URL
  • Web site links
  • Social media links
  • Track responses
  • Send reminders
statistical significance
Statistical Significance

Response Rate

Number of responses divided by number of surveys distributed

90 responses

200 surveys sent = 45% response rate

Deduct undeliverable surveys

Margin of error

When sampling populations, larger samples have smaller margin of error

Use online calculator

If you survey 200 students and 90 respond, margin of error = 7.7

If 120 respond, margin of error is 5.7

Some apparent results may be due to chance – not statistically significant

No margin of error if entire populationsampled, e.g., all participants in your program

analysis of data
Reduce large amount of data

Display it in notes, tables, or charts

Make sense of it

Statistical analysis of numbers

Content analysis of words

Analysis of data
making sense of words
For each question, read all answers

Look for patterns and clusters

Code similar statements

Count clusters of similar statements

Select typical quotes

Write summary

Do the same for next question

Making sense of words
making sense of numbers
Count

Average

Note patterns and clusters

Note most frequent responses

Cross tabulate to compare variables

Check statistical significance

Write summary

Making sense of numbers
report results with text
Report Results with Text

“Findings: The Arts Partnership Program works as intended. The Arts Partnership program furthers the OAC strategic plan, especially Goal 3, to “Help citizens of all ages learn and thrive through the arts.”

Quantitative evidence cited Just over half of AP grantees (55%) cited quantitative evidence to support their evaluation results in the final report.”

Executive summary

Interpret results

Key-point summary

Headings

Explanation

Data

Quotes

Appendix with detailed data and methods

display results with graphics
Display Results with Graphics
  • Data tables
  • Bar charts
  • Histograms
  • Pie charts
  • Line charts
  • Scatter plots
report results
Report Results

Report intended learning objectives and actual learning outcomes

Summarize activities

Suggest best practices and needed improvements

Acknowledge funders and partners

Provide back-up data

Sample OLGA report form

use results
Use Results
  • Draw conclusions
  • Report to stakeholders
  • Use data to improve programs and account to funders authorizers