analyzing survey data l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Analyzing Survey Data PowerPoint Presentation
Download Presentation
Analyzing Survey Data

Loading in 2 Seconds...

play fullscreen
1 / 35

Analyzing Survey Data - PowerPoint PPT Presentation


  • 200 Views
  • Uploaded on

Analyzing Survey Data. Angelina Hill, Associate Director of Academic Assessment 2009 Academic Assessment Workshop May 14 th & 15 th UNLV. Prior to Analysis. What would you like to discover? Perceived competence Preferences, satisfaction Group differences Demographics

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Analyzing Survey Data


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
analyzing survey data

Analyzing Survey Data

Angelina Hill, Associate Director of Academic Assessment

2009 Academic Assessment Workshop

May 14th & 15th

UNLV

prior to analysis
Prior to Analysis
  • What would you like to discover?
    • Perceived competence
    • Preferences, satisfaction
    • Group differences
      • Demographics
  • What are your predictions?
prior to analysis3
Prior to Analysis
  • Your goals drive the make-up of the survey and how it should be analyzed.
  • Exploration can be informative, but with an analysis plan.
prior to analysis4
Prior to Analysis
  • Survey design & layout
    • Stylistic considerations are important because they increase response, validity, and reliability
survey design
Survey Design
  • Good questions reduce error
    • By increasing the respondent’s willingness to answer
    • Increases reliability and validity.
      • Less error = better data
reliability validity
Reliability & Validity
  • Reliability – Is the survey measuring something consistently?
    • Typically measured using Chronbach’s alpha
  • Validity – Is the survey measuring what it’s supposed to be measuring?
    • Typically measured using factor analysis
construct validity
Construct Validity
  • Does your measure correlate with a theorized concept of interest?
    • Correlate measure with values that are known to be related to the construct.
pilot
Pilot
  • Piloting the survey can inform:
    • Question clarity
    • Question format
    • Variance in responses
survey analysis
Survey Analysis
  • Using data from
    • Paper Surveys
    • SurveyMonkey
    • SelectSurvey.Net
survey analysis10
Survey Analysis
  • Paper surveys
    • Put data in spreadsheet format using excel or SPSS
      • Columns represent variables
      • Rows represent respondents
survey analysis11
Survey Analysis
  • Paper surveys
    • Create a data matrix

Variable name || Numeric Values || Numeric labels

    • Summarize open-ended questions separately
      • Response group || frequency
survey analysis12
Survey Analysis
  • SurveyMonkey
    • Available under the analyze results tab
      • Frequencies & crosstabs
      • Download all responses for further analysis
        • Select Download responses from menu
        • Choose type of download – select all responses collected
        • Choose format – select condensed columns and numeric cells.
survey analysis13
Survey Analysis
  • SelectSurvey.NET
    • Available under Analyze  Results Overview
      • Frequencies
      • Download all responses for further analysis
        • Select Export Data from Analyze page
        • Export Format – CSV (excel)
        • Data Format – SPSS Format Condensed
data cleaning
Data Cleaning
  • Process of detecting, diagnosing, and editing faulty data
  • Basic Issues:
    • lack or excess of data
    • outliers, including inconsistencies
    • unexpected analysis results and other types of inferences and abstractions
data cleaning15
Data Cleaning
  • Inspect the data
      • Frequency distributions
      • Summary statistics
      • Graphical exploration of distributions
        • Scatter plots, box plots, histograms
data cleansing
Data Cleansing
  • Out of range
    • Delete values and determine how to recode if possible
  • Missing Values
    • Refusals (question sensitivity)
    • Don’t know responses (can’t remember)
    • Not applicable
    • Data processing errors
    • Questionnaire programming errors
    • Design factors
    • Attrition
missing data
Missing Data
  • Missing completely at random (MCAR)
    • Cases with complete data are indistinguishable from cases with incomplete data.
  • Missing at random (MAR)
    • Cases with incomplete data differ from cases with complete data, but pattern of missingness is predicted from variables other than the missing variable.
  • Nonignorable
    • The pattern of data missingness is non-random and it is related to the missing variable.
missing data19
Missing Data
  • Listwise or casewise data deletion: If a record has missing data for any one variable used in a particular analysis, omit that entire record from the analysis.
    • Default in most packages, including SPSS & SAS
  • Pairwise data deletion: For bivariate correlations or covariances, compute statistics based upon the available pairwise data.
    • Useful with small samples or when many values are missing
  • Substitution techniques: Substitute a value based on available cases to fill in missing data values on the remaining cases.  
    • Mean Substitution, Regression methods, Hot deck imputation, Expectation Maximization (EM) approach, Raw maximum likelihood methods, Multiple imputation
descriptive statistics
Descriptive Statistics
  • Frequency distribution
descriptive statistics21
Descriptive Statistics
  • Cross-tabs
    • Excel Pivot tables
      • Excel menu  Data  PivotTable and PivotChart
      • PivotTable menu  Field setting summarize by count  show data as % of row or column
data analysis
Data Analysis
  • Measurement scale determines how the data should be analyzed:
    • Nominal, ordinal, interval, ratio
    • Move from categorical information, to also knowing the order, to also knowing the exact distance between ratings, to also knowing that one measurement in twice as much as another.
data analysis23
Data Analysis
  • Three instructors are evaluating preferences among three methods (lecture, discussion, activities)
    • 1) Identify most, second, and least preferred.
    • 2) Identify your favorite.
    • 3) Rate each method on a 10-point scale, where 1 indicates not at all preferred and 10 indicates strongly preferred.
data analysis24
Data Analysis
  • Nominal & ordinal variables are discrete
    • Can be qualitative or quantitative
  • Interval & ratio variables are continuous
    • Grades
    • Age
data analysis25
Data Analysis
  • Charts
    • Pie charts & bar charts used for categorical data
    • Histograms used for continuous data
    • Line graphs typically show trends over time
data analysis26
Data Analysis
  • Other descriptive statistics
    • Mean
      • preferred, uses all of the data
    • Median
      • ordinal data
      • open-ended scale
      • outliers
    • Mode
      • nominal data
data analysis27
Data Analysis
  • Other descriptive statistics
    • Interquartile range
      • Variability accompanying the median
    • Standard deviation
      • Variability accompanying the mean
correlations
Correlations
  • Are the variables related?
    • Determine variables that relate most to your item of interest
      • Correlate Likert-scale questions with each other
      • Correlate interval/ratio demographic information (e.g., age) to Likert-scale questions
correlation
Correlation
  • Which correlation coefficient to use?
    • Pearson’s r
      • Used with interval and ratio data
    • Spearman & Kendall’s tau-b
      • Used with ordinal data
      • Spearman used for linear relationship
      • Kendall’s tau-b for any increasing or decreasing relationship
mean differences
Mean Differences
  • Are there meaningful differences between groups?
    • class sections
    • instructors
    • on-line vs. off-line courses
    • major vs. non-major
mean differences31
Mean Differences
  • Which test to run?
    • Interval and ratio data
      • t-test when comparing 2 groups
        • Independent
        • Dependent (paired-samples in spss)
      • ANOVA when comparing > 2 groups
        • Independent (One Way ANOVA in spss)
        • Dependent (general linear model-repeated measure in spss)
presenting results
Presenting Results
  • Describe the purpose of the survey
    • List the factors that motivated you to conduct this research in the first place.
  • Include the survey!
    • On assessment reports
    • When the survey is new/still being fine tuned
  • How it was administered
presenting results33
Presenting Results
  • Present the breakdown of results
    • Tables and graphs should complement text
  • Conclusions
    • Explain findings, especially facts that were important or surprising
  • Recommendations
    • Describe an action plan based on concise concluding statements
presenting results34
Presenting Results
  • Share results in formal venues
  • Familiarize yourself with key findings so that you can mention results at every opportunity
moving forward
Moving Forward
  • Continuously improve the survey
    • Delete, add, change questions
    • Evaluate method of administration
  • Compare results across semesters to look for improvements
  • Compare with other assessment data for a broader picture