Using multivariate techniques for the analysis of survey data a case example
This presentation is the property of its rightful owner.
Sponsored Links
1 / 18

Using Multivariate Techniques for the Analysis of Survey Data: A Case Example PowerPoint PPT Presentation


  • 57 Views
  • Uploaded on
  • Presentation posted in: General

Using Multivariate Techniques for the Analysis of Survey Data: A Case Example. Isabelle Bourgeois Natural Sciences and Engineering Research Council University of Ottawa Joint Conference of the CES and the AEA Toronto, October 26 th 2005.

Download Presentation

Using Multivariate Techniques for the Analysis of Survey Data: A Case Example

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Using multivariate techniques for the analysis of survey data a case example

Using Multivariate Techniques for the Analysis of Survey Data: A Case Example

Isabelle Bourgeois

Natural Sciences and Engineering Research Council

University of Ottawa

Joint Conference of the CES and the AEA

Toronto, October 26th 2005


Rationale for study

Performance measurement activities in R&D often use complex techniques taken from the sciences and economics

Methods are needed that capture more fully the non-economic benefits of research to support decision makers

The use of multivariate analysis techniques taken from the social sciences can provide useful information based on survey results

Rationale for Study


Examples of survey measures

Industry awareness of government research and researchers

Satisfaction with past interactions between industry and government

Level of trust in researchers and staff

Types of new processes or products introduced to market

Examples of Survey Measures


Use of sophisticated analytical methods

Traditional survey data analysis relies on descriptive statistics with little exploration of the relationships that exist between variables

Harman (2004) example: Use of t-tests with little interpretation and no further research

Use of Sophisticated Analytical Methods


Case example

Multivariate analysis conducted on data from the Postgraduate Scholarship (PGS) program managed by NSERC

Data collection instrument: survey administered in the summer of 2005 (N=901)

Attrition: 101

Response rate: 69.6% (n=557)

Survey results estimated to be accurate ±3%, 19 times out of 20

Case Example


Variables

Independent: gender, type of award received (master’s or doctoral), and main field of study

Dependent: four seven-point scales on different aspects of the student’s experience with the PGS award

Variables


Preliminary examination of data

Descriptive statistics to get a general sense of the survey results

Verification of statistical assumptions most critical to multivariate analysis, normality and equality of the variance-covariance matrix (homoscedasticity)

Preliminary Examination of Data


Multivariate respondent profile

MANOVA (Multivariate Analysis of Variance): yields information to answer questions about the combined effect of the dependent variables

MANOVA conducted on each of the four scales using the gender, award type and field variables (Tables included in the paper)

Multivariate Respondent Profile


Findings of manova 1

Field variable found significant on its own in scale A2 and in combination with other dependent variables in scales A2 and C3

Suggests that the field of study has an impact on a student’s undergraduate experiences (A2), as well as his or her graduate experiences (C3)

Findings of MANOVA (1)


Findings of manova 2

Award variable found significant on scale C2 which focused on relationship with supervisor and future career plans

The award variable has an impact on the student’s relationship with supervisor and future career plans – Master’s and Ph.D. students have different experiences and expectations

Findings of MANOVA (2)


Discriminant function analysis

Close relative of the MANOVA

Instead of looking at the influence of the independent variables on survey responses, it predicts group membership within each independent variable by using the responses to the scale items.

Findings focus on items loading most highly for each variable – these provide the best discrimination between levels of the variable (examples)

Discriminant Function Analysis


Exploration of scale properties

Aside from providing more sophisticated information about the results of the survey, multivariate analysis methods also provide information about the properties of the instrument itself.

Exploration of Scale Properties


Exploratory factor analysis

Data summarization technique

Identifies underlying, or latent, dimensions in a dataset that, when interpreted, describe the data in a smaller number of concepts

The responses to the items on the PGS survey scales are hypothesized to vary as a function of one or several latent variables

Exploratory Factor Analysis


Factors extracted

Nine meaningful factors extracted and retained for interpretation

Example (Factor 1, Learning and Skill Development):

Funding from NSERC will help me to complete my degree faster

Theoretical knowledge of the discipline

Analytical techniques/experimental methods

Use of laboratory equipment or instruments

Project management

Etc.

Factors Extracted


Factors extracted1

Learning and Skill Development

Supervision

Resources

Career Orientation

No Debt

Encouragement

Collaborative Experiences

Debt Load

Job Prospects

Factors Extracted


Scale reliability

Reliability: Internal consistency of a scale – minimizes the contribution of random error to item scores

Alpha coefficient is used to measure internal consistency and is interpreted as a correlation

Alpha coefficients calculated for each of the scales and the combined items, as well as for the Factors extracted in the EFA to see which model fits best

Scale Reliability


Reliability comparison

Original Scales

Scale A2: 0.59

Scale C1: 0.81

Scale C2: 0.77

Scale C3: 0.85

Combined Items: 0.82

Factors

Factor 1: 0.84

Factor 2: 0.91

Factor 3: 0.81

Factor 4: 0.64

Factor 6: 0.50

Factor 8: 0.42

Factor 9: 0.24

Reliability Comparison


Advantages of multivariate analysis

Easy to do using standard statistical software packages

Interpretation is fairly straightforward

Adds to descriptive data by highlighting the influence of the combined dependent variables on the survey responses

Allows for a review of the instrument properties and provides suggestions for improvement

Advantages of Multivariate Analysis


  • Login