cultural differences as statistical artefacts l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Cultural differences as statistical artefacts? PowerPoint Presentation
Download Presentation
Cultural differences as statistical artefacts?

Loading in 2 Seconds...

play fullscreen
1 / 28

Cultural differences as statistical artefacts? - PowerPoint PPT Presentation


  • 190 Views
  • Uploaded on

Cultural differences as statistical artefacts?. Reanalysing cross-national data with more advanced techniques. Dr Michael Hoelscher Department of Education University of Oxford. QMSS Conference Prague 21/06/2007. Overview. Context of the study Introduction to data

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Cultural differences as statistical artefacts?' - libitha


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
cultural differences as statistical artefacts

Cultural differences as statistical artefacts?

Reanalysing cross-national data with more advanced techniques

Dr Michael Hoelscher

Department of Education

University of Oxford

QMSS Conference

Prague

21/06/2007

overview
Overview
  • Context of the study
  • Introduction to data
  • Cultural differences within Europe – a first approach
  • Reanalysing the data with CFA
  • Applying a correction for measurement errors
  • Conclusions
1 context of the study
1. Context of the study
  • European integration and enlargement often discussed in economic terms only
  • However: Cultural influences might play a crucial role
  • Comparison of values in different spheres for all countries in the EU

(Religion, Family and Gender, Economy, Welfare State, Democracy)

  • “Normative” starting point: Position of the EU institutions, as found in its body of law and the treaties
  • Three year project, financed by VolkswagenStiftung
2 introduction to data
2. Introduction to data
  • European Values Study
    • 1999/2000
    • Wide variety of topics
    • Including all member and applicant countries of the EU (except Cyprus)
  • 28 countries are compared in our study
  • Today the focus is on “Democracy and Civic Society”
  • Secondary analysis
    • Indicators are not always “perfect”
2 introduction to data5
2. Introduction to data

Democracy:

  • 4 Indicators
    • “Having a strong leader” (v216)
    • “Having the army ruling” (v218)
    • “Having a democratic political system” (v219)
    • “Democracy may have problems, but best form of government” (v220)

(all measured on a scale with 4 categories)

2 introduction to data6
2. Introduction to data

Civic Society:

  • 2 Indicators
    • “People can be trusted” (v66)
    • “Membership in voluntary organisations”

(Index of membership in 14 groups; trade union membership is ignored)

3 cultural differences within europe a first approach
3. Cultural differences within Europe – a first approach

Methods

  • Comparisons of raw country means for each indicator
  • Integration of single indicators by using a discriminant analysis

(see Fuchs/Klingemann 2002 in “West European Politics”)

  • Explanation of differences on the individual level by multiple regressions
3 cultural differences within europe a first approach8
3. Cultural differences within Europe – a first approach

Results

  • Large differences between the countries, but also within the countries
  • Old member countries support position of EU most, followed by new members
  • Bulgaria, and especially Romania and Turkey showed much lower support
3 cultural differences a first approach
3. Cultural differences – a first approach

Overall support for the EU’s position in the field of Democracy/Civic Society

(RANK)

4 re analysing the data with cfa
4. Re-analysing the data with CFA
  • Aim
    • To compare two different methods
    • Not: Building the best model!
    • Balance of model fit and equivalence of approaches is needed
4 re analysing the data with cfa11
4. Re-analysing the data with CFA

Advantages of CFA

  • Generally
    • CFA is the more appropriate technique
    • More flexible
    • Can easily be extended to an explanatory SEM
4 re analysing the data with cfa12
4. Re-analysing the data with CFA

Advantages

  • Measurement model
    • Test if measurement is the same in different countries and therefore a comparison is appropriate
    • Correction for measurement error possible (Saris/Gallhofer 2007)
4 re analysing the data with cfa13
4. Re-analysing the data with CFA

Advantages

  • Structural model
    • Relationship between “democracy” and “civic society” can be estimated
4 re analysing the data with cfa14
4. Re-analysing the data with CFA

Great Britain, N = 728

4 re analysing the data with cfa15
4. Re-analysing the data with CFA

Running the model for all 25 countries without constraints

  • Chi-square = 333.29, df = 175, p-value=.000
  • CFI = .988
  • RMSEA = .006 (adjusted: 0.032)

All modification indices within the countries are well below 20, in most cases below 5

  • One can assume configural invariance
slide16

4. Re-analysing the data with CFA

1. Model “Unconstrained”

Chi-square= 333,29 df = 175

CFI = .988

RMSEA = .006 (adjusted .032)

Introducing constraints: Model comparison

2. Model “Equal Measurement Weights”

Chi-square= 725,5 df = 271

CFI = .965

RMSEA = .008 (adjusted .04)

3. Model “Equal Measurement Weights and Intercepts”

Chi-square= 4752.256 df = 367

CFI = .666

RMSEA = .023 (adjusted .115)

slide17

4. Re-analysing the data with CFA

1. Model “Unconstrained”

=> Configural invariance can be assumed

Introducing constraints: Model comparison

2. Model “Equal Measurement Weights”

=> Metric invariance can be assumed

3. Model “Equal Measurement Weights and Intercepts”

=> Scalar invariance can not be assumed!

=> Mean comparison is (in priniciple) not appropriate with this model

=> Adjustments (freeing some parameters)

5 correction for measurement errors
5. Correction for measurement errors
  • SEM allows to correct for measurement errors
  • Saris, Gallhofer et al. (2007) have introduced a tool to estimate the quality (reliability and validity) of an instrument
  • From a huge amount of MTMM experiments they estimated the influence of certain characteristics on the quality
  • By coding one’s own questions one can predict their quality

=> http://www.sqp.nl/

5 correction for measurement errors20
5. Correction for measurement errors

Idea:

  • What has to be equal for cross-country-comparisons is the factor structure
  • The quality of the instrument might influence this factor structure,
    • if one does not correct for measurement error
    • if the quality is different in different countries
  • “We suggest that equivalence should (…) be tested by the equality of loadings based on the observed covariance matrix corrected for measurement error”
5 correction for measurement errors21
5. Correction for measurement errors

Indicators

True scores

Latent concept (by definition)

T1

q1

y1

e1

λ1

F

λ2

T2

q2

y2

e2

T3

λ3

q3

y3

e3

5 correction for measurement errors22
5. Correction for measurement errors

Applying the correction to a subsample of 9 countries:

  • “Democracy”-indicators
    • The validity was nearly 1 for all countries
    • Reliability is different in countries, but reasonably good
  • Problems with the “Civic Society”-indicators
    • Unable to code the quality of the index straightforward
    • Low quality of the “Trust” variable
5 correction for measurement errors23
5. Correction for measurement errors

Results:

  • Factor loadings increase
  • Model fit decreases very slightly
  • At least for this specific subsample the ranks do not change
  • Check for whole sample, especially the “difficult” cases, is still missing
6 conclusions
6. Conclusions
  • Advantages of the SEM approach
  • More appropriate
  • More flexible (integration of additional indicators)
  • Can detect problems with measurement model
  • Easily extendable to an explanatory model
  • Relationship between the latent constructs can be estimated
6 conclusions25
6. Conclusions
  • “Problems” of the SEM approach
  • More demanding (data quality)
  • Is it realistic to assume equal means and factor loadings over so many countries?
    • Partial invariance?
  • Taking requirements very seriously wouldn’t allow a comparison of all countries
6 conclusions26
6. Conclusions
  • Comparing the “outcome” of the three methods:
  • Small differences for the overall ranking
  • The methods seem to come to pretty similar results
  • However: Some extreme cases (Turkey), couldn’t be included or shifted quite a lot (Romania)
thank you
Thank you!

Quantitative Methods in the

Social Sciences Conference, Prague, 20-23 June 2007

Dr Michael Hoelscher

Department of Education

University of Oxford

michael.holscher@edstud.ox.ac.uk

literature
Literature
  • Michael Hoelscher (2006): Wirtschaftskulturen in der erweiterten EU. Die Einstellungen der Buergerinnen und Buerger im europaeischen Vergleich. Wiesbaden: VS Verlag
  • Juergen Gerhards (unter Mitarbeit von Michael Hoelscher) (2005, second edition 2006): Kulturelle Unterschiede in der Europaeischen Union. Wiesbaden: VS Verlag
  • Dieter Fuchs/Hans-Dieter Klingemann (2002): Eastward Enlargement of the European Union and the Identity of Europe. West European Politics, 25, 2: 19-54.
  • Willem E. Saris/Irmtraud Gallhofer (2007): Design, Evaluation, and Analysis of Questionnaires for Survey Research. Wiley.