1 / 34

Research Designs

Research Designs. Nonexperimental designs 1 Causal modelling 2 tests of the models. Characteristics. No manipulation No random assignment In order to test causal hypotheses all confounding variables have to be included in the study in order to control for their effects. The Problem.

fbernal
Download Presentation

Research Designs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Designs Nonexperimental designs 1 Causal modelling 2 tests of the models

  2. Characteristics • No manipulation • No random assignment • In order to test causal hypotheses all confounding variables have to be included in the study in order to control for their effects college titel en nummer

  3. The Problem • We are interested in the effect of a X (seeing a movie) on a dependent variable Y (Opinion). • X -------------> Y • X= 0 if people have not seen a movie • X= 1 if people have seen a movie. • One can not simply use the difference of means between the two groups because these two groups can differ on several variables X1 to Xk and not only X. college titel en nummer

  4. Similar and Different • This is the same problem as in quasi experimental designs • The difference is that one can not use a pretest score • So the only solution is to control for all so called confounded variables. • Looking for these variables is called “Causal modeling” college titel en nummer

  5. Causal Modelling • All variables which can give an alternative explanation for the relationship between X and Y have to be introduced in the study in order to control for their effect : • X Y • X1 ...... Xk college titel en nummer

  6. Research in Russia • Some years ago somebody suggested: Russia represents a real life experiment for social change. • It is clear this is not a real experiment: We can only observe changes. • The problem to study was: • What change occurs ? • What is the effect of change on satisfaction with some aspects of life ? college titel en nummer

  7. Three causal hypotheses • We expect that satisfaction is determined by an objective and a subjective characteristic: • the situation as it is and the preferred ideal situation • So we expect the following DIRECT EFFECTS • 1. the satisfaction (S) will remain the same as before unless • 2. A change in the objective situation(DS) will change satisfaction • 3. A change in the preferred ideal situation (DI) college titel en nummer

  8. Path diagram • This leads to the following causal model: • DS • S(t) S(t-1) • DI • The direct effects are represented by directed arrows. college titel en nummer

  9. Indirect effects and Spurious relationships • S (t-1) has also indirect effect on S(t) because • hypothesis 1: • Some people who are dissatisfied will try to change their situation • hypothesis 2: • Some people who are dissatisfied and do not expect any change will adjust their ideals downwards. college titel en nummer

  10. Adjustment of the model • Two more direct effects of S(t-1) on DS and DI: • DS • S(t) S(t-1) • DI • A consequence of this extention of the theory with two direct effects is that S(t-1) has two indirect effects on S(t) and that • part of the relationship between S(t) and DS and between S(t) and DI is spurious. Check this. college titel en nummer

  11. OTHER SPURIOUS RELATIONS • Some of the relationships of S(t-1) on the other variables can also be spurious because of effects of the Difference between the objective situation and the ideal at time t-1 called DI(t-1). • We hypothesize: • DI(t-1) has an effect on S(t-1) and DI college titel en nummer

  12. More • We also hypothesize also that there are effects of the expectations concerning the possibilities to Change the situation will cause spurious relations. We call this EPD. • We hypothesize : • EPD has an effect on S(t-1) and DS. college titel en nummer

  13. Second Adjustment of the model • This leads to the following model • DS EPD • S(t) S(t-1) • DI DI(t-1) • Check where spurious relationships arize due to this extention of the causal model. college titel en nummer

  14. WHEN IS A MODEL COMPLETE ? • As was mentioned before: • all variables which cause spurious relationships have to be introduced. • Variables which only influence the cause or only the effect variable can be omitted • Intervening variables can also be omitted. college titel en nummer

  15. More variables • Part of the relationship between S(t-1) and DI(t-1) and EPD are certainly spurious due to satisfaction at a previous point in time , S(t-2), effecting S(t-1) and DI(t-1) and EPD. college titel en nummer

  16. Solution • But this variable S(t-2) does not directly affect S(t) and DS and DI. • So ignoring this variable will only disturb the relationships between the variables S(t-1) , DI(t-1) and EPD . • This problem can be solved by introduction of a correlations between S(t-1), EPD and DI(t-1) representing the effect of S(t-2). college titel en nummer

  17. The final model • The leads to the following final model • DS EPD • S(t) S(t-1) • DI DI(t-1) • The double headed arrows are correlations • The effects of the predetermined variables on each other can not be studied. college titel en nummer

  18. Testing Causal models • In nonexperimental research testing of the model is essential. • Without the test one does not know if the estimated values are correct • A two steps procedure will be discussed to test these causal hypotheses: • 1 estimation of the effects • 2 testing the model college titel en nummer

  19. Data:Correlations • s(t) Ds Di s(t-1) epD di(t-1) • -------- -------- -------- -------- -------- -------- • s(t) 1.00 • Ds -0.04 1.00 • Di -0.40 -0.14 1.00 • s(t-1) 0.34 -0.12 0.29 1.00 • epD 0.02 -0.09 0.02 0.02 1.00 • di(t-1) 0.31 -0.06 0.58 0.61 -0.02 1.00 college titel en nummer

  20. Correlation is not effect • Correlation = direct effect + indirect effects + spurious relationships + joint effects • So the correlation is not the proper coefficient to use for tests • The direct effects are the correlation - indirect effects - spurious relationships - joint effects • These effects are fundamental for causal analysis college titel en nummer

  21. Estimation of the direct effects • Regression equations: • S(t) = a1+ b12DS + b13DI +b14 S(t-1) • DS = a2 + b24 S(t-1) + b25 EPD • DI = a3 + b34 S(t-1) + b35 DI(t-1) college titel en nummer

  22. Estimation of equation 1 • Equation Number 1 Dependent Variable.. • S(t) How satisfied are you • ------------------ Variables in the Equation ------------------ • Variable B SE B Beta T Sig T • DI -.451272 .027172 -.536938 -16.608 .0000 • DS .061065 .275531 .006916 .222 .8247 • S(t-1) .476791 .031331 .495480 15.218 .0000 • (Constant) 2.918733 .189812 15.377 .0000 college titel en nummer

  23. Estimation of equation 2 • Equation Number 2 Dependent Variable.. • DS Change in situation • ------------------ Variables in the Equation ------------------ • Variable B SE B Beta T Sig T • S(t-1) -.012817 .004240 -.117603 -3.022 .0026 • EPD -.027567 .011930 -.089906 -2.311 .0212 • (Constant) .232339 .039426 5.893 .0000 college titel en nummer

  24. Estimation of equation 3 • Equation Number 3 Dependent Variable.. • DI change in their ideal • ------------------ Variables in the Equation ------------------ • Variable B SE B Beta T Sig T • S(t-1) -.127727 .045763 -.111556 -2.791 .0054 • DI(t-1) .769304 .047097 .652877 16.335 .0000 • (Constant) -2.948347 .217671 -13.545 .0000 college titel en nummer

  25. The result • Before drawing conclusions one has to know if these results can be trusted • In nonexperimental research model testing is essential college titel en nummer

  26. The expected correlations • sums of direct, indirect effects , spurious relations and joint effects assuming that the model is correct • s(t) Ds Di s(t-1) epD di(t-1) • -------- -------- -------- -------- -------- -------- • s(t) 1.01 • Ds -0.10 1.00 • Di -0.40 -0.03 1.00 • s(t-1) 0.34 -0.12 0.29 1.00 • epD 0.02 -0.09 -0.01 0.02 1.00 • di(t-1) -0.02 -0.07 0.59 0.61 -0.02 1.00 college titel en nummer

  27. Residuals • Residual = observed correlation - expected correlations assuming that the model is correct • s(t) Ds Di s(t-1) epD di(t-1) • -------- -------- -------- -------- -------- -------- • s(t) -0.01 • Ds 0.06 0.00 • Di 0.01 -0.10 0.00 • s(t-1) 0.00 0.00 0.00 0.00 • epD 0.00 0.00 0.03 0.00 0.00 • di(t-1) 0.33 0.01 0.00 0.00 0.00 0.00 college titel en nummer

  28. The adjusted model college titel en nummer

  29. The new estimates college titel en nummer

  30. Does the model fit ? • The residuals are significantly different from zero • So the model still has to be rejected but is much better than before • However we will show that this test is too simple and an alternative will be introduced. college titel en nummer

  31. The new estimates college titel en nummer

  32. Some substantive conclusions • Changes in the living conditions (DS) has hardly any effect • The difference between the reality and the ideals (DI(t-1)) is the most determining factor for • S(T-1) and DI • The strongest effect on satisfaction (S(t) ) comes from Changes in ideals DI : • People who adjust their ideals downwards become more satisfied !!! college titel en nummer

  33. Methodological Conclusion • Also in case of nonexperimental research causal hypotheses can be tested • But it requires a lot of preparation in advance : causal modeling • because there is no randomization and • also a pretest is normally not available • So one has to control for all variables which can cause a spurious relationship between the variables of interest. college titel en nummer

  34. Methodological conclusions • Most of the time regression analysis is used • Regression coefficients estimate indeed the effect of an independent variable on a dependent variable • but only if • the causal model is well developed so that all relevant variables are introduced • and the model fits to the data college titel en nummer

More Related