Quasi Experiments Non-Experimental Research

1. Quasi ExperimentsNon-Experimental Research Research Methods & Statistics Summer 2014

2. Quasi Experiments • “Research procedure in which the scientist must select subjects for different conditions from preexisting groups” • Research Methods, McBurney & White • “An empirical study used to estimate the causal impact of an intervention on its target population. Quasi-experimental research designs share many similarities with the traditional experimental design or randomized controlled trial, but they specifically lack the element of random assignment to treatment or control. Instead, quasi-experimental designs typically allow the researcher to control the assignment to the treatment condition, but using some criterion other than random assignment.” • Wikipedia, Nov. 17, 2011

3. It is a matter of control True Experiment Quasi Experiment Selection of subjects for the conditions Observe categories of subjects If the subject variable is the IV, it’s a quasi experiment Don’t know whether differences are caused by the IV or differences in the subjects • Random assignment of subjects to condition • Manipulate the IV • Control allows ruling out of alternative hypotheses

4. Other features • In some instances cannot completely control the what, when, where, and how • Need to collect data at a certain time or not at all • Practical limitations to data collection, experimental protocol

5. Validity • Internal validity is reduced due to the presence of controlled/confounded variables • But not necessarily invalid • It’s important for the researcher to evaluate the likelihood that there are alternative hypotheses for observed differences • Need to convince self and audience of the validity

6. External validity • If the experimental setting more closely replicates the setting of interest, external validity can be higher than a true experiment run in a controlled lab setting • Often comes down to what is most important for the research question • Control or ecological validity?

7. Nonequivalent Control Group Designs • 2 groups, non-random allocation of subjects and groups, pre-test, treatment (Y/N), post-test • Desired pattern: • Dependent variables have equal pre-test values, difference seen between experimental and control groups on post-test (Pre: LL, Post: HL) • Want to show that any differences that exist did not impact the value of the variable of interest

8. Exercise • Interpret the following graphs (experimental, control): • Pre: (L, H) Post: (M, H) • Pre: (M, L) Post: (H, M) • Pre: (L, M) Post: (H, M)

9. Mixed Factorial Design with One Non-manipulated Variable • Example: experiment on pain perception (effect of caffeine, expected differences between men and women) • Protocol: • 25 men/25 women, each takes part in two sessions, one week apart • One session: drink coffee (decaf) and put hand in ice-water until feel pain • Other session: drink coffee (caffeinated) and put hand in ice-water until feel pain • Between subjects variable (male/female) • Non-manipulated • Within subjects variable (caffeine intake) • Manipulated

10. Non-Experimental Research Read and understand: • Gabriella Belli - Nonexperimental Quantitative Research • http://media.wiley.com/product_data/excerpt/95/04701810/0470181095-1.pdf • “Any quantitative study without manipulation of treatments or random assignment is a non-experimental study” • Experimental research shows cause and effect • Non-experimental research studies variables as they exist

11. Purpose of the Research • Descriptive: • primary focus for the research is to describe some phenomenon or to document its characteristics. • Predictive: • primary focus for the research is to predict some variable of interest (criterion) using information from other variables (predictors). • Explanatory : • the primary focus for the research is to explain how some phenomenon works or why it operates.

12. Correlation vs causation • In a widely studied example, numerous epidemiological studies showed that women who were taking combined hormone replacement therapy (HRT) also had a lower-than-average incidence of coronary heart disease (CHD), leading doctors to propose that HRT was protective against CHD. • http://en.wikipedia.org/wiki/Correlation_does_not_imply_causation

13. But randomized controlled trials showed that HRT caused a small but statistically significant increase in risk of CHD. • Re-analysis of the data from the epidemiological studies showed that women undertaking HRT were more likely to be from higher socio-economic groups (ABC1), with better-than-average diet and exercise regimens. The use of HRT and decreased incidence of coronary heart disease were coincident effects of a common cause (i.e. the benefits associated with a higher socioeconomic status), rather than cause and effect, as had been supposed.[3]

15. Correlation doesn’t imply causation. Cut it does wiggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there http://xkcd.com/552/

18. Time Frame • Cross-sectional (case study): • data are collected at one point in time, often in order to make comparisons across different types of respondents or participants. • Prospective (longitudinal, cohort, repeated measures): • data are collected on multiple occasions starting with the present and going into the future for comparisons across time. • Retrospective (historical): • look back in time using existing or available data to explain or explore an existing occurrence.

19. Techniques • Surveys • http://www.socialresearchmethods.net/kb/survey.php • Interviews • http://www.socialresearchmethods.net/kb/intrview.php • Observations • http://interactionarchitect.com/knowledge/article19991212shd.htm • http://www.edu.plymouth.ac.uk/resined/observation/obshome.htm • Unobtrusive methods: • http://www.socialresearchmethods.net/kb/unobtrus.php