1 / 36

Sampling & Experimental Control

Sampling & Experimental Control. Psych 231: Research Methods in Psychology. Sampling. Why do we do we use sampling methods? Typically don’t have the resources to test everybody, so we test a subset. Goals of “good” sampling: Maximize Representativeness:

Download Presentation

Sampling & Experimental Control

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Sampling & Experimental Control Psych 231: Research Methods in Psychology

  2. Sampling • Why do we do we use sampling methods? • Typically don’t have the resources to test everybody, so we test a subset • Goals of “good” sampling: • Maximize Representativeness: • to what extent do the characteristics of those in the sample reflect those in the population • Reduce Bias: • a systematic difference between those in the sample and those in the population

  3. Population Sample Sampling everybody that the research is targeted to be about the subset of the population that actually participates in the research

  4. Sampling to make data collection manageable Inferential statistics used to generalize back Population Sample Sampling

  5. Sampling Methods • Probability sampling • Use some form of random sampling • Non-probability sampling • Don’t use random sampling • These are typically not considered as good

  6. Simple random sampling • Every individual has a equal and independent chance of being selected from the population

  7. Systematic sampling • Selecting every nth person

  8. Stratified sampling • Step 1: Identify groups (strata) • Step 2: randomly select from each group

  9. Convenience sampling • Use the participants who are easy to get

  10. Quota sampling • Step 1: identify the specific subgroups • Step 2: take from each group until desired number of individuals

  11. Experimental Control • Our goal: • to test the possibility of a relationship between the variability in our IV and how that affects our DV. • Control is used to minimize excessive variability. • To reduce the potential of confoundings.

  12. Sources of variability (noise) • Sources of Total (T) Variability: T = NonRandomexp + NonRandomother + Random Nonrandom (NR) Variability - systematic variation A. (NRexp)manipulated independent variables (IV) i. our hypothesis is that changes in the IV will result in changes in the DV

  13. Sources of variability (noise) • Sources of Total (T) Variability: T = NonRandomexp + NonRandomother + Random Nonrandom (NR) Variability - systematic variation B. (NRother)extraneous variables (EV) which covary with IV i. other variables that also vary along with the changes in the IV, which may in turn influence changes in the DV (Condfounds)

  14. Sources of variability (noise) • Sources of Total (T) Variability: T = NonRandomexp + NonRandomother + Random Non-systematic variation C. Random (R) Variability • imprecision in manipulation (IV) and/or measurement (DV) • randomly varying extraneous variables (EV)

  15. Sources of variability (noise) • Sources of Total (T) Variability: T = NRexp + NRother +R • Goal: to reduce R and NRother so that we can detect NRexp. • That is, so we can see the changes in the DV that are due to the changes in the independent variable(s).

  16. NR NR NR R R other other exp Weight analogy • Imagine the different sources of variability as weights Treatment group control group

  17. NR NR NR R R other other exp Weight analogy • If NRother and R are large relative to NRexp then detecting a difference may be difficult

  18. NR NR NR R R other other exp Weight analogy • But if we reduce the size of NRother and R relative to NRexp then detecting gets easier

  19. Using control to reduce problems • Potential Problems • Excessive random variability • Confounding • Dissimulation

  20. Potential Problems • Excessive random variability • If control procedures are not applied, then R component of data will be excessively large, and may make NR undetectable • So try to minimize this by using good measures of DV, good manipulations of IV, etc.

  21. NR NR NR R R other other exp Excessive random variability Hard to detect the effect of NRexp

  22. Co-vary together Potential Problems • Confounding • If relevant EV co-varies with IV, then NR component of data will be "significantly" large, and may lead to misattribution of effect to IV IV DV EV

  23. NR R R other Confounding Hard to detect the effect of NRexp becausethe effect looks like it could be from NRexpbut is really (mostly) due to the NRother NR exp

  24. Potential Problems • Potential problem caused by experimental control • Dissimulation • If EV which interacts with IV is held constant, then effect of IV is known only for that level of EV, and may lead to overgeneralization of IV effect • This is a potential problem that affects the external validity

  25. Methods of Controlling Variability • Comparison • Production • Constancy/Randomization

  26. Methods of Controlling Variability • Comparison • An experiment always makes a comparison, so it must have at least two groups • Sometimes there are control groups • This is typically the absence of the treatment • Without control groups if is harder to see what is really happening in the experiment • it is easier to be swayed by plausibility or inappropriate comparisons • Sometimes there are just a range of values of the IV

  27. Methods of Controlling Variability • Production • The experimenter selects the specific values of the Independent Variables • (as opposed to allowing the levels to freely vary as in observational studies) • Need to do this carefully • Suppose that you don’t find a difference in the DV across your different groups • Is this because the IV and DV aren’t related? • Or is it because your levels of IV weren’t different enough

  28. Methods of Controlling Variability • Constancy/Randomization • If there is a variable that may be related to the DV that you can’t (or don’t want to) manipulate • you should either hold it constant (control variable) • let it vary randomly across all of the experimental conditions (random variable) • But beware confounds, variables that are related to both the IV and DV but aren’t controlled

  29. Poorly designed experiments • Example: Does standing close to somebody cause them to move? • So you stand closely to people and see how long before they move • Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”)

  30. Poorly designed experiments • Does a relaxation program decrease the urge to smoke? • One group pretest-posttest design • Pretest desire level – give relaxation program – posttest desire to smoke

  31. Poorly designed experiments • One group pretest-posttest design • Problems include: history, maturation, testing, instrument decay, statistical regression, and more Dependent Variable Independent Variable Dependent Variable participants Pre-test Training group Post-test Measure

  32. Poorly designed experiments • Example: Smoking example again, but with two groups. The subjects get to choose which group (relaxation or no program) to be in • Non-equivalent control groups • Problem: selection bias for the two groups, need to do random assignment to groups

  33. Poorly designed experiments • Non-equivalent control groups Self Assignment Independent Variable Dependent Variable Training group Measure participants No training (Control) group Measure

  34. “Well designed” experiments • Post-test only designs Random Assignment Independent Variable Dependent Variable Experimental group Measure participants Control group Measure

  35. “Well designed” experiments • Pretest-posttest design Random Assignment Independent Variable Dependent Variable Dependent Variable Experimental group Measure Measure participants Control group Measure Measure

  36. Next time • Read: Chpt 8

More Related