REVIEW

1. REVIEW • Reliability: • of a quantitative measure is a major criterion for assessing it’s “quality” • The consistency with which an instrument measures the attribute • Less variation an instrument produces in repeated measures of an attribute = higher reliability • Accuracy Three aspects of reliability: • Stability • Internal consistency • equivalence

2. Internal Validity Define: the degree to which an instrument measures what it is intended to measure

3. Internal Validity Threats to Internal Validity: • History • Selection • Maturation • Mortality

4. External Validity Define – the “generalizability” of the research findings to other settings or samples • adequacy of the sampling design • environmental or the research situation influences a study’s external validity

5. Class Exercise #1 • What is the independent variable in this study? • What is the dependent variable in this study? • Was there random assignment in this study? • Is the design experimental, pre-experimental, or non-experimental? • What is the specific name of the design used? • Is the study design within-subjects or between-subjects? • Was selection a possible threat to the internal validity? • Was history a possible threat to the internal validity? • Was mortality a possible threat to the internal validity? • Would it possible to address the same research question with a stronger research design?

6. Class Exercise #2 • What is the independent variable in this study? • What is the dependent variable in this study? • Is the design experimental, quasi-experimental, pre-experimental or non- experimental? • Is the study design within-subjects or between-subjects? • Is the study design cross-sectional or longitudinal? • Is the study design prospective or retrospective? • Was matching used to control extraneous variables? • Was there attrition in this study? • Was random assignment used to control extraneous variables? • Was selection a possible threat to internal validity? • Would it be possible to address the same research question with an experimental design?

7. SAMPLING Dr. Anne Snowdon Professor Lisa High University of Windsor November 16th, 2005

8. What is the purpose of Sampling? • Gather information • Make decisions • Formulate predictions “Researchers derive knowledge and draw conclusions from samples”

9. QUALITATIVE not concerned with generalizability to achieve an in-depth, holistic understanding of the phenomena of interest sampling decisions emerge during course of data collection QUANTITATIVE to generalize develop an appropriate plan before data collection Sampling

10. SAMPLING -Quantitative What is a “population?” • An entire aggregation of cases that meets a specified set of criteria • Populations not restricted to human participants What are some examples of populations?

11. Sampling Criteria • Specified characteristics • Delimit the population • Inclusion criteria and exclusion criteria Definition: criteria used by a researcher to designate the specific attributes of the target population, by which participants are selected for participation in a study.

12. Populations (1) Accessible population: - generalizing to the entire population - comprises cases from the target population, accessible to the researcher, “pool of participants” - might be elements within a state, city, hospital or nursing unit (2) Target population: - entire population in which the researcher is interested in

13. Samples & Sampling Sampling: is the process of selecting a portion of the population to represent the entire population Sample: is a subset of the population Elements: individual units of the samples and populations Subjects: elements (when elements are persons, referred to as “subjects”) Strata: are mutually exclusive segments of a population based on specified characteristics

14. Categories of Sampling Plans Two broad categories: • PROBABILITY • NON-PROBABILITY

15. Sampling Rationale • Samples vs. populations – why? • Can lead to erroneous decisions • Sampling bias: - over or under representation of some segment of the population in terms of a characteristic relevant to the research question - affected by the homogeneity of the population

16. Sampling Rationale cont’d Homogeneity – Heterogeneity -

17. REPRESENTATIVENESS Definition: means that the sample must be like the population in as many ways as possible sample to be representative in relation to the variables being studied and to other factors that may influence the study variables

18. Non-probability Sampling Quantitative Studies - Three primary methods: • Convenience sampling • Quota sampling • Purposive sampling

19. Evaluation of Non-probability Sampling • Rarely representative of the target population • Usually some segment of the population is under-represented • Cautious of conclusions • Be aware of “sampling bias” • Advantage: convenient, economical

20. Probability Sampling • Involves the random selection of elements from the population Random Selection Define:

21. Random Selection 4 Most Commonly used are: • Simple random sampling • Stratified random sampling • Cluster sampling • Systematic sampling

22. Evaluation of Probability Sampling • Only reliable method for obtaining representative samples in quantitative studies • Avoids the risk of conscious or unconscious biases • All elements have equally opportunity to be selected = > likelihood the sample will represent the population adequately • Allows the researcher to estimate the “magnitude of sampling error”

23. Sampling Error • The difference b/w a sample statistic and a population parameter • Large sampling error = sample not providing a precise picture of the population = it is not representative • Sampling error is higher with small samples • As the sample size increases = sampling error decreases

24. Sample Size • The number of subjects in a sample • Important issue in conducting and evaluating quantitative research • Best rule of thumb: use the largest sample possible (Refer to Polit & Beck, pp. 245 – 246)

25. Sample Size To test a hypotheses – researchers can estimate how large a sample should adequate “POWER ANALYSIS” Define:

26. Sampling in Qualitative Research • Usually small non-random samples • Use a different criteria for selecting participant • Small sample does not reflect poor quality Logic of Qualitative Sampling: • aim to discovery meaning and uncover multiple realities • “Generalizability” is not a guiding criterion

27. Types of Qualitative Sampling Sampling Types: • Volunteer sampling (convenience) • Snowball sampling • Purposive sampling or theoretical sampling Purposive sampling strategies: • Maximum variation sampling • Extreme/deviant case sampling • Typical case sampling (Patton, 1990)

28. Sample Size • No firm established criteria or rules Is largely a function of : • Purpose of the inquiry • Quality of the informants • Type of sample strategy used Guiding Principle: data saturation

29. Critiquing – A Sampling plan Researcher adequately describes the sampling plan Includes a description of the following: • Sampling approach • Population • Eligibility criteria • Nature of setting • Number of participants • Rationale for sample size • Description of main characteristics of participants • In quantitative - # of participants who agreed to participant and then withdrew

30. Evaluating - Quantitative • Sampling strategy weak or sample small = reason to suspect some bias • Bias depends on: population’s homogeneity • Response rates • Non-response bias • Gap b/w accessible and target population too large • Reasonableness of generalizing the findings (Review Loiselle & Profetto-McGrath, p. 251, Box 10-1)

31. Evaluating - Qualitative Adequacy and appropriateness Adequacy: Appropriateness: (Review Loiselle & Profetto-McGrath, p. 252, Box 10-2)