ScWk 240 Week 8

1. ScWk 240Week 8 Sampling (continued) Intro to Causality and Group Designs “The tendency of the casual mind is to pick out or stumble upon a sample which supports or defies its prejudices, and then to make it the representative of a whole class.” Walter Lippman (1889-1974), journalist

2. Review: Name the implied type of reliability or validity testing: • Two clinical observers of parent-child interaction mostly agree on whether the parent’s response to the child’s behavior is flexible or rigid _______________ • A researcher wants to make sure that the instrument she is developing covers all the various dimensions of the concept “empowerment.” __________________ • A new instrument called the Self Perceived Empowerment Scale has a coefficient alpha of .85 __________________ • A person scoring higher on a depression scale (more depressed) will also often score higher on an anxiety measurement __________________ • A person scoring higher on a depression scale (more depressed) will also often have a diagnosis of depression after a clinical interview _______________ • Inter-rater reliability • Test-retest reliability • Internal consistency reliability • Criterion-related validity • Content validity • Face validity • Construct validity

3. News flash—much much better definition of construct validity: Extent to which an instrument accurately measures one concept. For example, an instrument designed to measure alcoholism accurately measures alcoholism (not depression or other diagnosis). In addition, construct validity can also refer to the extent to which a measure of a construct (like alcoholism) is correlated with another theoretically-related measure (such as “addictive personality”) Source: Faulkner, C. A. & Faulkner, S. S. (2009). Research methods for social workers: A practice-based approach. Chicago, IL: Lyceum

4. Student report • What is the main difference between “open-ended” and “closed-ended” questions? Selene • Discussion question for class: what are the implications for reliability and validity for both types of questions?

5. Two main types of sampling: • Probability sampling -- everyone in the study population has an equal chance of selection into your study. • Non-probability sampling -- everyone in the study population does not have an equal chance of selection ***Discussion: what are the pros and cons of both?

6. The Logic of Probability Sampling • Overall purpose of probability sampling: to allow the researcher to estimate characteristics (parameters) of the study population from a sample’s characteristics • e.g. “How much does the sample represent the population from which it was drawn?” • Why do we want to “estimate population characteristics”? • Because in most cases we don’t really know the real population characteristics (but we’d like to know!) • So, we estimate a population parameter (e.g. average depression score of the population), from a sample statistic (average depression score of the sample) • Since research means “never having to say you’re certain”, we won’t be 100% perfect in our estimates, although we can quantify how confidently we feel our sample matches the population. How? Stay tuned for ScWk 242.

7. Probability Sampling Example: “How do stress levels of teen parents compare with stress levels of other youth?” Population: All youth From the sample statistic we can “infer” stress levels in the larger population of youth Probability sample of teen parents Select random samples of teen parents and non-parent youth Statistic –comparison of stress levels Probability sample of non-teen parent youth

8. How to do probability sampling • Simple random sample—assigning numbers to potential participants and selecting numbers (people) randomly. Similar to picking them out of a hat • Systematic random sample—choosing every kth person--selecting persons at a predetermined interval • Stratified random sampling--Simple or systematic random sampling with sub-groups • Cluster sampling—a multi-stage procedure of randomly choosing levels of analysis units (e.g. cities, neighborhoods, schools, classrooms)

9. Probability sampling “ripped from the headlines…” http://www.pbs.org/newshour/bb/politics/july-dec12/makingsense_10-12.html

10. Non-probability sampling • Quota sampling—sample chosen based on predefined characteristics of study so that sample will have same proportion of those characteristics as in the study population • Think of a journalist standing on a street corner interviewing people who walk by, but who also wants the opinions of both men and women ***What’s the difference between this and stratified random sampling? • Snowball sampling—for difficult-to-locate populations: finding more sample participants based on recommendations from others in the study • You’re interviewing a victim of domestic violence and you ask “Can you think of anyone else at this shelter who might be able to talk with me about this?” • Purposivesampling —selecting sample that is thought to yield the most comprehensive understanding of the study’s topic • You put up a flyer inviting students to participate whose parents immigrated to the US • Convenience sampling (a.k.a. “availability sampling”)—selects participants simply based on their immediate availability. (Note—participants may also coincidentally comprise a purposive sample, but not necessarily) • You decide to interview staff of a county mental health agency about their training in cultural competence

11. Two families of sampling Probability sampling Non-probability sampling Quota sampling Snowball sampling Purposive sampling Convenience sampling • Simple random sampling • Systematic sampling • Stratified random sampling • Cluster sampling

12. Student report • Name some strategies to recruit difficult-to-reach populations, and retain them for follow up -- Annalisse

13. What type of sampling is this? (hint: first decide if it’s prob or non-prob) • For an experimental study of cognitive behavioral therapy, every 10th adult with anxiety disorders is selected randomly from a sampling frame • An agency manager walks down the hall and asks staff opinions about the new caseload policy • The same agency manager makes sure to ask both women & men • For a study on standardized testing in schools, researchers randomly select cities in California, then randomly select elementary, middle and high schools in those cities • A social work researcher seeks out Latino women who were victims of domestic violence • Simple random sampling • Systematic sampling • Stratified random sampling • Cluster sampling • Quota sampling • Snowball sampling • Purposive sampling • Convenience sampling

14. Student report • What is required in order to prove causality? Jenna

15. Establishing Causality—What are the Requirements*? • Time sequence: the cause (independent variable, or treatment) precedes the effect (dependent variable, or outcome) • The IV and the DV are associated. (They must be related or correlated statistically.) • The relationship between the IV and DV cannot be explained by a third variable (or rival hypothesis—an alternate explanation) *Original source: John Stuart Mill (1859). A system of logic

16. Causality and an Evaluative Study—”Does group therapy reduce anxiety?” X Group therapy (indep variable) Y Anxiety (dep variable) • Time sequence 2. Correlation (or association) Those getting group therapy will also have reduced level of anxiety. (Correlation alone doesn’t ensure causality!) 3. Outcome not caused by 3rd variable (rival hypothesis) Medication (X2) Group tx (X1) Anxiety

17. Independent variable should precede DV in time* Treatment is “manipulated” as an IV Establish statistical association between IV and DV* Eliminate rival hypotheses* (How? See 5.) Use control group with Random assignment Indep var (X) Set up experiment to show causality* Dep var (Y) Indep var (X) Treatment Attributes: No treatment Treatment is associated with some change in the DV X2 ? X1 Y Sample Assign to control (no tx)? Assign to treatment?

18. Random Assignment Example: “The impact of a parenting class on teen parents’ stress levels” All teen parents From the sample statistic we can “infer” how the population of teen parents will benefit from the class Those in teen parenting class Randomly assign a sample of teens to parenting class, and those not in class Those not in teen parenting class Statistic –how much better teens in parenting class did

19. Alternative Ways to Manipulate Independent Variable (not complete list)

20. Random sampling, random assignment (they’re different & easy to confuse) Study Population N =10,000 Sample n=100 Probability sampling (or Non-probability procedure) Random* assignment Experimental Group n=50 Control Group n=50 *Note: random assignment can also be stratified within the experimental and control groups

21. Student reports next week: • What is the most important difference between experimental group designs and all others? (Hint—has to do with group assignment) -- Ashley • Give an example of a one group, pretest posttest design -- Kristen

22. Concepts to know • Three requirements for establishing causality • Random assignment • Rival hypothesis • Experimental group • Control group • “infer” from a sample statistic • “Manipulating the treatment variable” • X is independent variable, Y is dependent variable