1 / 82

Producing Data

No matter how sophisticated the statistical techniques may be, reasonably accurate inferences about a population’s characteristics can’t be made without a sample that is: representative of the population and is large enough. What is inferential statistics?. Producing Data

vlora
Download Presentation

Producing Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. No matter how sophisticated the statistical techniques may be, reasonably accurate inferences about a population’s characteristics can’t be made without a sample that is: • representative of the population • and is large enough. • What is inferential statistics?

  2. Producing Data • Anecdotal evidence: based on haphazardly selected cases which often come to our attention because they are striking in some way. • Available data: data that were produced in the past for some other purpose but that may help answer a present question. • Statistical vs. non-statistical designs for producing data: e.g., statistical vs. ethnographies or other case studies. • Statistical designs for producing data: these rely on either experimentsor observation based onsampling.

  3. Sampling: selects part of a population (a sample) to gain information about the whole population. • A sample survey is a kind of observational study because it does not attempt to impose treatments that influence responses. • Census: attempts to collect data on every individual in a population. • What are the advantages of a sample survey versus a census?

  4. Experiments: deliberately impose some treatment on individuals to observe their responses. • Experimental design, when carried out correctly, is the most effective of all research designs in controlling the effects of lurking variables. Why? • Because experimental design is intrinsically comparative.

  5. But experimental results typically cannot be generalized. Why not?

  6. The Principles of Experimental Design

  7. The main principles of experimental design are control, randomization, & replication.

  8. A fundamental research challenge is to minimizebias. • Bias: systematic over or under estimation of values—the systematic favoring of higher or lower values. • Bias usually can’t be detected by inspecting a given data set by itself. Why not? • Properly done, experimental design is the most effective way of minimizing bias.

  9. Control: comparative design makes it more likely—when carried out correctly—that influences other than the experimental variable operate equally on all experimental units (or subjects). • That is, it makes more it likely that lurking variables operate equally on all experimental units. • Control, then, reduces bias.

  10. Randomization (i.e. randomized assignment): enables impersonal chance to allocate experimental units (or subjects) to the treatment or control groups. • If done correctly, it makes more likely that influences other than the experimental variable operate equally on all experimental units (or subjects). • Randomization, then, reduces bias, as does comparative design.

  11. Replication: make sure that the number of experimental observations is large enough to permit a result that is statistically significant (i.e. an observed effect so large that it would rarely occur by chance).

  12. Why, though, can’t experimental results typically be generalized? • Because a properly conducted experiment randomizes assignment to the treatment and control groups, but it does not randomly sample from a wider population.

  13. Summary: Experimental Design • What are the basic features of experimental design? • How is experimental design intrinsically comparative? • In what ways is it the most effective research design? • What is statistical significance (versus practical or theoretical significance)?

  14. Potential Limitations of Experimental Designs • What are the potential limitations of experimental designs? • What critical questions need to be asked when assessing an experimental study?

  15. The Principles of Sample Design

  16. Sample Design • Population: the entire group of individuals (i.e. phenomena or entities) that we are trying to understand. • Sample: the part of the population that we actually examine in order to obtain information. • Sample design: the method used to choose the sample from the population. Inadequate sample design & execution lead to misleading conclusions by creating bias in the selection of observations.

  17. There are two kinds of error: • Bias: non-random (i.e. systematic) error—favors either higher or lower values. (2) Variability: random error—just as likely to involve higher as lower values & thus has neutral affect on central tendency. • We want to minimize variability, but bias is a greater concern: Why?

  18. The Two Basic Types of Samples • Voluntary (i.e. non-probability) sample: consists of people who choose themselves by responding to a general appeal. Not a probability sample & thus highly biased. • Probability sample: chosen by impersonal chance. We must know what samples are possible & what chance, or probability, each possible sample has. If conducted correctly, bias is minimized.

  19. Basic Types of Probability Sampling Design • Sampling relies on a sample frame: a list of elements from which a probability sample is drawn. • Simple random sample: consists of n individuals from the population chosen so that every set of n individuals (i.e. units) has an equal chance to be the sample actually selected. • Systematic random sample: randomize the list of elements to choose from; select a random number between 1 and 10; and then choose every kth element for inclusion in the sample. The result is virtually identical to random sampling.

  20. Stratified random sample: • Divides the population into groups of similar individuals (i.e. phenomena or entities), called strata. • (2) Chooses a separate simple random sample within each stratum & combines them to form the full sample.

  21. Stratified random sampling is based on an exhaustive list of the target population. • The sample is proportionalif the proportions of the sample chosen in the various strata are the same as those existing in the population. • Required statistical correction: see Stata manual.

  22. According to Freedman et al., Statistics, we shouldn’t exaggerate the benefits of stratification in reducing a sample’s variance.

  23. Multistage cluster sample: • Selects successively smaller groups (such as geographic units) within the population in stages, resulting in a sample consisting of clusters (i.e. groups) of individuals (i.e. phenomena or entities). • Samples from the clusters (i.e. not all clusters end up providing samples observations for the study). (3) Samples observations only from within the sample-selected clusters.

  24. Each stage in a multi-stage cluster sample may employ a simple random sample, a stratified random sample, or some other type of sample. • Sample observations are drawn from within the sample-selected clusters only, not from within every cluster. • Cluster sampling is used when it’s impossible or impractical to compile or observe an exhaustive list from the target population.

  25. Stratified vs. Cluster Sampling • In stratified sampling, a sample is drawn within every stratum, and the strata are the groups compared. • In cluster sampling, clusters are ways of identifying groups of observations, but a sample is not drawn from within every cluster. • See, e.g., Agresti & Finlay, Statistical Methods for the Social Sciences, pages 26-29.

  26. Multistage cluster sample example – U.S. Census: • Divide U.S. into geographic areas within states: primary sampling units (PSUs). • Divide each PSU into smaller geographic units--census blocks—then stratify the blocks by ethnic and other data. • Take a stratified sample of census blocks. • Sort each census block’s housing units into clusters of four nearby units. • Interview the households in a probability sample of these clusters.

  27. While a multistage cluster sample is the most commonly used sophisticated sample, it involves a key statistical problem: the observations within each sampled cluster tend to be more alike than are the observations between the sampled clusters. • This is because such sampling violates an assumption of inferential statistics: that the individuals (i.e. units or observations) are sampled not only randomly but also independently from each other. • This reduces the amount of statistical information about the sample’s variability.

  28. Required statistical correction: In Stata, type ‘help svy’.

  29. Sources of Non-Probability Bias in Probability Samples Even well-designed & well-executed probability samples can suffer from bias due to non-probability problems of:

  30. Undercoverage: the sample frame (i.e. list of elements from which the probability sample is drawn) does not adequately cover all relevant categories of elements. • Nonresponse: especially if non-randomly distributed. • Response bias: due to traits/behavior of the interviewer/researcher or respondent/subject • Poorly worded questions or problems due to order of questions

  31. Descriptive vs. Inferential Statistics • Descriptive statistics: summarizes the data. • Inferential statistics: makes inferences from a sample to a population.

  32. Toward Statistical Inference • Statistical inference: based on impersonal chance, we use data on sampled individuals (i.e. phenomena or entities) to infer conclusions about the wider population. • Parameter: a number that describes a population. It is a fixed number, but in practice we usually don’t know its value. • Statistic: a number that describes a sample. A statistic’s value is known when we’ve taken a sample, but it can vary from sample to sample. We often use a statistic to estimate an unknown parameter. This is called inferential statistics.

  33. A parameter is what we want to know about in a population: e.g., we want to know about the cholesterol levels of adults between the ages of 25 & 64 in South Florida. • A statistic is what we’ll learn from a random sample: e.g., the cholesterol levels of randomly sampled adults between the ages of 25 & 64 in South Florida.

  34. Sampling variability: the value of a statistic varies with repeated random sampling of the same size from the same population. • All of statistical inference is based on one idea: to see how trustworthy a procedure is, ask what would happen if it were repeated independently many times. See Freedman et al., Statistics.

  35. That is, ask what would happen if we took many independent random samples of the same size from the same population. • Take a large number of random samples of the same size from the same population. • Compute the mean for each sample. • Make a histogram of the values of the sample means. • Examine the distribution’s shape, center, & spread.

  36. E.g., a medical researcher wants to estimate the cholesterol levels of South Florida adults ages 25-64. • Let’s say that, as a start, the researcher measures the cholesterol values of 500 randomly sampled adults ages 25-64 (based on their places of residence & conducting a door-to-door random-sample survey).

  37. This, however, is only one sample. • What would happen if we repeated the random sample independently over & over again with the same size & in the same population?

  38. The sampling distribution of a statistic is the distribution of values taken by the statistic in all possible random samples of the same size from the same population. • It’s a way of conceptualizing what distribution would emerge if we could see all possible samples of the same size-n from the same population.

  39. Regarding the cholesterol study, what if the researcher could draw all possible random samples of n=100 from South Florida’s population of adults ages 25-64? • What would be the shape, center, & spread (including outliers) of the resulting distribution of cholesterol means from each of the random samples, lined up together?

  40. Thinking about sampling distribution helps us to understand what we’re trying to accomplish in drawing one or more actual random samples of size-n from the same population: big picture versus little picture. • Later in the course we’ll see its crucial importance for tests of statistical significance.

  41. How do we begin to connect the little picture of the actual random samples of size-n to the big picture of the conceptual ideal of the sampling distribution? • We do so by using a histogram to describe the means of each actual random sample of size-n, lined up together.

  42. E.g., regarding the cholesterol levels of the South Florida population of adults ages 25-65, the researcher obtains a huge amount of funding to take 1000 random samples of size 100. • The variable, of course, is cholesterol value: in each random sample of size 500, what’s the mean cholesterol level?

  43. Each time we take a random sample of 100, we compute the sample’s mean & standard deviation for cholesterol level. • And as we accumulate samples, we examine them together in one histogram to find out how much the sampled means of cholesterol either converge toward the center or spread out.

  44. At last, we have our total of 1000 random samples of size 100 from the South Florida population of adults ages 25-64. • According to the histogram, what’s the shape, center, & spread (including outliers) of the mean number of high-cholesterol persons for each of the 1000 random samples of size 100, lined up together?

  45. In short, we draw a random sample to obtain a statistic that we’ll use to estimate the unknown parameter (i.e. the unknown population value). • Our objective is not to do descriptive statistics, but rather to do inferential statistics.

More Related