Beginning the Research Design

1. Beginning the Research Design Theory, Questions, Hypotheses Designing Tests for the above: Conceptualization, Operationalization, and Measurement

2. Conceptualization • Process of specifying what we mean when we use particular terms. • Produces an agreed upon meaning for a concept for the purposes of research. • Describes the indicators we'll use to measure the concept and the different aspects of the concept.

3. From Concept to Measurement • Progression from what a term means to measurement in a scientific study: • Conceptualization • Nominal Definition • Operational Definition • Measurements in the Real World

4. Four Levels of Measurement • Nominal - offer names for labels for characteristics (gender, birthplace). • Ordinal - variables with attributes we can logically rank and order.

5. Four Levels of Measurement • Interval - distances separating variables (temperature scale). • Ratio - attributes composing a variable are based on a true zero point (age).

6. Measurements Things Scientists Measure • Direct observables - things that can be observed simply and directly. • Indirect observables - things that require more subtle observations. • Constructs - based on observations that cannot be observed.

7. Measurement Quality • Reliability • Validity

8. Reliablity GENERAL DEFINITION: Accuracy or precision of a measuring instrument. SPECIFIC DEFINITIONS: 1. Similar results - stability, dependability predictability 2. Accuracy – consistency 3. Absence of random or chance error -- extent to which errors of measurement are present in a measuring instrument

9. Tests for Checking Reliability • Test-retest method - take the same measurement more than once. • Equivalence: use "essentially the same" measurement items on the same instrument or on different instruments and compare the answers (same time period). Split-half, Random half, alternate forms. Use established measures.

10. Internal Validity DEFINITION: the ability of the measuring instrument to measure one's theoretical concepts. METHODS OF ASSESSING VALIDITY: PRAGMATIC (or Criterion) VALIDITY: predict to an outside criterion and compare the outcome to the outside criterion a. Concurrent: comparison to an existing or current outside criterion b. Predictive: comparison to a future outside criterion FACE VALIDITY: obvious and self-evident content

11. Validity (cont.) CONTENT VALIDITY: representativeness of what is being measured to the intended concepts (capturing all the dimensions of the social concept) CONSTRUCT VALIDITY: adequacy of the measuring instrument for measuring the theoretical concepts and relationships; also adequacy of the logical structure of the conceptualization and operationalization.

12. Construct Validity

13. External Validity

14. Political Polls and Survey Sampling • In the 2000 Presidential election, pollsters came within a couple of percentage points of estimating the votes of 100 million people. • To gather this information, they interviewed fewer than 2,000 people.

15. Election Eve Polls - U.S. Presidential Candidates, 2000

16. Observation and Sampling • Polls and other forms of social research, rest on observations. • The task of researchers is to select the key aspects to observe, or sample. • Generalizing from a sample to a larger population is called probability sampling and involves random selection.

17. Types of Nonprobability Sampling • Reliance on available subjects: • Only justified if less risky sampling methods are not possible. • Researchers must exercise caution in generalizing from their data when this method is used.

18. Types of Nonprobability Sampling • Purposive or judgmental sampling • Selecting a sample based on knowledge of a population, its elements, and the purpose of the study. • Used when field researchers are interested in studying cases that don’t fit into regular patterns of attitudes and behaviors

19. Types of Nonprobability Sampling • Snowball sampling • Appropriate when members of a population are difficult to locate. • Researcher collects data on members of the target population she can locate, then asks them to help locate other members of that population.

20. Types of Nonprobability Sampling • Quota sampling • Begin with a matrix of the population. • Data is collected from people with the characteristics of a given cell. • Each group is assigned a weight appropriate to their portion of the population. • Data should provide a representation of the total population.

21. Probability Sampling • Used when researchers want precise, statistical descriptions of large populations. • A sample of individuals from a population must contain the same variations that exist in the population.

22. Probability Sampling • Most effective method for selection of study elements. • Avoids researchers biases in element selection. • Permits estimates of sampling error.

23. Populations and Sampling Frames • Findings based on a sample represent the aggregation of elements that compose the sampling frame. • Sampling frames do not always include all the elements their names imply. • All elements must have equal representation in the frame.

24. Types of Sampling Designs • Simple random sampling (SRS) • Systematic sampling • Stratified sampling

25. Simple Random Sampling • Feasible only with the simplest sampling frame. • Basic method assumed in most statistical computations

26. Systematic Sampling • Slightly more accurate than simple random sampling. • Arrangement of elements in the list can result in a biased sample.

27. Stratified Sampling • Rather than selecting sample for population at large, researcher draws from homogenous subsets of the population. • Results in a greater degree of representativeness by decreasing the probable sampling error.

28. Multistage Cluster Sampling • Used when it's not possible or practical to create a list of all the elements that compose the target population. • Involves repetition of two basic steps: listing and sampling. • Highly efficient but less accurate.