Research Methods

Research Methods Operationalization And measurement

Second Stage: Operationalization • Formulation of Theory • Operationalization of Theory • Selection of Appropriate Research Techniques • Observation of Behavior (Data Collection) • Analysis of Data • Interpretation of Results

Hypotheses Generation • Hypothesis: • an explicit statement that indicates how a researcher thinks the phenomena of interest are related. • It represents the proposed explanation for some phenomenon. • Indicates how an independent variable is thought to affect, influence, or alter a dependent variable. • A proposed relationship that may be true or false.

Good Hypotheses • Hypotheses should be empirical statements: proposed explanations for relationships that exist in the real world. • Hypotheses should be general: a hypothesis should explain a general phenomenon rather than a particular occurrence. • Hypotheses should be plausible: some logical reason for thinking that it may be confirmed. • Hypotheses should be specific: it should specify the expected relationship between two variables. • Hypotheses should relate directly to the data collected.

Directional Hypotheses • Hypotheses should be specific. IOW, they should state exactly how the independent variable relates to the dependent variable. • Positive Relationship: where the concepts are predicted to increase or decrease in size together. • Negative Relationship: where one concepts increases in size or amount while the other decreases in size or amount.

Unit of Analysis • One of the most important aspects of research design is determining the unit of analysis. • This is where we specify the types or levels of political actor to which t he hypothesis is thought to apply. • There are numerous kinds of units we can collect data on: • Individuals • Groups • States • Agencies • Organizations

U of A continued • Cross-level analysis: sometimes we collect data on one unit of analysis to answer questions about another unit of analysis. • The purpose in CLA is to make an ecological inference: the use of aggregate data to study the behavior of individuals. • Data on voting districts  individual voting behavior • CAVEAT: Avoid the ecological fallacy: where a relationship found at the aggregate level is not operative at the individual level. • State voting data used to infer about the relationships b/w district voting data.

Measurement • Measurement: systematic observation and representation by scores or numerals of the variables we have decided to investigate. • Operational definition: deciding what kinds of empirical observations should be made to measure the occurrence of an attribute or behavior.

The level of measurement refers to the relationship among the values that are assigned to the attributes for a variable It is important to distinguish between the values of a variable and the level of measurment Measuring Variables

Levels of Measurement • There are typically four levels of measurement that are defined: • Nominal • Ordinal • Interval • Ratio

Knowing the level of measurement helps you decide how to interpret the data from that variable. Knowing the level of measurement helps you decide what statistical analysis is appropriate on the values that were assigned. It's important to recognize that there is a hierarchy implied in the level of measurement idea. Levels of Measurement

Nominal & Ordinal • In nominal measurement the numerical values just "name" the attribute uniquely. • No ordering of the cases is implied. For example, jersey numbers in basketball are measures at the nominal level. A player with number 30 is not more of anything than a player with number 15, and is certainly not twice whatever number 15 is. • In ordinal measurement the attributes can be rank-ordered. • Here, distances between attributes do not have any meaning. For example, on a survey you might code Educational Attainment as 0=less than H.S.; 1=some H.S.; 2=H.S. degree; 3=some college; 4=college degree; 5=post college. In this measure, higher numbers mean more education. But is distance from 0 to 1 same as 3 to 4? Of course not. The interval between values is not interpretable in an ordinal measure.

Interval & Ratio • In interval measurement the distance between attributes does have meaning. • For example, when we measure temperature (in Fahrenheit), the distance from 30-40 is same as distance from 70-80. The interval between values is interpretable. Because of this, it makes sense to compute an average of an interval variable, where it doesn't make sense to do so for ordinal scales. But note that in interval measurement ratios don't make any sense - 80 degrees is not twice as hot as 40 degrees • In ratio measurement there is always an absolute zero that is meaningful. • This means that you can construct a meaningful fraction (or ratio) with a ratio variable. Weight is a ratio variable. In applied social research most "count" variables are ratio, for example, the number of clients in past six months. Why? Because you can have zero clients and because it is meaningful to say that "...we had twice as many clients in the past six months as we did in the previous six months."

Levels & Research Design • At lower levels of measurement, assumptions tend to be less restrictive and data analyses tend to be less sensitive. At each level up the hierarchy, the current level includes all of the qualities of the one below it and adds something new • In general, it is desirable to have a higher level of measurement (e.g., interval or ratio) rather than a lower one (nominal or ordinal).

True Score Theory • True Score Theory is a theory about measurement. Like all theories, you need to recognize that it is not proven -- it is postulated as a model of how the world operates. Like many very powerful model, the true score theory is a very simple one. • Essentially, true score theory maintains that every measurement is an additive composite of two components: true ability (or the true level) of the respondent on that measure; and random error.

We observe the measurement -- the score on the test, the total for a self-esteem instrument, the scale value for a person's weight. We don't observe what's on the right side of the equation (only God knows what those values are!), we assume that there are two components to the right side. The ‘true’ value The error in our measurement of that value True Score Theory

Error • The true score theory is a good simple model for measurement, but it may not always be an accurate reflection of reality. • In particular, it assumes that any observation is composed of the true value plus some random error value. But is that reasonable? What if all error is not random? • Isn't it possible that some errors are systematic, that they hold across most or all of the members of a group? • One way to deal with this notion is to revise the simple true score model by dividing the error component into two subcomponents, random error and systematic error. here, we'll look at the differences between these two types of errors and try to diagnose their effects on our research.

Random Error • Random error is caused by any factors that randomly affect measurement of the variable across the sample. • For instance, each person's mood can inflate or deflate their performance on any occasion. In a particular testing, some children may be feeling in a good mood and others may be depressed. • If mood affects their performance on the measure, it may artificially inflate the observed scores for some children and artificially deflate them for others. • Random Error is often referred to as ‘noise.’ • Random Error does not effect averages.

The important thing about random error is that it does not have any consistent effects across the entire sample. Instead, it pushes observed scores up or down randomly. This means that if we could see all of the random errors in a distribution they would have to sum to 0 -- there would be as many negative errors as positive ones. Random Error

Systematic Error • Systematic error is caused by any factors that systematically affect measurement of the variable across the sample. • For instance, if there is loud traffic going by just outside of a classroom where students are taking a test, this noise is liable to affect all of the children's scores -- in this case, systematically lowering them. • Unlike random error, systematic errors tend to be consistently either positive or negative -- because of this, systematic error is sometimes considered to be bias in measurement.

Systematic error, or bias, is a real threat to your research. Because it affects the average results, it may cause you to report a relationship that doesn’t exist or miss a relationship that does exist. Avoiding bias in our research is an important technique for producing good research. Systematic Error

Reducing & Eliminating Errors • So, how can we reduce measurement errors, random or systematic? • One thing you can do is to pilot test your instruments, getting feedback from your respondents regarding how easy or hard the measure was and information about how the testing environment affected their performance. • Second, if you are gathering measures using people to collect the data (as interviewers or observers) you should make sure you train them thoroughly so that they aren't inadvertently introducing error.

R & E Errors • Third, when you collect the data for your study you should double-check the data thoroughly. All data entry for computer analysis should be "double-punched" and verified. This means that you enter the data twice, the second time having your data entry machine check that you are typing the exact same data you did the first time. • Fourth, you can use statistical procedures to adjust for measurement error. These range from rather simple formulas you can apply directly to your data to very complex modeling procedures for modeling the error and its effects. • Finally, one of the best things you can do to deal with measurement errors, especially systematic errors, is to use multiple measures of the same construct. Especially if the different measures don't share the same systematic errors, you will be able to triangulate across the multiple measures and get a more accurate sense of what's going on.

How do we measure Unemployment? • Concepts • Definitions • How do we collect data on it? • What should that data tell us? • Why do we want to know about unemployment to begin with?

Unemployment: Federal definition • The definition of unemployment used in this report is the standard Federal definition of the percent of individuals in the labor force who were not employed. • The labor force is defined as individuals who were employed, were on lay-off, or had sought work within the preceding four weeks. Although this is the most commonly used measure of unemployment, other measures are used.

Unemployment: How is it measured? • Because unemployment insurance records relate only to persons who have applied for such benefits, and since it is impractical to actually count every unemployed person each month, the Government conducts a monthly sample survey called the Current Population Survey (CPS) to measure the extent of unemployment in the country. The CPS has been conducted in the United States every month since 1940 when it began as a Work Projects Administration project.

Unemployment Defining the Concepts • The basic concepts involved in identifying the employed and unemployed are quite simple: • People with jobs are employed. • People who are jobless, looking for jobs, and available for work are unemployed. • People who are neither employed nor unemployed are not in the labor force.

Operational Definition of Unemployment • The survey is designed so that each person age 16 and over who is not in an institution such as a prison or mental hospital or on active duty in the Armed Forces is counted and classified in only one group. • The sum of the employed and the unemployed constitutes the civilian labor force. • Persons not in the labor force combined with those in the civilian labor force constitute the civilian noninstitutional population 16 years of age and over.

Reliability & Validity • In research, the term "reliable" can means dependable in a general sense, but that's not a precise enough definition. What does it mean to have a dependable measure or observation in a research context? • In research, the term reliability means "repeatability" or "consistency". • A measure is considered reliable if it would give us the same result over and over again (assuming that what we are measuring isn't changing!).

Research Methods