# Evaluation Research Step by Step - PowerPoint PPT Presentation

1 / 16

Evaluation Research Step by Step. Step 1 – Formulate Research Question Step 2 – Operationalize Concepts Valid and Reliable Indicators Step 3 – Decide on Sampling Technique Draw Sample Step 4 – Select Data Collection Technique Collect Data Step 5 – Analyze Data

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Evaluation Research Step by Step

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

### Evaluation Research Step by Step

Step 1 – Formulate Research Question

Step 2 – Operationalize Concepts

• Valid and Reliable Indicators

Step 3 – Decide on Sampling Technique

• Draw Sample

Step 4 – Select Data Collection Technique

• Collect Data

Step 5 – Analyze Data

Step 6 – Write up the Report

### Evaluation Research Step by Step

• Steps 1 and 2 have already been covered in previous power points.

### Step 3 - Drawing A Sample

• Identify Population

• Decide on either Random (probability) or Nonrandom (nonprobability) Sampling Technique

### Sampling

• Terms to Know

• Population: Total group to be studied

• Case: Units to be sampled

• (individual, county, etc.)

• Sample: Group to be interviewed

• Sampling Frame: List from which we obtain the sample

• (a roster from a class, or a directory from an organization, etc.)

### Sampling

• Types of Sampling Designs

• Simple Random Sampling

• Single number assigned to each case

• Table of random numbers is used

• Assumes a complete list of the population

• Laborious

• Systematic Sampling

• Every “x”th element from a sample frame is selected

• Determine size needed

• Take random start

• Select every “x”th element

• Probability Proportionate to Size (PPS)

• Larger ones (clusters) have greater probability of being selected

### Sampling

• Types of Sampling Designs

• Stratified Sampling

• Sample separately from different key categories

Example:

Determine number needed from each racial/ethnic group (this number should be approximately the same for each group).

Total #

# Needed

African Americans300100

Hispanics200100

Asian Americans100100

Anglo Americans500100

Total Population1000400

Sample separately from each group taking proportion needed to get approximately equal numbers from each racial/ethnic group

African AmericansNeed 50% sample

HispanicsNeed 50% sample

Asian AmericansNeed 100% sample

Anglo AmericansNeed 20% sample

### Sampling

• Types of Sampling Designs

• Multistage Cluster Sampling

• Used when don’t have a complete list

• Sample in stages (units within units)

Example of multistage random sample:

(1) Universities

USU U of UBYU

(4) Classes

Landscape, Architecture and Environmental Planning (LAEP)

(2) Colleges

Agriculture

Engineering

Theory of Design

Plants, Soils and Climate Department (PSC)

Landscape Materials

(3) Departments

Molly

Trish

(5) Students

Jim

### Determining Sample Size

N

=

D2

Appendix 2Recommended sample sizes for two different precision levels

N + 1

.9604

Sample Size

+/- 5 % +/- 10%

Sample Size

+/- 5 % +/- 10%

Population Size

Population Size

101027516374

151430017276

201932518077

252435018778

302837519480

353240020181

403642520782

454045021282

504447521883

554850022283

6052100028691

6556200033395

7059300035397

7563400036498

8066500037098

8570600037598

9073700037899

9576800038199

1008151900038399

125965610,00038599

1501106115,00039099

1751226420,000392100

2001346725,000394100

2251447050,000397100

25015472100,000398100

### Nonrandom Sampling

• Availability or Convenience Samples

• Select those who are available or convenient

• Quota Samples - select representative size of different types of respondents

• Purposive Samples

• Need specific types of information – select those who will have that information

• Theoretical Samples

• Theory specifies characteristics of your units of analysis. For instance, a theory of inequality may require you to have respondents at different levels of income.

• Snowball Samples

• Hard to Reach – Hard to identify populations

### Nonrandom – Nonprobability Sampling

• Availability or Convenience Samples

• Sample classrooms of students here on campus because they are available and convenient

• Availability or Convenience Using Quota

• Sample classrooms of students here on campus because they are available and convenient, BUT

• Select classes with approximately equal numbers of men and women because there are approximately equal numbers in university

### Nonrandom - Nonprobability Sampling

• Purposive Samples

• Select Key Informants who can provide information you need

• If you want to look at community leader’s attitudes toward homeless then you would select

• Law enforcement

• Heads of social service agencies

• Three Guidelines

• Knowledge about the culture/situation or experience

• Willing to talk

• Representative of the range of points of view

• Two Tests

• Completeness – overall sense of what they think

• Saturation – not hearing anything new

### Nonrandom – Nonprobability Sampling

• Theoretical Samples

• Theory specifies characteristics of your units of analysis

• Example – Research on parent/teacher interaction using Expectation States Theory.

• Research Questions - How do parents and teachers interact? How is interaction influenced by race and social class of parent? How are expectations influenced by socioeconomic characteristics of parents.

• Theory specifies that as inequality between parent and teacher increase, so does “expert power” of teacher – and conformity of parent.

### NonRandom- Nonprobability Sampling

• Snowball Samples

• Key Informant or GATEKEEPER who then refers you to someone else

• Sample size “snowballs” or increases rapidly.

• Used when you are working with groups that are reluctant to be studied, and you need to gain confidence of group members.

• Example of groups where snowball samples are used– Gang members, Alcoholics anonymous members, etc.

### Ways that People Generally GatherInformation – Data Collection Techniques

• Participant Observation

• Content Analysis

• Surveys/Intakes/Questionnaires

• This excludes non-empirical ways of collecting information (intuition, inspiration, 6th sense etc.)

### Types of Data Collection Techniques Paired with Most Likely Sampling Technique

• Participant Observation – generally uses nonrandom sample.

• Content Analysis – generally uses nonrandom sample.

• Surveys/Intakes/Questionnaires – generally uses random sample.

• This excludes non-empirical ways of collecting information (intuition, inspiration, 6th sense etc.)

### Contact Information

• Dr. Carol Albrecht

• Assessment Specialist

• 979-777-2421

• Carol.albrecht@usu.edu