slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Evaluating the Experiment from the Inside: Internal Validity PowerPoint Presentation
Download Presentation
Evaluating the Experiment from the Inside: Internal Validity

Loading in 2 Seconds...

play fullscreen
1 / 29

Evaluating the Experiment from the Inside: Internal Validity - PowerPoint PPT Presentation


  • 187 Views
  • Uploaded on

Evaluating the Experiment from the Inside: Internal Validity Taking a Broader Perspective: The Problem of External Validity Handling a Nonsignificant Outcome. An experiment is internally valid when the effects on the dependent variable are due to the independent variable.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Evaluating the Experiment from the Inside: Internal Validity' - ronda


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Evaluating the Experiment from the Inside: Internal Validity

Taking a Broader Perspective: The Problem of External Validity

Handling a Nonsignificant Outcome

what is internal validity

An experiment is internally validwhen the effects on the dependent variable are due to the independent variable.

An internally valid experiment is free of confounding.

What is internal validity?

Evaluating the Experiment from the Inside

what is a manipulation check

A manipulation check evaluates how well the experimenter manipulated the experimental situation.

A manipulation check determines whether subjects followed directions and were appropriately affected by our treatments.

What is a manipulation check?

Evaluating the Experiment from the Inside

what did orne 1969 mean by a pact of ignorance

Subjectsexpect their data to be discarded if they guess the experimental hypothesis, and don’t volunteer this information to the experimenter.

Experimenters don’t want to test additional subjects and may take subject reports at face value.

What did Orne (1969) mean by a pact of ignorance?

Evaluating the Experiment from the Inside

how can researchers overcome this problem

Debrief subjects after the experiment and convey that you want to know if they guessed the hypothesis.

Provide incentives for guessing the hypothesis.

How can researchers overcome this problem?

Evaluating the Experiment from the Inside

which mistakes could produce this threat to internal validity

Selecting the wrong statistical test

Using a t-test to analyze ordinal data.

Improperly using a statistical test

Calculating multiple t-tests.

Drawing the wrong conclusions from the test Reporting p = .07 as a trend.

Which mistakes could produce this threat to internal validity?

Evaluating the Experiment from the Inside

explain external validity

An experiment is externally validwhen its findings can be extended to other situations and populations.

Explain external validity.

The Problem of External Validity

what two requirements must an externally valid study satisfy

1. The experiment must be internally valid.

2. The experimental findings can be replicated.

What two requirements must an externally valid study satisfy?

The Problem of External Validity

what does it mean to generalize across subjects why is this important

The findings can be extended to a larger group than our sample.

Generalizing across subjects is critical to the external validity and usefulness of experimental findings.

What does it mean to generalize across subjects? Why is this important?

The Problem of External Validity

which problems prevent us from generalizing across subjects

The samples used in psychological research are often biased and may not represent the larger population.

The samples may not always represent even college sophomores since we heavily depend on volunteers.

Which problems prevent us from generalizing across subjects?

The Problem of External Validity

explain generalizing from procedures to concepts

Experimental variables like anger may have multiple operational definitions.

When we generalize from our experimental results, we move from discussing our specific operational definition of anger to discussing the concept of anger itself.

Explain generalizing from procedures to concepts.

The Problem of External Validity

why is this a problem in research

It is dangerous to generalize from a single experiment’s operational definition of anger.

We cannot be sure of the reliability or validity of our procedures.

Why is this a problem in research?

The Problem of External Validity

what is research significance

A study achieves research significancewhen its findings clarify or extend knowledge gained from previous studies and raise implications for broader theoretical issues.

What is research significance?

The Problem of External Validity

when should we question novel findings

We should question novel findings when they contradict prior findings that have been successfully replicated. The burden of proof is on the experimenter who claims novel findings to explain this discrepancy.

When should we question novel findings?

The Problem of External Validity

what is the problem in extending laboratory findings

Since extraneous variables are uncontrolled in real world setting and operate in complex combinations, they can modify the influenceof our individual variables.

What is the problem in extending laboratory findings?

The Problem of External Validity

what is the trade off between laboratory and field experiments

The trade-off is between the laboratory’s more precise control of extraneous variables and the field experiment’s greater realism and external validity.

What is the trade-off between laboratory and field experiments?

The Problem of External Validity

what did hanson 1980 find about the agreement between laboratory and field studies

Hanson (1980) found that more laboratory than field studies reported a positive correlation between reported attitudes and behavior.

What did Hanson (1980) find about the agreement between laboratory and field studies?

The Problem of External Validity

when can we extend results from an experiment to everyday life

We can’t confirm external validity until additional studies are completed in field settings.

Researchers can increase and verify the external validity of laboratory findings using aggregation, multivariate designs, nonreactive measurements, field experiments, and naturalistic observation.

When can we extend results from an experiment to everyday life?

The Problem of External Validity

what is aggregation what is its logic

Aggregationisthe grouping together and averaging of data to increase external validity.

Combining the results of experiments with different subjects and methodologies increases the generality and external validity of our findings.

What is aggregation? What is its logic?

The Problem of External Validity

how is this logic shared by meta analysis

Meta-analysis uses statistical analysis to combine and quantify data from many comparable experiments to calculate anaverage effect size.

How is this logic shared by meta-analysis?

The Problem of External Validity

how does aggregation establish external validity

Aggregation establishes external validity by combining the results of experiments performed using different subjects, stimuli and/or situations, trials or occasions, and measures.

How does aggregation establish external validity?

The Problem of External Validity

what is a multivariate design

A multivariate designstudies multiple DVs.

For example, a study of repetitive strain places a computer keyboard at different distances from the subject IV) and measures the effect on three different muscle groups (3 DVs).

What is a multivariate design?

The Problem of External Validity

what is the advantage of multivariate designs

Multivariate designs allow us to study the effect of an independent variable on combinations of dependent variables.

These designs better simulate the complexity of the real world than univariate designs and provide more detailed information.

What is the advantage of multivariate designs?

The Problem of External Validity

how should researchers handle a nonsignificant outcome

Accept the outcome, don’t reframe your result as “almost significant.”

Examine the experimental procedures for design flaws.

If the design appears sound, decide whether the hypothesis was reasonable.

How should researchers handle a nonsignificant outcome?

Handling a Nonsignificant Outcome

how should we handle the possibility of faulty procedures

Check for possible causes of a nonsignificant outcome like:

confounding

extraneous variables that increase within-subjects variability

weak manipulation of the IV

How should we handle the possibility of faulty procedures?

Handling a Nonsignificant Outcome

how should we handle the possibility of faulty procedures1

inconsistent or flawed procedures

ceiling and floor effects

insufficient power

How should we handle the possibility of faulty procedures?

Handling a Nonsignificant Outcome

how should we handle the possibility of a faulty hypothesis

If previous studies supported the hypothesis and ours did not, look for differences in experimental design or sample.

If there was no previous support and our design and execution were good, we may have to revise or discard our hypothesis.

How should we handle the possibility of a faulty hypothesis?

Handling a Nonsignificant Outcome