Phd research seminar series valid research designs
This presentation is the property of its rightful owner.
Sponsored Links
1 / 20

PhD Research Seminar Series: Valid Research Designs PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on
  • Presentation posted in: General

PhD Research Seminar Series: Valid Research Designs. Dr. K. A. Korb University of Jos. Outline. Threats to valid Research Design Construct Validity Internal Validity External Validity Reliability. Dr. K. A. Korb University of Jos. Reliability: Consistency of results. Reliable.

Download Presentation

PhD Research Seminar Series: Valid Research Designs

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Phd research seminar series valid research designs

PhD Research Seminar Series:Valid Research Designs

Dr. K. A. Korb

University of Jos


Outline

Outline

  • Threats to valid Research Design

    • Construct Validity

    • Internal Validity

    • External Validity

    • Reliability

Dr. K. A. Korb

University of Jos


Reliability consistency of results

Reliability: Consistency of results

Reliable

Unreliable

Dr. K. A. Korb

University of Jos


Validity measuring what is supposed to be measured

Validity: Measuring what is supposed to be measured

Valid

Invalid

Dr. K. A. Korb

University of Jos


Research example

Research Example

  • The effect of regular leisure reading on reading achievement in primary school.

  • Students in a primary school will be randomly assigned to either a treatment or a control group.

    • The treatment group will spend 10 minutes everyday reading a book with an adult.

    • In an attempt to keep the treatment and control groups as similar as possible, the control group will also spend 10 minutes with an adult everyday, but will instead do math problems.

Dr. K. A. Korb

University of Jos


Research example1

Research Example

IV:

Regular

Leisure

Reading

DV:

Reading

Achievement

Experimental Group

Reading

Achievement

Read Book

Every Day

with Adult

Reading

Achievement

Control Group

Do Maths

Every Day

with Adult

Reading

Achievement

Reading

Achievement

Dr. K. A. Korb

University of Jos


Construct validity

Construct Validity

  • Construct Validity: Establishing valid operational measures for the concepts being studied

    • This relates to:

      • Independent Variable

      • Dependent Variable

  • Review:

    • Construct Definition: General explanation of the construct by relating it to other constructs

    • Operational Definition: Statement of specifically how the construct will be measured or implemented in your study

Dr. K. A. Korb

University of Jos


Review construct vs operational definitions

Review:Construct vs. Operational Definitions

Construct

Operational Definition

WAEC Scores

Final Course Exam

AcademicAchievement

NECO Scores

State Exams

Self-Report of Parents’

Yearly Income

Economic

Status

Free/Reduced

Lunch Eligibility

Dr. K. A. Korb

University of Jos


Research example construct validity focuses on

Research ExampleConstruct validity focuses on:

IV:

Regular

Reading

DV:

Reading

Achievement

Experimental Group

Reading

Achievement

Read a Book

Every Day

Reading

Achievement

Reading

Achievement

Reading

Achievement

Control Group

Dr. K. A. Korb

University of Jos


Construct validity1

Construct Validity

  • When you develop your research project, you start with a theory.

    • For example, regular leisure reading will lead to increased reading achievement scores.

  • Then you operationalize your theory into your research study.

    • You translate “regular leisure reading” into how it will be implemented in your study – reading a book with an adult everyday for 10 minutes.

    • You translate “reading achievement scores” into how it will be measured in your study – classroom exam scores.

  • When you finish collecting data, you hope to be able to move back to theory.

    • You want to be able to say that not only did reading a book for 10 minutes everyday with an adult lead to higher classroom exam scores on reading achievement, but also that regular leisure reading will lead to higher reading achievement scores in general.

    • However, in order to do that, you must have construct valid measures of your independent and dependent variables.

Dr. K. A. Korb

University of Jos


Research example2

Research Example

IV:

Regular

Reading

DV:

Reading

Achievement

Theory:

Scores on

reading section

of classroom

exams

Read book

every day with

an adult for

10 min.

Research

Study:

IV:

Regular

Reading

DV:

Reading

Achievement

Theory:

Dr. K. A. Korb

University of Jos


Construct validity2

Construct Validity

  • Enables you to generalize from the treatment and measures in your study to the general constructs of your treatment and measures

  • Threats to Construct Validity

    • Inadequate explanation of constructs in your paper

    • Flawed matching of operationalization to constructs

      • The measures do not align with the constructs they were designed to measure.

Dr. K. A. Korb

University of Jos


Internal validity

Internal Validity

  • Internal validity: Extent to which variables other than the treatment provide plausible explanations to the experimental results

    • Only relevant for experimental designs

      • Primary consideration for interventions

    • Extraneous variable: Any variable other than treatment variable that if not controlled can affect the experimental outcome

    • Hold constant or eliminate all extraneous variables that might affect the posttest

    • Goal of Research Design: Create set of conditions so any observed changes in your dependent variable can be attributed to experimental treatment instead of extraneous variables

  • Do variables other than the treatment provide plausible explanations to the experimental results?

Dr. K. A. Korb

University of Jos


Research example3

Research Example

Maturation

IV:

Regular

Reading

DV:

Reading

Achievement

Interest

Experimental Group

Reading

Achievement

Read a Book

Every Day

Reading

Achievement

Reading

Achievement

Reading

Achievement

Control Group

Reading

Ability

Adult

Attention

Dr. K. A. Korb

University of Jos


Threats to internal validity

Threats to Internal Validity

  • History: If treatment lasts over time, other events may influence dependent variable

    • To prevent: Keep the control group equivalent in all aspects but treatment

  • Maturation: Physical or psychological changes may influence the dependent variable

    • To prevent: Use a control group

  • Testing: Giving a pretest may increase performance on the post-test

    • To prevent: Don’t use a pre-test OR Use different pre- and post-tests

  • Instrumentation: Nature of measuring instrument has changed between pre- and post-test

    • To prevent: Do not use a pre-test OR Use the same for pre- and post-test

  • Selection Bias: Treatment and control groups are different on an important extraneous variable

    • To prevent: Random assignment OR Use a pre-test

  • NOTE: Many of these preventions contradict. You must think about which threats are most relevant to your study and then develop a strategy for overcoming the most important threats.

Dr. K. A. Korb

University of Jos


External validity

External Validity

  • External Validity: Establishing the group of people to which the research findings can be generalized

Dr. K. A. Korb

University of Jos


Research example4

Research Example

Population

Experimental

Group

Reading

Achievement

Read a Book

Every Day

Reading

Achievement

Sample

Control

Group

Reading

Achievement

Reading

Achievement

Dr. K. A. Korb

University of Jos


Types of external validity

Types of External Validity

  • Population Validity: Generalizing the results from experimental sample to a defined population

    • To establish: Give a thorough explanation of the constitution of your sample and how your sample was selected.

  • Ecological Validity: Generalizing the results of a study from the artificial conditions created by researcher to real-life conditions

    • To establish:

      • Give an adequate description of treatment

      • Give an adequate description of the measure for the dependent variable

    • Threats to Ecological Validity:

      • Novelty: The treatment was different than typical treatment

      • Experimenter Effect: The particular person delivering the treatment affects results

Dr. K. A. Korb

University of Jos


Reliability

Reliability

  • Reliability: Demonstrating that the operations of the study can be repeated with the same results

    • Conduct research as if someone were always looking over your shoulder

    • Report the procedure of your study in explicit detail

Dr. K. A. Korb

University of Jos


Other issues in valid research design

Other Issues in Valid Research Design

  • Experimenter Bias: Researcher’s expectations about the outcome of experiment influence participants’ response

    • To prevent: Use research assistants who are blind to the study (aka do not know the purpose of the study)

  • Treatment Fidelity: Extent to which treatment conditions are implemented according to the researcher’s specifications

    • To support:

      • Carefully train research assistants

      • Periodically check up on research assistants unexpectedly to determine if they are following guidelines

  • Strong Experimental Treatments: Developing a treatment that will make a robust effect on the dependent variable

    • To support:

      • Develop a thorough understand of your dependent variable and how it can be influenced.

      • Spend much thoughtful time developing your treatment.

Dr. K. A. Korb

University of Jos


  • Login