1 / 7

Evaluating a Meta-Analysis

Evaluating a Meta-Analysis. Validity Construct validity Internal validity Statistical conclusion validity External validity Theoretical contribution. Construct Validity. Refers to how accurately the theoretical variables being discussed are implemented in a study

dagmar
Download Presentation

Evaluating a Meta-Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluating a Meta-Analysis • Validity • Construct validity • Internal validity • Statistical conclusion validity • External validity • Theoretical contribution

  2. Construct Validity • Refers to how accurately the theoretical variables being discussed are implemented in a study • Definitions of variables in the effect and moderators should be consistent with the literature • Meta should combine comparable studies • Must consider the construct validity of primary research

  3. Internal Validity • Refers to the validity of causal relations inferred from the results of a study • Moderator analyses only provide correlational information, so causal inferences should be examined critically • Should look for confounds among moderators

  4. Statistical Conclusion Validity • Refers to whether the analytic procedures were performed correctly • Use of fixed vs. random effects models in relation to the inferences being drawn • Assumption of independence • Potentially low power of moderators • Unreliable coding • Uneven distribution among moderator levels • Fishing effects

  5. External Validity • Refers to how well the research captures fundamental aspects of the situation to which you want to draw conclusions • Are the boundaries of the analysis appropriate? • Was the literature search exhaustive? • To what extent did the primary research have strong external validity?

  6. Theoretical Contribution • What implications does the meta have for existing theories? • How novel are the implications? • How broad are the implications? • Are the implications made clear in the presentation? • Are the results from the meta used to support a new theory? • Does the meta-analysis identify new areas for research?

  7. Miller & Pollock’s (1994) Classification of Meta-Analyses • Type A • Summarizes the strength of an effect • Type B • Summarizes the strength of an effect and uses moderator analyses to explain hetereogeneity • Type C • Provides new evidence for a theory by examining variables that have not been manipulated within a study

More Related