1 / 25

Construct Validity and Reliability

Construct Validity. . What is Construct Validity. . Idea. Program. . Construct. Operationalization. Construct Validity refers to the degree to which inferences can legitimately be made from the operationalizations in your study to the theoretical constructs on which those operationalizations were based..

MikeCarlo
Download Presentation

Construct Validity and Reliability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Construct Validity and Reliability Presented by Khalid Elmansor

    2. Construct Validity

    3. What is Construct Validity 1/24/2012 3 CSC 426 Values and Computer Technology

    4. What is Construct Validity 1/24/2012 4 CSC 426 Values and Computer Technology

    5. Construct Validity Types 1/24/2012 5 CSC 426 Values and Computer Technology Translation validity attempts to assess the degree to which you accurately translated your construct into the operationalization. You focus on whether your program is a good reflection of the construct or not. In Criterion-Related validity, you examine whether your program behaves the way it should given you theory of constuct. It is based on validating a measure based on its relationship to another independent measure.Translation validity attempts to assess the degree to which you accurately translated your construct into the operationalization. You focus on whether your program is a good reflection of the construct or not. In Criterion-Related validity, you examine whether your program behaves the way it should given you theory of constuct. It is based on validating a measure based on its relationship to another independent measure.

    6. Construct Validity Types 1/24/2012 6 CSC 426 Values and Computer Technology Example is Math-ability test.Example is Math-ability test.

    7. Construct Validity Types 1/24/2012 7 CSC 426 Values and Computer Technology This approach assumes that you have a good detailed description of the content domain. This approach assumes that you have a good detailed description of the content domain.

    8. Construct Validity Types 1/24/2012 8 CSC 426 Values and Computer Technology This approach assumes that you have a good detailed description of the content domain. This approach assumes that you have a good detailed description of the content domain.

    9. Construct Validity Types 1/24/2012 9 CSC 426 Values and Computer Technology This approach assumes that you have a good detailed description of the content domain. This approach assumes that you have a good detailed description of the content domain.

    10. Construct Validity Types 1/24/2012 10 CSC 426 Values and Computer Technology This approach assumes that you have a good detailed description of the content domain. This approach assumes that you have a good detailed description of the content domain.

    11. Construct Validity Types 1/24/2012 11 CSC 426 Values and Computer Technology This approach assumes that you have a good detailed description of the content domain. This approach assumes that you have a good detailed description of the content domain.

    12. Threats to Construct Validity Inadequate Preoperational Explication of Constructs It means you did not do a good enough job of defining what you mean by construct Mono-Operation Bias It means if you only use a single version of a program in a single place at a single point in time, you may not be capturing the full picture of the construct Mono-Method Bias It means if you only use a single measure or a single observation, you may only measure part of the construct 1/24/2012 12 CSC 426 Values and Computer Technology Solutions: 1. Get experts to critique your operationalization.Solutions: 1. Get experts to critique your operationalization.

    13. Threats to Construct Validity (cont) Interaction of Different Treatments Is the observed outcome a consequence of your treatment or a combination of separate treatment? Interaction of Testing and Treatment Does testing or measurement itself make the observers more sensitive to the treatment? Restricted Generalizability Across Constructs Refer to unintended consequences that threat construct validity Confounding Constructs and Levels of Constructs Occur when the observed results are only true for a certain level of construct 1/24/2012 13 CSC 426 Values and Computer Technology Solutions: 1. Get experts to critique your operationalization.Solutions: 1. Get experts to critique your operationalization.

    14. How to assess Construct Validity The Nomological Network The Multitrait-Multimethod Matrix (MTMM) Pattern Matching 1/24/2012 14 CSC 426 Values and Computer Technology

    15. The Nomological Network It includes a theoretical framework for what you are trying to measure, an empirical framework for how you are going to measure it, and specification of linkages among and between these two frameworks It does not provide a practical and usable methodology for actually assessing construct validity 1/24/2012 15 CSC 426 Values and Computer Technology

    16. The Multitrait-Multimethod Matrix (MTMM) MTMM is a matrix of correlations arranged to facilitate the assessment of construct validity It is based on convergence and discriminant validity It assumes that you have several concepts and several measurement methods, and you measure each concept by each method To determine the strength of the construct validity: Reliability coefficients should be the highest in the matrix Coefficient in the validity diagonal should be significantly different from zero and high enough The same pattern of trait interrelationship should be seen in all triangles. 1/24/2012 16 CSC 426 Values and Computer Technology

    17. Pattern Matching It is an attempt to link two patterns: theoretical pattern and observed pattern It requires that you specify your theory of the constructs precisely! you structure the theoretical and observed patterns the same way so that you can directly correlate them Common Example: ANOVA table 1/24/2012 17 CSC 426 Values and Computer Technology

    18. Reliability

    19. Reliability In research, reliability means repeatability or consistency True score Theory is a simple model for measurement. It assumes that any observation is composed of a true value plus an error value X = T + e Measurement errors can be random or systematic To reduce measurement error If you collect data from people, make sure to train them Double-check the collected data thoroughly Use statistical procedures to adjust measurement error Use multiple measures of the same construct 1/24/2012 19 CSC 426 Values and Computer Technology

    20. Theory of Reliability Reliability can be expressed as You cannot compute reliability because you cannot calculate the variance of true scores! The range of reliability is between 0 and 1 The formula for estimating reliability 1/24/2012 20 CSC 426 Values and Computer Technology

    21. Types of Reliability Estimation Inter-rater or inter-observer reliability Is used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon Test-retest reliability Is used to assess the consistency of a measure from one time to another Parallel-forms reliability Is used to assess the consistency of the results of two tests constructed in the same way from the same content domain Internal consistency reliability Is used to assess the consistency of results across items within a test 1/24/2012 21 CSC 426 Values and Computer Technology

    22. Reliability and Validity 1/24/2012 22 CSC 426 Values and Computer Technology

    23. Thank you Any Questions?

    24. Extra slides

    25. Single correlation matrix 1/24/2012 25 CSC 426 Values and Computer Technology

    26. Pattern Matching Example 1/24/2012 26 CSC 426 Values and Computer Technology

More Related