chapter 5 developing a measurement strategy
Download
Skip this Video
Download Presentation
Chapter 5: Developing a Measurement Strategy

Loading in 2 Seconds...

play fullscreen
1 / 19

Chapter 5: Developing a Measurement Strategy - PowerPoint PPT Presentation


  • 138 Views
  • Uploaded on

Chapter 5: Developing a Measurement Strategy. Scales of Measurement Reliability and Validity Modalities of Measurement Locating and Evaluating Measures. Reliability & Validity. Why is reliability important? Theory can’t do without it Constructs can’t be valid unless they are reliable

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Chapter 5: Developing a Measurement Strategy' - milt


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
chapter 5 developing a measurement strategy
Chapter 5: Developing a Measurement Strategy
  • Scales of Measurement
  • Reliability and Validity
  • Modalities of Measurement
  • Locating and Evaluating Measures
reliability validity
Reliability & Validity
  • Why is reliability important?
    • Theory can’t do without it
    • Constructs can’t be valid unless they are reliable
    • A phenomenon must be reliably demonstrated before it can have construct status
    • E.g. ESP, Loch Ness Monster
  • What are some examples of phenomena important to:
    • Counselors?
    • I/O psychologists?
reliability validity1
Reliability & Validity
  • Manifest variables:
    • Directly observable
  • Hypothetical Constructs:
    • Not directly observed (inferred)
  • What are some examples?
    • Behavioral?
    • Psychological constructs?
  • Which types (manifest/hypothetical) are:
    • Operational definitions?
    • Hypothetical constructs?
  • How are Operational definitions and Hypothetical constructs related?
    • Operational definitions represent hypothetical constructs
reliability validity and measurement error
Reliability & Validityand measurement error
  • Reliability & Validity
    • Reliability – consistency
    • Validity – accuracy
    • Polygraph
      • Reliable and valid?
    • Can something be reliable but not valid?
    • Can something be valid but not reliable?
  • Measurement Error

Xo= T+E (Observed score = True score + Error)

Randon v. systematic error

What kind of error is introduced when an applicant fakes on a personality measure? Other examples?

assessing reliability
Assessing Reliability
  • Forms
    • Test-retest
    • Alternate forms (equivalent forms)
    • Interrater reliability
    • Internal consistency
      • Split half
      • Cronbach’s alpha (coefficient alpha)
        • mean r among all items
  • Which one to use?
    • Give some examples
  • Standards for reliability
    • What’s acceptable?
assessing validity
Assessing Validity
  • Validity
    • Inferred
    • Depends on different types of evidence
    • Varies in how much (degree)
    • Trichotomized into: high, moderate, low
    • Specific to what it’s used for (valid for what?)
    • refers to inferences drawn, not the measure itself
    • A unitary construct (with three types of evidence)
      • Content,
      • criterion-related (predictive, concurrent)
      • Construct
validity
Validity
  • Convergent
    • Related to other constructs it should be related to
  • Divergent
    • Not related to other constructs it should not be related to

which should be convergent and which divergent?

    • IQ and Depression
    • Work motivation and conscientiousness
    • Happiness and wealth
    • Need for approval and caring about how one looks
    • Ability to sell and friendliness
    • Aggression and frustration level
validity determining degree of validity
Validity:Determining Degree of Validity
  • Measure validation process
    • See figure 5-4
  • Theory of trait
  • Test hypotheses
  • Confirm/disconfirm
  • Do it again
  • Never ending (seemingly)
    • Eg. CPI, WGCTA, WPT, NEO, Beck DI
validity differential
Validity: Differential
  • Valid for specific population, group?
    • Moral reasoning?
      • Different cultures, Gender (Gilligan, ’82)?
    • Math tests?
      • Gender?
    • PONS test?
      • Gender?
    • Personality inventories?
      • Fakers v. honest?
      • Job setting v. home setting?
reliability validity modalities of measurement
Reliability & Validity: Modalities of Measurement
  • Self-Report Measures
    • Advantages
    • Limitations
  • Behavioral Measures
    • Advantages
    • Limitations
  • Physiological Measures
    • Advantages
    • Limitations
  • Choosing a Measurement Modality
modalities
Modalities:
  • Self-Report
    • Cognitive
    • Affective
    • Retrospective v. Hypothetical (behavioral intention)
    • Kinesthetic
  • Advantages
    • Easy to collect
    • Easy to administer
    • Feelings (cannot observe)
    • Thought process (Policy capturing)
    • May be more accurate than observation. Why?
  • Limitations
    • Accuracy of recall
    • Willingness to report
    • Verbal skills needed
self report modalities
Self Report Modalities:
  • How to
    • Ask about actual previous behaviors
    • Use dichotomous or categorical for or behavior or behavioral intentions
    • Use Likert (5, 7 or 9) for attitudes
    • Include both positively and negatively scaled items
    • Use graphic labels or benchmarks
    • Use multiple items to capture a concept
    • Avoid leading double barreled questions (usually)
modalities behavioral measures
Modalities:Behavioral Measures
  • Uses
    • Behavior is object of study
    • Operational definitions
    • Nonverbal may be clue to feelings/ physio state
  • Types of measures
    • Frequency (bar presses)
    • Rate
    • Duration
    • Intensity
    • Accuracy
    • Persistence
  • Examples?
behavioral measures
Behavioral Measures
  • Advantages
    • Surreptitious (sneaky)
      • Avoids evaluation apprehension
    • Capture automatic behaviors
      • Often not premeditated
    • More accurate than behavioral intention
  • Limitations
    • “what you see is what you get” What limitation is implied here?
      • Interpretation is inferred by observer
    • Highly situation-specific
      • That’s why R. Hogan likes to use personality trait as predictor
    • Need trained observers
modalities physiological measures
Modalities: Physiological Measures
  • Purposes
    • Of interest in and of itself
    • As op definition of psychological state
      • E.g. anxiety, arousal, lying
  • Advantages
    • Most direct (no intervening human observer
    • Highly precise
  • Limitations
    • Need equipment, trained administrators
    • Obtrusive
    • Constrain freedom
    • Source of unreliability (testing effects)
      • E.g. polygraph
modalities choosing locating measurement modality
Modalities:Choosing & Locating Measurement Modality
  • Choosing: Self-report, behavioral, physiological
    • Rank them for level of validity
    • Which one should be used?
    • See table 5-2 relative advantages and limitations
    • Multiple operationism
    • Categories of Measures (figure 5-7, p. 147)
      • Type: manifest v. hypothetical
      • Hypothetical construct:
        • Psychometric (used for individual scores)
        • research measures (used for mean scores; norms)
          • Developed (use when it fits procedure)
          • Ad hoc (when you have to develop it)
modalities locating evlauting measures
Modalities: Locating & Evlauting Measures
  • Locating Measures
    • MMY
      • (Buros)
    • Tests in Print
    • Directory of Unpublished Experimental Measures
      • (Goldman & Sanders, 1997)
    • Measures of Personality and Social Psychological Attitudes
      • (Robinson, Shaver,& Wrightsman, ’91)
    • Organizational Measures
      • Warr, Cook and Wall??
modalities evaluating them
Modalities: Evaluating Them
  • Theoretical Background
    • How much construct standing?
  • Quality of Development
    • Participant samples, norms
  • Reliability & Validity
    • What are good coefficients for each?
    • Valid for what?
  • Freedom from Response Bias
    • Social desirability
    • Acquiescence bias
developing a measurement strategy summary
Developing a Measurement StrategySummary
  • Scales of Measurement
  • Reliability and Validity
  • Modalities of Measurement
  • Locating and Evaluating Measures
ad