Scwk 240 week 6
This presentation is the property of its rightful owner.
Sponsored Links
1 / 19

ScWk 240 Week 6 PowerPoint PPT Presentation


  • 60 Views
  • Uploaded on
  • Presentation posted in: General

ScWk 240 Week 6. Measurement Error Introduction to Survey Development “England and America are two countries divided by a common language.” George Bernard Shaw. Review--Variables:.

Download Presentation

ScWk 240 Week 6

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Scwk 240 week 6

ScWk 240 Week 6

Measurement Error

Introduction to Survey Development

“England and America are two countries divided by a common language.”

George Bernard Shaw

ScWk 240 Week 6


Review variables

Review--Variables:

  • When we use the word “outcome” are we talking about the independent variable or the dependent variable?___________

  • In an evaluative study, the primary independent variable is the __________

  • “This study examines the impact of acculturation on the use of health services by immigrants, controlling for income, age, and health status.

    • What is the dependent variable? ___________

    • What is (are) the independent variable(s)? _______________________________________________________________

ScWk 240 Week 6


Review conceptualization

Review: Conceptualization

  • We’re talking about complex ideas

    • Depression

    • Self esteem

    • Acculturation

  • How would you conceptualize “acculturation”?

    • Give a one sentence conceptual definition

    • What dimensions would you include?

ScWk 240 Week 6


Operationalization the next step

Operationalization – the next step

  • “Operationalization” – the process of defining a specific variable, and identifying the relevant attributes of the variable

  • Example – Depression “loss of energy” dimension

    • Variable: “loss of energy”

    • Measured by: (CIDI instrument) During the period when your sadness was most severe and frequent, did you feel tired or low in energy even when you had not been working very hard?

    • Attributes:

      • Yes

      • No

      • Don’t know

ScWk 240 Week 6


Now operationalize acculturation

Now operationalize “acculturation”

  • Pick one dimension

  • For that dimension, describe the variable in a simple statement

    • First, give the variable a label (name) in 1-3 words

    • Include in the simple statement “…as measured by…”

  • For that variable, identify one set of attributes

ScWk 240 Week 6


Small group exercise

Small group exercise:

  • Pick a concept as a group

  • Identify three dimensions of that concept

  • Choose one dimension

  • For that dimension, identify and describe one variable in a simple statement

    • First, give the variable a label (name) in 1-3 words

    • Include in the simple statement “…as measured by…”

  • For that variable, identify one set of attributes

ScWk 240 Week 6


Student report

Student Report

What is “culturally competent measurement” in research? Relate this to one or more (of the 5) areas of the Transcultural Perspective. Elisse

ScWk 240 Week 6


Error in measurement

“Error” in Measurement

  • What is “measurement error”?

    Measurement error occurs when we obtain data that do not accurately reflect the concept we want to measure.

  • There are two main sources of measurement error:

    • Random error: inconsistent error. There’s no pattern or predictability

    • Systematic error: consistent error. There is pattern and predictability

ScWk 240 Week 6


Random error inconsistent response

Random Error – inconsistent response

  • Environmental factors: i.e. surveys administered in a hot room will vary from surveys administered in a more comfortable setting

  • Questions with jargon or technical terms will be interpreted differently by different respondents (or other factors related to a poorly constructed instrument)

  • Written survey response will differ by educational level or reading ability of respondents

ScWk 240 Week 6


Systematic error consistently and predictably wrong response

Systematic error—consistently and predictably “wrong” response

  • Social desirability bias’ – respondents portray themselves in the best light

  • ‘Acquiescent response set’ – Agree or disagree with everything

  • Cultural ‘bias’ – cultural definitions of concepts different than those intended by researcher (i.e. you can always expect physical somatization as a stated primary symptom of depression in many non-Western cultures)

  • Bias from wording of questions – i.e. “Are you resistant to treatment?”

ScWk 240 Week 6


Review potential for systematic or random error

Review: Potential for systematic or random error?

  • People are more alert in the morning than after lunch ____________

  • The meaning of “living independently” varies cross-culturally __________

  • Male respondents tend to minimize their aggressive tendencies _________

  • A 16 y.o. taking a test: “I just circle ‘C’ each time and hope for the best” (hint-this is not Acquiescent Response) _________

  • The person entering data is not careful enough and mis-codes some of the responses ________

ScWk 240 Week 6


Student reports

Student reports

  • What’s the difference between “reliability” and “validity” of a measure? Glorialee

  • How is validity related to cultural issues? Dawei

ScWk 240 Week 6


Measuring error validity and reliability

Measuring error—Validity and reliability

  • The presence of error is not “all or nothing” – there’s always going to be some error

  • We have to measure it to understand its impact

  • Reliability – measures random error—consistency of meaning

  • Validity – measures systematic error—accuracy of meaning

ScWk 240 Week 6


Reliability

Reliability

  • Concerned with whether or not a particular technique done repeatedly would yield the same result each time

    • Inter-rater reliability—Two researchers rating the same behavior: do they rate them the same?

    • Test-retest reliability—Is the response from the first test the same as the response from a re-test?

    • Alternate forms reliability—Is there the same response from slightly different versions of same measure?

    • Internal consistency reliability & use of coefficient alpha—Are similar questions (about the same concept) in survey are answered in the same way?

ScWk 240 Week 6


You ll see this in your lit review

You‘ll see this in your lit review…

  • Methods section—measures used: the author will report on reliability testing (if done) using the coefficient alpha

    • A number that tells you, overall, how well the questions in an instrument measuring the same concepts correlate with each other (internal consistency reliability)

    • A coefficient alpha is considered good if it is .80 or above

    • The author will then report “The instrument showed good internal consistency reliability”

ScWk 240 Week 6


Validity

Validity

  • Concerned with the accuracy of meaning—are you measuring what you originally intended?

  • Types of validity

    • Face validity – experts’ understanding of concept

    • Content validity – Does measure cover all relevant dimensions?

    • Criterion-related validity – Does measure of concept agree with (or relate with) another related concept (external criterion)?

    • Construct validity – Does measure agree with or correlate with measures of other theoretically related concepts?

ScWk 240 Week 6


Validity examples

Validity examples

  • Validity studies show that a set of questions about alcohol dependence correlates well with the observed daily amount of alcohol consumption. Criterion-related validity

  • Most experts would agree that children with Attention Deficit Disorder with Hyperactivity have difficulty staying on tasks (even those the child likes). Face validity

  • Couples who test as having low levels of marital satisfaction communicate less effectively, leading to high levels of conflict and, eventually, divorce. Construct validity

  • The Beck Depression Scale covers those dimensions of depression that are most supported by evidence. Content validity

ScWk 240 Week 6


Student reports for next week

Student reports for next week

  • 1. Dawei

  • What are some strategies for ensuring reliability when developing survey questions?___Amanda

  • What are some strategies for ensuring validity when developing survey questions?____Sarah

  • What is the main difference between “open-ende” and “closed-ended” questions? Selene

ScWk 240 Week 6


Concepts to know

Concepts to know

  • Measurement error

  • Random error

  • Systematic error

  • Reliability (Inter-rater, Test-retest, Alternate forms, Internal consistency)

  • Coefficient alpha

  • Validity (Face, Content, Criterion-related, Construct)

ScWk 240 Week 6


  • Login