Chapter Thirteen

1. Chapter Thirteen Data Collection and Measurement

2. Measurement • The process by which categories or numbers are used to reflect or indicate concepts and constructs • A concept is a general idea not directly observable in the real world • A construct is a concept specified in such a way that it is observable in the real world

3. Levels of a Research Study • Theoretical - interconnected propositions or statements of relationship between concepts • Conceptual - statements of relationships between two or more constructs • Operational - indicates how each of the constructs will be measured or operationalized. It refers to the indicators used to reflect the constructs as well as to the procedures used to collect & analyze data

4. Theoretical Substruction • The dynamic thinking process used to move from the theoretical level to the operational or measurement level of a study • It illustrates the hierarchical order among the major constituents of a study • It identifies the foundational elements of a study, determines the relationships among the elements, & presents this in a diagram

5. Measurement • Measurement is the linkage between the conceptual and the operational levels of a research project • Two key issues in this linkage: validity or the the congruence between a concept and the indicators of the concept, and reliability or the extent to which an instrument yields similar results on repeated measures

6. Validity • Face validity..on the face of it... • Content validity…reflects the dimension implied by the concept • Concurrent validity…correlation of one measure with another • Predictive validity...predict accurately • Construct validity…distinguishes participants who differ on the construct • Internal validity…treatment produces changes in dependent variable

7. Validity Cont. • Internal validity…treatment produces changes in dependent variable • External validity…extrapolation from study to the other groups in general • In qualitative research…”credibility” is the issue

8. Validity in Qualitative Research • A qualitative study is credible when it presents descriptions of experiences that the people having had that experience immediately recognize as their own • … the best test of rigor in qualitative work is when the researcher creates “true-to-life, and meaningful portraits, stories, & landscapes of human experiences…” (Sandelowski, 1993)

9. Rigor in Qualitative Research • Keep careful records • Avoid the holistic fallacy • Guard against elite bias • Don’t be taken over by respondent

10. Reliability • Instruments ability to produce the same results on repeated measures • Terms such as dependability, consistency, stability & accuracy are often used interchangeably • accuracy reflects the instrument’s ability to measure the true value (free from random measurement error) being measured

11. Reliability in Quantitative Research • Reliability is a relative term, expressed as a correlation …1.00 (perfect reliability) to 0.00 (absence of reliability) • Reliability coefficients of .70 are acceptable (Nunnally, 1978) • Estimates of reliability need to be determined each time the instrument is used

12. Three Attributes of Reliability • Stability • Internal Consistency • Equivalence

13. Stability • Concerned with consistency of results with repeated measures • Test-retest procedures - response should be identical on both occasions assuming the variables measured remain the same at the two testing times • Gillis (1997) tested the reliability of the ALQ using the test-retest procedure

14. Internal Consistency • Refers to the homogeneity of the instrument or the ability of the items in the instrument to measure the same variable • Items are strongly correlated to each other • The > intercorrelations, the > internal consistency • Measures to test internal consistency: KR-20, item-total correlations, split-half method, cronbach’s alpha

15. Equivalence • Degree of agreement among 2 or more different observers using the same measurement tool, or • Degree of agreement among 2 or more alternate forms of an instrument or tool • Determined by correlating the 2 scores with each other • Interrater reliability may be determined several times in a study

16. Reliability in Qualitative Research • In qualitative research replication is not possible because the circumstances & individuals can never be the same at some later time

17. Measurement Error • Any deviation from the true value • True value is the underlying exact quantity of a variable at any given time • Variables change over time & any measure will vary slightly from 1 day to the next • Measure are made up of the following:Measure=TV+ (SE+RE)

18. Measurement Error Cont. • Systematic error…non-random error that systematically over- or under-estimates a value (eg., persons not answering a question are given the lowest value • Random error…random fluctuations around the true value. Not a problematic…should average out.

19. Tips for Reducing Measurement Error • Take average of several measures • Use different indicators • Use random sampling procedures • Use sensitive measures • Avoid confusion in wordings • Error check data carefully • Reduce subject/experimenter expectations

20. Levels of Measurement The level of measurement achieved is important because it constrains the type of statistical analysis that can be performed on your data. • Nominal • Ordinal • Ratio

21. The Effects of Reduced Levels of Measurement • Underestimating the relative importance of a variable if it is poorly measured • The greater the reduction in measurement precision, the greater the drop in correlations between variables • Precisely measured variables will appear to be more important than poorly measured ones

22. Data Collection • Process of gathering data from identified participants to answer a research question • A variety of quantitative & qualitative methods are available depending upon research question • indexes or scales, biochemical & physiological measures, projective techniques, delphi techniques, unstructured interviews, focus groups, observation sessions, historical documents

23. Item Analysis • Good indexes “discriminate well” • Example of test item development • test graded, students divided into upper and lower quartile • examine performance on each question • select those questions that discriminate best

24. Discrimination of Items Percent Correct Each Item Bottom Top Question # 25% 25% 1 40.0 80.0 2 5.0 95.0 3 60.0 55.0 4 80.0 80.0 5 10.0 40.0 6 20.0 60.0

25. Selecting Index Items • Review conceptual definition • Develop measures for each dimension • Pre-test index • Pilot test index

26. Tips for Wording Likert Items • The “and” alert: avoid multiple dimensions • Strongly Agree on right hand side 9-points • response set issue • Avoid negatives like “not” simply use negative wording. • Vary strength of wording to produce variation in response • Exercise….items for a euthanasia index

27. Other Scales • Semantic Differential: Here a variety of anchors are used and people place themselves or others on a continuum: shy/outgoing; bookworm/social butterfly

28. Other Scales Cont. • Magnitude Estimations: subjects use numbers or line lengths to indicate perceptions. Very good for comparisons: yields ratio level measures. Comparing liking of teachers; seriousness of crimes; liking of one community compared to another one, etc.

29. Other Scales (cont’d) • Visual Analogue Scales (VAS) measure the intensity of participant’s sensations & feelings about the strength of their attitudes, beliefs, & opinions about specific stimuli such as fatigue, pain, health, etc. • Usually a 100 mm line is used with anchor words or phrases at each end

30. Delphi technique • A panel of experts used for multiple data collection, analysis and processing • Obtains the opinions of experts without the financial cost or inconvenience of bringing expert • opinions of a variety of experts are condensed into precise statements people together

31. Physiological Measures • Particularly appropriate in studies designed to assess the impact of nursing interventions on bodily functions • Provide objective & sensitive measurements that are difficult for the participant to distort • e.g. vital signs, % body fat, muscle strength, salivary enzyme levels, serum glucose, etc.

32. Observational Measurement • Well suited to phenomena that are best viewed from a holistic rather than a reductionistic perspective • Observations maybe structured, unstructured, or semi-structured; occur in natural or controlled settings • To be scientific they must meet four critieria: consistent with study objectives; systematic & standard plan for recording; checked & controlled; related to scientific concepts or theories

33. Interviews • A face-to-face verbal interaction to illicit information from the respondent usually through direct questioning • structured, semi-structured, nonstructured • Advantage of probing, in-depth data, used with participants who are not literate • Limited by time, cost , sample size