Read by Tuesday February 28 th and bring copies to class

1. Read by Tuesday February 28thand bring copies to class • Class workshop: Critical appraisal of 2 outcome measure articles • Owens E et al. Paraspinal skin temperature patterns: an interexaminer and intraexaminer reliability study • Humphreys et al. An investigation into the validity of cervical spine motion palpation using subjects with congenital block vertebrae as a 'gold standard' Palmer College of Chiropractic West

2. Properties of Useful Clinical Outcome Measures Used in clinical practice as well as research Palmer College of Chiropractic West

3. Outcome Measures • The tools used to track the progress of a patient's health • The more measurable or quantifiable the OM is the better it is • ROM is a good example • A patient stating they are “feeling a little better today” is a bad example • OMs are collectively used to assess outcomes Palmer College of Chiropractic West

4. Outcomes Assessment (AO) • The collection and recording of information relative to healthcare processes • Measurements of quality, satisfaction, and effectiveness are essential for making healthcare decisions • Also needed to develop health policy • A means of documenting medical necessity (primarily for third party payers) Palmer College of Chiropractic West

5. OA Supports Continuing Care Palmer College of Chiropractic West

6. Properties of Useful Outcomes • Utility • The outcome measure must be useful • Reliability • It must be dependable • Validity • It must be able to measure what it is supposed to measure • Relatively easy to use • Cost effective Palmer College of Chiropractic West

7. Properties of Outcomes Cont. • Sensitivity • An OM’s ability to identify patients with a specific condition • Specificity • An OM’s ability to identify those who do not have the condition • Responsiveness • Ability to measure differences over time Palmer College of Chiropractic West

8. Useful Measures of Outcome • Questionnaires concerning: • General health status • Pain • Functional status • Patient satisfaction • Physiological outcomes • i.e., ROM, orthopedic tests, blood tests • Utilization measures • Cost measures Palmer College of Chiropractic West

9. OA Tools • Classified as being subjective or objective • Subjective • Relies on information derived from patients • Some think they are unreliable, but research has shown the opposite • Objective • Tests carried out on patients • ROM, strength, proprioception, etc. Palmer College of Chiropractic West

10. Subjective OA Tools • General Health • Pain Perception • Condition-specific • Psychometric • Disability Prediction • Patient Satisfaction Palmer College of Chiropractic West

11. General Health OA Tools • These tools can be used for all patients, regardless of their ailment • Since they are not condition specific • Because they deal with quality of life issues can be used for patients with HAs, back pain, digestive problems, etc. • Will usually parallel the patient’s level of disability Palmer College of Chiropractic West

12. General Health OA Tools Cont. • A weakness of general health OA tools • Other co-existing health conditions may “cloud” the main problem • Should be combined with a specific outcome measure such as a pain scale Palmer College of Chiropractic West

13. Pain Perception OA Tools • Pain is always subjective • Its measurement is primarily dependant upon the patient • Also how the practitioner draws out responses during the history and exam • Scales are often used where the patient chooses a level of pain that best matches theirs • i.e., mild, moderate, severe • or scaled 0-10 Palmer College of Chiropractic West

14. Pain Perception Cont. • Pain scales • Visual analog scales • Patient marks their pain level on a line that represents a scale ranging from no pain to terrible pain • Numerical pain scales • A range of numbers (usually from 0-10) with 0 being no pain and 10 being worst imaginable • Patient circles a number that is closest to their pain Palmer College of Chiropractic West

15. Pain Perception Cont. • McGill Pain Questionnaire • Consists of 20 pain-related items • Short form is 15 items • Pain Diagrams • Patient draws a representation of their pain on an outline of a body • Key to help them characterize the pain • i.e., 000 = ache, XXX = stabbing, /// = burning Palmer College of Chiropractic West

16. Condition-specific OA Tools • Spinal and non-spinal subcategories • Spinal condition-specific instruments • Numerous types are available • Also includes those that deal with headaches, dizziness, and others • Non-spinal • Extremities, knees, shoulders, etc. • Asthma, arthritis, depression, etc. Palmer College of Chiropractic West

17. Psychometric OA Tools • Include various instruments designed to evaluate: • Affect* • Depression/Psychological status • Malingering and symptom magnification *Expression of a subjectively experienced emotion like joy or anger Palmer College of Chiropractic West

18. Disability Prediction OA Tools • Questionnaires designed to help predict patients who are likely to become “problem cases” caused by • Chronic pain • Work dissatisfaction • Fear avoidance Palmer College of Chiropractic West

19. Patient Satisfaction OA Tools • Assess patients’ perception regarding • Acceptance of care • Technical competence of the doctor • Setting where care was provided • Effectiveness of the doctor Palmer College of Chiropractic West

20. Questionnaires • Subjective outcomes information is gathered from the patient • Self-administered • Scored by either the doctor or staff • Some are computerized • Typically very valid and reliable • Often more so than objective tests like x-rays, examination findings, etc. Palmer College of Chiropractic West

21. Questionnaires Cont. • The same outcome measures should be used throughout the course of treatment with each patient • Provide valid evidence to third parties in support of the care being rendered • Questionnaires are often called “survey instruments” • Especially in epidemiology Palmer College of Chiropractic West

22. Physiological Outcomes • Physical performance tests • Strength and endurance • Psychosocial • How the patient reacts to pain stimulus • ROM and flexibility • Including inclinometer evaluation, SLR, etc. • Proprioception • One-leg standing test • Functional activity measures • e.g., Timed “up and go” test Palmer College of Chiropractic West

23. NOIR Applies to Outcomes Measures • Nominal • Palpating for the presence or absence of subluxation • Ordinal • Grading muscle strength on a 0-5 scale • Interval • Temperature • Ratio • Range of motion Palmer College of Chiropractic West

24. Outcome Meanings • Health Care Customer - Meaning of Outcomes • Payers-purchasers Cost containment • Regulators Compliance • Administrators Efficiency - low utilization • Clinical Researchers Support for a premise • Outcomes Experts Patient’s benefit • Health Care Providers Clinical-health status (Hansen DT, Mior S, Mootz RD in Yeomans SG: The Clinical Application of Outcomes Assessment, Stamford Connecticut, Appleton & Lange, 2000) Palmer College of Chiropractic West

25. Reliability Experimental Designs Palmer College of Chiropractic West

26. Why Bother With Test Validity & Reliability? • Whenever a test or other measuring device is used as part of the data collection process, the validity and reliability of that test is important • It would be wrong to use a test that does not truly measure what it’s purported to measure • Use of a thermometer to test BP (extreme) • Use of AK to diagnose a brain tumor Palmer College of Chiropractic West

27. Test Validity & Reliability? Cont. • Test results are relied upon to show patient progress (or lack of) • If the measures are erroneous, the diagnosis and treatment plan may also be erroneous • Measurement tools used in practice must be both valid and reliable Palmer College of Chiropractic West

28. So What! • Some tests/devices marketed to chiropractors are not valid/reliable • You should question new or non-standard tests and equipment before wasting money • Both yours and the patients • May have insurance reimbursement problems • Possible legal problems Palmer College of Chiropractic West

29. Reliable Not Reliable Test Reliability • The degree of consistency of a test when it is repeated • Consistency is a function of random error • All tests have some random error (variability) • Reliable tests have less Palmer College of Chiropractic West

30. Test-retest Reliability • Measures stability over time in repeated applications of a test • Test-retest reliability checks how similar results are when the research is repeated under similar circumstances • In studies looking at test-retest reliability • All other variables that might affect the test results must be controlled • The condition being studied must remain constant from test to test Palmer College of Chiropractic West

31. Internal Consistency • Applicable to clinical questionnaires • e.g., Pain, disability, and general health • Tests the degree of correlation between • Individual questions and the total score • Individual questions and other questions on the instrument • Determines the degree that questions • Measure the same construct • And are consistent with each other Palmer College of Chiropractic West

32. Test Reliability • The consistency of a test • Both between repeated tests carried out by the same evaluator • Intraexaminer reliability • And between different evaluators • Interexaminer reliability • Determined by having two or more observers test the same patients and then correlate their observations Palmer College of Chiropractic West

33. A Reliable Test • Is reproducible when performed by different examiners • (Interexaminer Reliability) • And when repeatedly performed by the same examiner • (Intraexaminer Reliability) Palmer College of Chiropractic West

34. Test Validity • Refers to the degree which a test or other measuring device is truly measuring what it is intended to measure • For a test to be considered valid, it must be able identify the condition it is being used to test for • Some tests are more valid than others • e.g, Immune response markers for infection Palmer College of Chiropractic West

35. Face Validity • When evaluating the validity of an outcome measure - • Does it seem to have merit as a useful test? • Does it have reasonable “face” value? • Examples • Does it make sense to use measurement of cranial girth as a predictor for LB injury? • Initial ROM and pain level as a predictor of recovery from LB injury? Palmer College of Chiropractic West

36. Content Validity • A test’s ability to include or represent all of the content of a particular construct • As an example: • Can low back pain be adequately assessed by asking a question about neck pain? • No, because the question lacks content validity • How about by a question about smoking in a health survey? Palmer College of Chiropractic West

37. Content Validity cont. • Another example: • A test designed to detect radicular neurological involvement that only deals with muscle strength would have poor content validity • Deep tendon reflexes and superficial sensations are also major constructs that should be included Palmer College of Chiropractic West

38. Concurrent Validity • A measurement’s ability to vary directly with a measure of the same construct • Test validity is determined by comparison with an already valid test • An exam test to detect meniscus tears is compared with the findings of arthroscopic surgery (Gold standard) • Concurrent validity is established if tests yield similar results Palmer College of Chiropractic West

39. Concurrent Validity Cont. • Gold Standard • The most specific and sensitive test to diagnose a disease • The term used to describe a method or procedure that is widely recognized as the best available • Can be used to compare with other tests Palmer College of Chiropractic West

40. Convergent Validity • The ability of a measurement to correlate well with another measure that is related, but different • Range of motion measured with an inclinometer compared with a goniometer • Comparison of one version of a health survey instrument with another • Measures of constructs that theoretically should be related to each other are, in fact, observed to be related to each other Palmer College of Chiropractic West

41. Convergent Validity Cont. • To establish convergent validity, one must show that measures that should be related are in reality related • Example survey questions attempting to measure the construct of disability – • I am unable to lift more than X pounds • I am unable to walk more than X blocks • Are they related regarding the construct? • Do they both depict disability? Palmer College of Chiropractic West

42. Discriminant Validity • The ability of a test to discriminate findings into categories like “normal vs. abnormal” or “positive vs. negative” • Discriminant validity is important for screening tests • Measures of constructs that theoretically should not be related to each other are, in fact, observed to not be related to each other Palmer College of Chiropractic West

43. Discriminant Validity Cont. • To establish discriminant validity, one must show that measures that should not be related are in fact not related • The relationship between measures from differentconstructs should be very low • For example: • A measure of visual acuity and a measure of knee function should be independent of one another Palmer College of Chiropractic West

44. Predictive Validity • A test’s capacity to be a valid screening tool for some future health problem • Are positive findings detected on a postural exam valid predictors for future back pain • Is elevated cholesterol a valid predictor for developing diabetes Palmer College of Chiropractic West

45. Predictive Validity cont. • Predictive validity is established by measuring the trait now and waiting to see if the event occurs as predicted Palmer College of Chiropractic West

46. Responsiveness • For a test to be responsive • It must be able to detect clinically important changes • It must be sensitive to real change • A responsive test will accurately measure the amount of change in a condition when change actually occurs Palmer College of Chiropractic West

47. Percent Change • A way of evaluating responsiveness • Typically expressed as a percentage • Initial (baseline) score is compared to the post-treatment score • Example – ROM 30 degrees at baseline and 60 degrees post-treatment • Change score = 30/60 = .5 = 50% Palmer College of Chiropractic West

48. t-test • Studies involving groups may be evaluated for change using a t-test (the equivalent groups variety) • Initial mean scores are compared with post • ANOVA when more than two evaluations are involved Palmer College of Chiropractic West