Disability Management and the Effects of Workplace Interventions: A Meta-Analysis IFDM GLADNET-IDMRN Symposium John Lui

1. Disability Managementand the Effects of Workplace Interventions:A Meta-AnalysisIFDM GLADNET-IDMRN SymposiumJohn LuiNorm HurshDavid Rosenthal

2. Our underlying interests… • DM is acknowledged as a proven approach that effectively decreases the economic costs and human impact of disability • Earliest DM practices targeted RTW outcomes as a cornerstone of DM • BUT, no meta-analyses examining an aggregation of empirical studies has been conducted meta-analysis

3. Our underlying interests… • DM is now a global movement, with developing standards, and accepted practices • Early RTW practices have expanded, DM has evolved, yet benefit systems across countries vary widely • We wanted to look at the state-of-the-art outcome studies conducted internationally meta-analysis

4. Our underlying interests… • Our thinking was…lets look at the most likely DM practice that most countries adopt • How strong is the research around what we may find to be most consistently defined practice • What are the drivers of evidence-based practice…of developing best-practice approaches to our work meta-analysis

5. Meta-analysis • Gold standard of best practice in medicine • In medicine, with its positivist scientific methods tradition, the “gold standard” for scientific evidence is still randomized clinical trials and the method of choice for determining the cumulative evidence of the effectiveness of a treatment is meta-analysis. • Randomized clinical trials • Meta-analysis meta-analysis

6. Study Authors • David A. Rosenthal • University of Wisconsin- Madison • John Lui • University of Wisconsin- Stout • Norm Hursh • Boston University • Li Jialiang • National University -Singapore • James A. Curcio • Hewitt Associates meta-analysis

7. This study applied meta-analytic processes to investigate the aggregate effect size of the studies that met criteria for inclusion in the research. Initial aggregation processes are within studies (when necessary) and subsequently, across studies, both providing ES estimates. meta-analysis

8. Criteria for inclusion of studies consisted of six areas: • 1) population of interest • 2) nature of intervention or strategy • 3) provider of intervention • 4) receiver of intervention • 5) outcomes and • 6) study design – quantitative meta-analysis

9. Populations of interest include a) workers who are off work due to MSK conditions; pain‑related conditions that were episodic or non-episodic or associated with a degenerative or nondegenerative condition; chronic pain; or b) the workers’ compensation claimant population. meta-analysis

10. The category of nature of intervention or strategy is specifically aimed at improving RTW outcomes, including disability management; case management; education to workplace staff, insurance case managers or workers; changes in general organizational factors, but specifically aimed at improving RTW outcomes and decreasing durations of disability and associated direct labor costs.. meta-analysis

11. Providers of intervention come from the workplace, or from an insurance company (private or governmental) or third-party claims administrator (TPA) which could be provided from the workplace. They could also come from healthcare providers in very close collaboration with the workplace. meta-analysis

12. Receivers of intervention include workers, workplace staff, and insurance company/TPA case managers. meta-analysis

13. Outcomes include work disability durations, described as self-reported time to return to work, time on wage replacement benefits (Workers’ Comp or Short/Long-Term Disability), total duration of lost time (in calendar or business days), total labor cost of direct lost time, recurrences; and point-prevalence of status (e.g., back at work versus not back at work). meta-analysis

14. Other outcomes include the associated costs of healthcare, indirect costs associated with wage replacement and intervention (training, administrative, etc.); as well as the quality of life, including mental health, functional status, general physical health during and/or after work interruption, quality of work life, medication taking during and/or after work interruption. meta-analysis

15. Search • EBSCOhost was used as a primary searching reference system. Four databases were searched for relevant studies including CINAHL Plus with Full Text, MEDLINE, PsycARTICLES, and PsycINFO between January 1996 and October 2009. In addition, relevant research institutes, including Institute for Work and Health (IWH), and National Institute for Occupational Safety and Health (NIOSH) were also explored. meta-analysis

16. Terms used in searching included workers’ compensation, workplace intervention, return to work, return to work and intervention, disability management, short-term disability, long-term disability, .standardized metrics, benefit program performance, and benefit program evaluation.. In total, 2765 papers were identified by the search. meta-analysis

17. Study designs must be quantitative and include randomized controlled trial (RCT), nonrandomized trial, cross-sectional, pre-postdesign, time series, case control, and cohorts (retrospective and prospective). meta-analysis

18. Our Criteria for exclusion of studies consisted of: • 1) population of interest • Populations of interest include mental health conditions as a primary condition, phantom limb pain, short duration of self-limiting pain, and pain associated with a malignant condition. meta-analysis

19. Exclusion • 2) nature of intervention or strategy • Natures of intervention or strategy include policies, primary prevention, workplace ergonomic interventions, and clinical interventions provided outside the workplace. meta-analysis

20. Exclusion • 3) provider of intervention • Providers of intervention come from healthcare providers with no or minimal integration with the workplace. meta-analysis

21. Exclusion • 4) outcomes • Outcomes include absenteeism unrelated to MSK or other pain-related conditions. meta-analysis

22. Exclusion • 5) study design – not quantitative or quantitative consisting of non-comparative studies, such as case series and case studies. meta-analysis

23. Studies included (n=15): US – 4 Norway - 1 Netherlands – 3 Sweden – 2 Finland – 1 North America – 1 Canada – 3 meta-analysis

24. Results • we obtained an aggregate effect size d+ = 0.372 What does this mean? meta-analysis

25. More specifically, computation of an aggregate effect size for each study was accomplished on a Microsoft Excel worksheet. For the 15 studies retrieved for the Disability Management interventions in business and industry that met the criteria for workplace interventions to address managing disability in the workplace and measurable outcome criteria such as cost savings or reduced lost days at work. meta-analysis

26. we obtained an aggregate effect size d+ = 0.372, indicating that at the conclusion of treatment, workers in the intervention groups scored, on average, nearly half of a standard deviation higher than workers not receiving such interventions. meta-analysis

27. Meta-analysis Process Issues • Comprehensive review • Inclusion criteria • Finding studies – data bases • Publication bias • Significance bias meta-analysis

28. The Tyranny of P Values • Cohen (1994) argued that the results of statistical significance tests (i.e., the test statistic and associated p value) are not valid indicators of study findings, primarily because they are largely determined by the sample size of the study. • Meta-analysis goes beyond this “illusion of conflicting findings” (Hunter & Schmidt, 1996, p. 326) by ignoring p values in favor of an index of the strength of association between variables, referred to as the observed effect size for each study. meta-analysis

29. Sample size • Effect sizes derived from small N studies tend to have large sampling variances, and correspondingly large standard errors of estimate (reflecting relatively low precision). • Effect sizes derived from large N studies have relatively small sampling variances, correspondingly small standard errors, reflecting their relatively greater precision. • For this reason, the inverse of the variance for each effect size serves as a weight in meta-analysis. This procedure gives greater weight to more precise effect estimates and lesser weight to less precise effect estimates in computing the aggregate d+. (See VCV matrix in Excel example) meta-analysis

30. Effect sizes (Cohen) • Standardized Mean Difference Effect Size • large effect: d = .80 • medium effect: d = .50 • small effect: d = .20 meta-analysis

31. Fixed Versus Random Effects • Used Random effects versus fixed (two sources of error - model for generalizability). • In a random-effects meta-analysis, the variance of study effect sizes has two components: • variance due to sampling error (estimated in the effect size variances computed earlier) and • random effects variance component reflecting differences in effect sizes attributable to systematic differences between studies (Raudenbush, 1994). meta-analysis

32. Q Statistic – Test of Homogeneity • Q Statistic – Test of Homogeneity • Examines whether the variability among study effect sizes is greater than would be expected due to chance (i.e., to sampling error) alone. • The Q statistic represents a ratio of between-studies variance to within-study variance, analogous to the F ratio in ANOVA. • The Q statistic has (k – 1) degrees of freedom (where k is the number of studies included in the meta-analysis) to determine whether the null hypothesis (i.e., variability among observed effect sizes is attributable to sampling error alone) can be rejected at a desired probability level (usually a = .05) (Shadish & Haddock, 1994). meta-analysis

33. - Q Statistic – Test of Homogeneity • For this study data, Q(k-1=14) = 16.82, p > .05, so the null hypothesis could not be rejected. • That is, the variance among study effect sizes was not greater than expected based on sampling error, so that there is no warrant for suspecting that systematic differences among studies contributed to differences in their obtained effect sizes. meta-analysis

34. Additional analyses • Also will conduct moderator Analyses - (a) type of observation - behavioral performance, or observer judgment) and (b) hours/amount of intervwention provided. • Moderator analyses using both continuous (hours of treatment) and categorical (type of observation) variables. meta-analysis

35. Implications for ongoing international research • Lets discuss: • What are outcome research of stakeholders from different countries, different systems • How valued is research by different stakeholders • How can we develop a common language related to DM outcomes to promote research meta-analysis

36. Implications for ongoing international research • Defining DM – what does it mean to different countries when disability benefit systems and healthcare systems are so different • Is DM passe? How does it Line up for employers who promote wellnes • With these differences, how do we align collaborative international research to talk the same language…what are common metrics and metrics of WHAT meta-analysis

37. Other questions? meta-analysis