The ATOMS Project: Measuring Assistive Technology Outcomes

1. The ATOMS Project:Measuring Assistive Technology Outcomes Dave L. Edyburn, Roger O. Smith, Todd D. Schwanke & Kelly S. Fonner Center for Rehabilitation Sciences & Technology Closing the Gap Conference, Minneapolis, MN October 17, 2002 View: Universal Access Features

2. Copyright • This Microsoft PowerPoint file has been made available as an accessible, electronic handout for the participants of the presentation. • You must obtain permission from the ATOMS Project before copying or further distributing this presentation. Update: The ATOMS Project

3. ATOMS Projects Vitals • Based at the University of Wisconsin – Milwaukee • National consortium • 5 year AT outcomes project • NIDRR funded (US Dept of Education) • DRRP - Disability and Rehabilitation Research Projects Update: The ATOMS Project

4. ATOMS Project Consortium Helen Hayes Hospital Steve Mendelsohn  Update: The ATOMS Project

5. AT Outcomes Vision and Needs? • What do you think an outcomes system should look like in 10 years? • How would you want to use it? Audience Vision Audience Need Update: The ATOMS Project

6. The Atoms Project Response • Difficult questions for response, aren’t they? • The ATOMS Project hopes to help clarify the field’s 10 year vision. Update: The ATOMS Project

7. Overall ATOMS Project goal • Explore, Pilot, and Test AT Outcome Measurement Ideas to Recommend an AT outcomes system (Go Where No One Has Gone Before) Update: The ATOMS Project

8. Our Overall Approach Reflected as the ATOMS Project Logo Update: The ATOMS Project

9. Challenges to an AT outcomes system • Diversity of perspectives of need for outcomes data • Diversity of populations • Diversity of domains for which we'd like outcomes data • Requirements for a data system to use reliable and valid measures • AT is only one of many interventions and it is an rarely used in isolation • Outcomes methodology has advanced, but so have new outcomes measurement ideas Update: The ATOMS Project

10. Context Baseline Environment • Person Function Task AT Outcome Model (1)- Context and Baseline Update: The ATOMS Project

11. Intervention Approaches Context Baseline Environment Interventions: Including the Use of Assistive Technology • Person Function Task AT Outcome Model (2)- Interventions Update: The ATOMS Project

12. Intervention Approaches Context Baseline Outcome Environment Interventions: Including the Use of Assistive Technology • Person Function Enhanced Function Task AT Outcome Model (3) - Outcome Update: The ATOMS Project

13. Intervention Approaches Context Baseline Outcome Environment Reduce the Impairment Compensate for the Impairment • Person Functional Performance Enhanced Functional Performance Task Use Assistive Technology Devices and Services Redesign the Activity Redesign the Environment Use Personal Assistance AT Outcome Model (4) - Interventions Detailed Update: The ATOMS Project

14. Intervention Approaches Context Baseline Outcome Environment Reduce the Impairment Compensate for the Impairment • Person Functional Performance Enhanced Functional Performance Task Use Assistive Technology Devices and Services Redesign the Activity Redesign the Environment Health Promotion Universal Design Use Personal Assistance Pre-intervention AT Outcome Model (5) - with Pre-interventions Update: The ATOMS Project

15. Intervention Approaches Context Baseline $$ Outcome Environment Reduce the Impairment Compensate for the Impairment • Person Functional Performance Enhanced Functional Performance Task Use Assistive Technology Devices and Services Redesign the Activity $$ $$ Redesign the Environment Health Promotion $$ Universal Design Use Personal Assistance Pre-intervention AT Outcome Model (6) - with Costs Update: The ATOMS Project

16. Face the Challenge or Run? • Does that mean that a coordinated outcome system is too difficult and that we are stuck with a fragmented, haphazard, homemade, anarchistic system? • Is a more cohesive and comprehensive AT outcomes approach possible? Update: The ATOMS Project

17. Strategies • Computerized data collection • Centralized database • Customized or multiple interfaces for various user perspectives • Decision analysis data elicitation strategies Update: The ATOMS Project

18. General ATOMS Project Timeline Year 1-2: Needs assessment & field feedback Year 2-3: Instrument/methodology exploration & development Year 3-5: Pilot ideas Year 5: Propose AT outcomes methods & system Update: The ATOMS Project

19. ATOMS Specific Activities 1. Needs Assessment • Field Scans • Stakeholder Focus Groups • Existing Database Analyses 2. Exploratory R&D Projects (Instrument Development) 3. Abandonment Analysis Update: The ATOMS Project

20. Field Scans Types • Current Instruments (1,2) • Newer methodologies & instrumentation (4,5,6) • Feedback from field (focus groups) • Literature reviews (3,7,8,10,12) • Legal/Policy (9) • Conference scientific reviews (11) Update: The ATOMS Project

21. Field Scan 1 – Instrument Update and Review Formal and informal tools Commercial and program specific tools Specific and general tool • Instrument nomination form • Identification of gaps and overlaps • Searchable directory • Identification of type of measurement domains addressed Update: The ATOMS Project

22. Field Scan 2 – Coverage of AT in Current/Emerging Health & Rehabilitation Outcome Measures ~100 instruments selected for review (2 scored two-ways) Preliminary review reveals: • N = 41 fail to acknowledge the use of assistive technology in their outcome scoring • N = 47 lower the outcome score if AT is used (nude independence) (most examine limited types of AT) • N = 24 allow for AT to elevate the outcome score, but many do not differentiate among type of aids and assistance • Of the 102, 4 acknowledge that AT contributes to outcomes and isolates the outcome. Update: The ATOMS Project

23. Field Scan 3 – Inventory of Measures Used in AT Research and Design Activity Do AT developers use appropriate outcomes instruments and methodologies for their projects? How severe is the problem? Or how is this perceived by product developers? • Request methodology of grant proposals from PI’s – 2001 funding (NIH n=34, NIDRR n=27) • Survey of product developers • Technology Exhibitors, RESNA&AOTA National Conference 2002 n=78 • Random sample (n=500/1100) from ABLEDATA Directory of Manufacturers Update: The ATOMS Project

24. Field Scan 4 – Ascertain Next Generation Data Collection Technology • Factors considered for review of Technology & Methodology • Content/outcomes information • Scaling potentials • Data collection processes/protocol • Equipment (hardware/software) characteristics e.g. interface, portability, cost, durability • Data handling protocols • Data Reporting ~ 50 hardware & software technologies being reviewed Update: The ATOMS Project

25. Field Scan 5 – Evaluate Cost Comparison Methods • Helen Hayes Hospital: Frances Harris, Ph.D. • Initiating methodology review of cost comparison methods and how various approaches match the needs for AT Outcomes measurement procedures that include cost variables. • Methods of measuring costs • Methods of comparing costs • Cost-comparison literature review relevant to assistive technology Update: The ATOMS Project

26. Field Scan 6 – Assess Application of Decision Analytical Approaches • Multi-attribute Utility (MAU) Theory Application analysis (selected articles) • 47 Engineering articles using MAU methods • 60 Health-related articles using MAU methods • 32 General articles on MAU methodology • Bayesian Approach analysis (initial search) • 3797 articles, MEDLINE (1966-present) • 140 ERIC (1967-present) • 1573 Engineering Village (1970-present) Update: The ATOMS Project

27. Field Scan 7 – Review Taxonomies of Outcome What intersection of domains across taxonomies provide common language for a more universal AT outcomes discussions? • e.g. • Nagi Model (1965) • ICIDH (1980, 1993) • Rehabilitation Indicators (1983) • NCMRR Research plan (1993) • Quality of Life Taxonomy (Spilker & Revicki, 1996) • Characterization of Rehabilitation Services (Duncan, Hoenig , Samsa , & Hamilton 1997) • Institutes of Medicine Model (1997) • ICIDH-2 draft (1997) • ICF (2002) Update: The ATOMS Project

28. Field Scan 8 – Identify Strategies for Isolating ATOutcomes Compare methodologies that can isolate & quantify the outcomes of assistive technology. ~410 articles e.g. • Multi-variate regression analyses • Structural equation modeling • Direct consumer qualitative input (perceived benefit / satisfaction of a device) • Randomized controlled trials • Sequential Clinical Trials • Time-series concurrent differential (TSCD) • Qualitative Update: The ATOMS Project

29. Field Scan 9 – Describe Legal Implications of AT Outcomes Instrumentation • What are legal & ethical issues related to AT outcomes systems? -- Steven Mendelsohn • Legal, responsible, & ethical data collection procedures • Potential legal ramifications of AT outcomes data (positive and negative) • Implications of AT outcomes for policy-making Update: The ATOMS Project

30. Field Scan 10– Review of Special Education Technology Literature • Review of 31 special education technology journals • Reviews published in Journal of Special Education Technology (1999, 2000, 2001) • >2700 articles • Iteration with outcome filter to identify relevant articles. • Summarize and report findings. Update: The ATOMS Project

31. Field Scan 11– Technology Conference Program Review • Last two years of CTG, CSUN, RESNA, and TAM conferences reviewed as fugitive literature. • Identify relevant papers on AT Outcomes. • Summarize and report findings. Update: The ATOMS Project

32. Field Scan 12– Chronology of Assistive Technology Outcomes Measurement • Literature review reveals that At outcomes have been measured differently over the decades • This is consistent with what types of outcomes studies have been performed and the mandate for accountability • This field scan will develop a chronology map and accompanying discussion Update: The ATOMS Project

33. Quiz on the field scans: • Just Kidding…. • Lots of data from field scans. What do we hope to learn? • Technical reports • Compendium document • Consensus meeting Update: The ATOMS Project

34. Stakeholder Focus Groups • Direct Feedback from field • Consumer/User groups (4) • Service Directors (for records & documentation assessment), • Payors of AT devices & services, • Researchers, Developers, & Manufacturers, and • Parents & Caregivers. Update: The ATOMS Project

35. Service Director Focus Group – April 2002 • Modified NGT developed current list of identified AT Outcomes data domains (“If we had a magic wand”) 1. Change in performance/function (body, structure, activity) 2. Change in participation 3. Usage and why or why not 4. Consumer satisfaction (process, devices) 5. Goal Achievement 6. QOL 7. Cost 8. Demographics 9. AT interventions (services + devices) 10. Environment context • List is similar to DeRuyter (1998) Update: The ATOMS Project

36. Databases – National • National Health Information Survey-Disability (NHIS-D) • Assistive and information technology survey (NIDRR/RESNA/University of Michigan) Update: The ATOMS Project

37. Databases – Clinical • UW-Stout • University at Buffalo (SUNY) • HHH • PROVAIL • What data are being collected? • Are available data sufficient for analysis? • Is there an intersection of outcome data fields among existing service file systems? • Can clinical programs adapt to collecting more data? Update: The ATOMS Project

38. Clinical Database Update • Data fields • Commonalities (few) and differences (numerous) • Setting & funding specific (education, vocation, hospital, university) • Stakeholders (variable) • Electronic records (variable & minimal) • Potentials • VR data • Willingness of clinical programs to modify data collection Update: The ATOMS Project

39. Exploratory R&D Projects (Instrument Development) a) AT DeviceInventory b) ATServices Inventory c) Cost Identification Feasibility Study d) Assistive Technology Approach Isolation Measure (Subjective) e) Web-based visualization f) Environmental access assessment (WebAUDIT, MED-AUDIT) g) Computer branching questioning (TTSS) h) AT supplements to existing instruments (SFA-AT) Update: The ATOMS Project

40. Public School AT Outcome Pilots & Collaborative Activity a) Ohio b) Colorado Ohio Colorado Colorado Update: The ATOMS Project

41. NIDRR DRRP Coordination • CATOR (Consortium of Assistive Technology Outcomes Research) http://www.atoutcomes.org • ATOMS Project http://www.atoms.uwm.edu Update: The ATOMS Project

42. Dissemination: What to Expect • ATOMS Project Website Products • Special Issues of Journals • Conference Presentations • 2003-2004 Symposium Update: The ATOMS Project

43. Dissemination: ATOMS Project Products in the Works • AT Outcomes Primer • Study Group Outline • Test Your Knowledge of AT Outcomes • FAQ’s • Course Guide (syllabi and more) • Drafts of Instruments • Technical Reports • Compilation of Needs & Current Outcome Directions • Implications for Next Generation AT Outcomes System Update: The ATOMS Project

44. Needs Assessment Participation Opportunities • Conversation • Product developer survey • Instrument collection Update: The ATOMS Project

45. Contacting the ATOMS Project Completing interest survey Web:www.atoms.uwm.edu Email:atoms@uwm.edu Voice: (414) 229-6568 TTY: (414) 229-5628 Update: The ATOMS Project

46. ? ? ? ? ? ? ? ? Questions, Answers and Discussion Update: The ATOMS Project

47. Thank-you…. . . . for your attention! Slides about the universal access features of this PowerPoint presentation follow. Update: The ATOMS Project

48. Universal Access Features • The “speaker notes” associated with each slide in PowerPoint are used as a Universal Access feature. • They contain long text descriptions of the graphics because it was not feasible to do this with PowerPoint’s “alternative text” function. The descriptions can be used by new presenters and presentation attendees, in addition to being used for accessibility by people with vision or cognitive impairments. Update: The ATOMS Project

49. Viewing the Speaker Notes(does not work within PowerPoint Viewer 97 or 98) • In the “Slide Show” view within PowerPoint: • Windows: right click on the slide or use the context key to bring up the context menu and then select “speaker notes” • Mac: [Ctrl] + click on the slide to bring up the context menu and then select “speakers notes” • The notes can also be seen as a frame or pane in the “Normal” view or directly by using the “Notes Page” view. • When in “Normal” view, F6 is used to switch between the slide, notes and outline panes respectively. Go back to the opening presentation slide Update: The ATOMS Project