1 / 18

HCMDSS Workshop Certification & Requirements Working Group Report (WG 6)

HCMDSS Workshop Certification & Requirements Working Group Report (WG 6). June 2-3, 2005 Philadelphia, PA. John Hatcliff, Kansas State University (hatcliff@cis.ksu.edu). Peter Coronado -- Varian Medical Systems Steve Dimmer – Calypso Medical Technologies

calder
Download Presentation

HCMDSS Workshop Certification & Requirements Working Group Report (WG 6)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. HCMDSS Workshop Certification & RequirementsWorking Group Report (WG 6) June 2-3, 2005 Philadelphia, PA John Hatcliff, Kansas State University (hatcliff@cis.ksu.edu)

  2. Peter Coronado -- Varian Medical Systems Steve Dimmer – Calypso Medical Technologies Jacob Flanz – Mass General Hospital Elsa L. Gunter -- University of Illinois at Urbana – Champaign John Hatcliff -- Kansas State University Mark P. Jones -- Oregon Health & Science University (OGI) Paul L. Jones -- FDA/CDRH/OSEL Brad Martin -- National Security Agency Rosemary C. Polomano -- University of Pennsylvania School of Nursing Stacy Prowell -- University of Tennessee John Rushby -- SRI Rick Schrenker -- Massachusetts General Hospital Oleg Sokolsky -- University of Pennsylvania Participants

  3. State of the Art-- What We Do Well • Develop and approve stand-alone medical devices based on mature technology with moderate complexity • well-understood domain with a lot of experience • we have a lot preceding examples with a lot of experience • we know how to gather requirements, code, test, do hazard analysis, etc. in a reasonably good way • Advanced development techniques such as model-driven development and software product-line architectures are working well in specific device domains (e.g., Guidant pace makers) • Established mechanisms for reporting, tracking, and publicizing device related “incidents” (analogous to reporting air plane crashes) • Light-weight formal methods (e.g., ESC Java) have been effective in security evaluations • in non-regulated or certified settings, tremendous progress has been made (Intel, Microsoft, etc.)

  4. State of the Art—Challenges Within Bounds of Existing Technology • We know how to… • test, write requirements, prototype, etc. • but how we can do this more accurately, in a more automated way, and achieve greater cross-leveraging of artifacts • Large-scale, complex devices still pose challenges • e.g., proton therapy facility • 100,000’s of validation procedures, test cases • …validation burden (time/costs) slows time to market

  5. State of the Art—Challenges Within Bounds of Existing Technology • System integration for devices based on existing technology • We are doing pretty well with integrating products developed by a single manufacturer, e.g., Varian (Linac, connected to imaging device, treatment machine) • But there are clinical cases where people are incorporating devices from other manufacturers (currently tech people must be trained to recognize potential incompatibilities) • even something as simple as incorporating tubing into pump (which were both approved separately) malfunctioned in clinical setting

  6. State of the Art –The “Heroic” Clinical Engineer • In general lack of a systems view or systems model that would enable one to reason about clinical environments, processes, and ad hoc integration of devices • “Clinical Engineers” are faced with the task of deploying collections devices in ad-hoc ways in constantly changing devices • some devices interact directly • some device interact through patients • some devices with even looser coupling • research on formalizing clinic environment, medical process that accommodates variability across different environments • Devices being delivered with little or no, or inappropriate documentation (especially integration documentation) • Incorporate inter-patient variability of (e.g., very obese patients) in requirements for monitoring devices, etc.

  7. Research Challenge --Requirements Research • More automated and rigorous manner for eliciting, capturing, and leveraging requirements • Support semantics querying or simulation of use cases directly from requirements • More effective automated derivation of test cases from requirements • Automated generation of visualizations and natural language description from requirements

  8. Modeling/Formalization of Clinical Environments and Processes • Trend: Proliferation of sophisticated devices in complex environments where exceptions are the norm • Clinical engineers currently act as informal “system integrators” of devices • Must develop a more rigorous approach to capturing, visualizing, reasoning about clinical environment & processes • How to incorporate expected variability within clinical environments? • Can we mine these rigorous descriptions (e.g., of processes, workflows, etc) to guide device requirements development or detect problematic device interaction within a clinical environment? • Can we use these models to clarify goals and focus the tasks to achieve more rigorous clinical validation?

  9. Re-orienting Current Procedures for Component-wise Certification • Components as commodity • framework that creates a business environment that produces a variety of re-usable components are marketed • Certification effort should focus on validity of composing components • certification artifacts should be reused (integrator should not have the burden of re-certifying component internals) • want to be able to synthesize certification of assembly from certification of components • Recognize the differences in integration scenarios • Commodity components purchased vendor to produce a plug-n-play system with a suite of components • Components purchased by hospitals and assembled by clinical engineers (should their be any oversight of such engineering?) • Requirements that capture all important aspects of interfaces • functionality • quality-of-service • safety/faults • Elevating methods for, e.g., protocol validation to a system of systems level • Inspiration from but looking beyond RTCA Integrated Modular Avionics

  10. Certifying In the Presence of Change/Variation/Evolution • Frequent software patches • Move from “software product lines” to “certification product lines” • FDA recognition of tools the perform impact analysis and traceability to reduce the effort in retesting/re-certification • “Make” for certification • More sophisticated tools for managing certification artifacts and for automatically detecting what things that need to be rework in the presence of change

  11. Problems with Existing Technology • Reporting, tracking, and publicizing, and interpreting device “malfunctions” (distinct from “incidents”) across manufacturers is problematic • need mechanisms for reducing the burden of re-certification • impact/change analysis tools, having FDA recognize artifacts produced by such tools • some devices incorporate “flight recorder” technology (e.g., Varian Linac) • open up certification resources for looking at bug-fixes, evolution of software

  12. Dealing Effectively With Security Certification Issues • Trend: desire to integrate a variety of device types across a variety of network infrastructures • Network types: • national, regional, local, home, body-area (wired, wireless), • Trend: Blending of medical devices and medical informatics systems • Devices that stream data (pace makers) into medical records that may be mined for broader clinical studies • Trend: Competing concerns • security, availability, … • competing quality of service when devices deployed on the same framework (leading to the possibility of denial of life-giving services) • Need to investigate and vet security standards such as Common Criteria (most stringent levels require formal methods) and MILS architecture

  13. Certification in Context ofData Communication • Ontology of medical information • Common language framework for exchanging a variety of forms of data • how does this relate to DICOM, HL7, etc.? • How do we certify that systems maintain the integrity of the data? • What’s keeping IEEE 1073 from being implemented?

  14. Other Issues • Unique identifiers • Physical Integration Standards • Use of COTS software

  15. Research Infrastructure • Academics will likely not be able to effectively address certification challenges with V & V technologies without a detailed understanding of certification process • Certification training for academics • Workshop on certification/approval process working with example artifacts • Mock “red team” certification for products/artifacts developed by research teams • …likewise, the need for effective communication of available V & V technologies to industry

  16. Research Infrastructure • “Open Experimental Platform” for High Confidence Medical Devices • Funding agency pays for… • …vendors to develop representative examples of certified systems • …vendors to provide description of development process • …vendors to provide “challenge problems” to researchers • …vendors to provide “face time” to meet with researchers and evaluate developed tools against base-line development processes • One example OEP proposed by Peter Coronado (Varian) et. al.

  17. Research Goals (5 Years) • Apply existing process modeling languages to model clinical environments and processes • Suite of sophisticated requirements tools… • requirements capture, simulation, querying • Paths to incorporating more tools into the certification effort (adding value) • Certification of development tools (analysis, traceability tools) for the purpose of reducing (re)-certification burden • Instrument conventional formal methods tools (static analysis, model-checking) to produce a variety of artifacts (test cases, natural language description of traceability steps) • Demonstration of pervasive use of model-based development techniques with automated reasoning for • component conformance to interfaces • component capability based on checking interface capability • end to end reasoning of system behavior • …emphasize management, integration, and automatic generation of certification artifacts – models form a substrate within which all certification-relevant artifacts are attached

  18. Research Goals (10 Years) • Re-orienting the certification process toward a component-based certification • Certified components as commodities • Pervasive use of secure, QoS-aware, fault-tolerant, certified middleware • Integrated end-to-end model-based development frameworks dealing with composition, evolution, change • Effective demonstration (metrics, etc.) necessary to change industry / regulatory practice • Wide body of interoperability standards • More sophisticated compositional analysis tools • Tools for compositional hazard analysis

More Related