Risk and credibility assessments for computational modeling of medical devices
Download
1 / 30

Risk and Credibility Assessments for Computational Modeling of Medical Devices - PowerPoint PPT Presentation


  • 68 Views
  • Uploaded on

Risk and Credibility Assessments for Computational Modeling of Medical Devices. Tina Morrison, PhD [email protected] Advisor of Computational Modeling Office of Device Evaluation, FDA Vice Chair ASME V&V40 Subcommittee Member MDIC CM&S Steering Committee.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Risk and Credibility Assessments for Computational Modeling of Medical Devices' - demetrius-glover


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Risk and credibility assessments for computational modeling of medical devices

Risk and Credibility Assessments for Computational Modeling of Medical Devices

Tina Morrison, PhD [email protected]

Advisor of Computational Modeling

Office of Device Evaluation, FDA

Vice Chair

ASME V&V40 Subcommittee

Member

MDIC CM&S Steering Committee


Role of v v for computational models of medical devices

If computational models are to be increasingly relied upon in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Need to establish …

If the model is correct and credible

Demonstrated predictive capabilities to justify use beyond domain of validation

Predictive confidence is commensurate with model risk

Role of V&V for Computational Models of Medical Devices


Asme subcommittee on v v
ASME Subcommittee on V&V in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

  • Standards Subcommittee

    • Provide procedures for assessing and quantifying the accuracy and credibility of computational modeling and simulation


Verification validation in computational modeling of medical devices
Verification & Validation in in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.Computational Modeling of Medical Devices

  • V&V-40 Charter

    • Provide procedures to standardize verification and validation for computational modeling of medical devices

    • Charter approved in January 2011

  • Medical device focus

    • Regulated industry with limited ability to validate clinically

    • Want increased emphasis on modeling to support device safety and/or efficacy

    • Use of modeling is hindered by lack of V&V guidance and expectations within medical device community


Guide for verification and validation for computational models of medical devices
Guide for Verification and Validation in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.for Computational Models of Medical Devices

  • Regulated industry with limited ability to validate clinically

  • Want increased emphasis on modeling to support device safety and/or efficacy

  • Use of modeling is hindered by lack of V&V guidance and expectations within medical device community

  • Focus of the Guide

    • Instead of focusing on how to perform V&V (established elsewhere) …

    • We developed a common V&V framework to standardize definitions, processes, and documentation requirements between industry, researchers, software developers and regulators.


Overall v v flow
Overall V&V Flow in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Assess Model Risk

Establish Credibility Requirements

Establish Work plan for VV

Define

COU

Purpose

Is the plan achievable?

NO

If the plan is not achievable, you will need to redefine the scope, purpose and context of use of the CM&S, which will effect model risk, credibility requirements and the work plan.

YES

Execute pre-defined M&S and V&V plan

Is the

CM&S Credible for COU?

NO

YES

Document M&S and VV Plan and Findings


Overall v v flow1
Overall V&V Flow in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Assess Model Risk

Establish Credibility Requirements

Establish Work plan for VV

Define

COU

Purpose

Risk Assessment Matrix

Is the plan achievable?

NO

If the plan is not achievable, you will need to redefine the scope, purpose and context of use of the CM&S, which will effect model risk, credibility requirements and the work plan.

YES

Execute pre-defined M&S and V&V plan

Is the

CM&S Credible for COU?

NO

YES

Document M&S and VV Plan and Findings


Risk assessment matrix ram
Risk Assessment Matrix (RAM) in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

  • Establish Context of Use

  • Model Risk: combination of decision influence and consequence

  • Decision Influence: contribution of the model outcome to the decision being made

  • Consequence: impact if the model outcomes prove incorrect

  • Model risk assessment

    • Directs/guides V&V activities

    • Defines model credibility requirements

HIGH

MEDIUM

INFLUENCE

LOW

CONSEQUENCE


Overall v v flow2
Overall V&V Flow in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Assess Model Risk

Establish Credibility Requirements

Establish Work plan for VV

Define

COU

Purpose

Credibility Assessment Matrix

Is the plan achievable?

NO

If the plan is not achievable, you will need to redefine the scope, purpose and context of use of the CM&S, which will effect model risk, credibility requirements and the work plan.

YES

Execute pre-defined M&S and V&V plan

Is the

CM&S Credible for COU?

NO

YES

Document M&S and VV Plan and Findings


Credibility assessment matrix cam
Credibility Assessment Matrix (CAM) in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.


Credibility assessment matrix cam1
Credibility Assessment Matrix (CAM) in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Establish Target Credibility Requirements based on the Context of Use


Overall v v flow3
Overall V&V Flow in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Assess Model Risk

Establish Credibility Requirements

Establish Work plan for VV

Define

COU

Purpose

Is the plan achievable?

NO

If the plan is not achievable, you will need to redefine the scope, purpose and context of use of the CM&S, which will effect model risk, credibility requirements and the work plan.

YES

Execute pre-defined M&S and V&V plan

Is the

CM&S Credible for COU?

NO

YES

Document M&S and VV Plan and Findings


Credibility level determination
Credibility Level Determination in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.


Example

Example in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Jeff Bischoff

Mehul Dharia

Zimmer, Inc.


Force on tibial spine of a knee implant
Force on tibial spine of a knee implant in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Posterior Tibial Spine Force in Deep Flexion

May Create Posterior Liftoff

Anterior Tibial Spine Force

in Hyperextension

May Create Anterior Liftoff

Locking mechanism between the (metal) tibial tray and (polyethylene) articular surface is intended to prevent disassembly (poly lift-off) of the modular tibial component during activities of daily living


Anterior lift off test
Anterior Lift-off Test in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Context of use of a test for anterior lift-off

Verify that the force required for lift-off of the articular surface from the tibial tray for a new design is greater than expected physiological loading, and therefore demonstrate that the new (locking mechanism) design sufficiently mitigates that risk.


Contexts of use of a model for anterior lift off
Contexts of use of a model for anterior lift-off: in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

  • Determine the size of component within the new design family that has the smallest force required for anterior lift-off, to then be assessed in a physical test relative to a predicate.

    FEA followed by Physical Test, Comparison to Predicate

  • Verify that the force required for lift-off of the articular surface from the tibial tray for a new design is greater than that required for a clinically successful predicate, and therefore demonstrate that the new (locking mechanism and/or geometry) design sufficiently mitigates that risk.

    FEA only, Comparison to Predicate

  • Determine the size of a component within the new design family that has the smallest force required for anterior lift-off, to then be assessed in a physical test without reference to predicate device.

    FEA followed by Physical Test, No Predicate

  • Demonstrate through analysis alone that the worst case size can sustain physiological loading without liftoff, without reference to a predicate device.

    FEA only, No Predicate


Risk assessment matrix
Risk Assessment Matrix in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Model influence

LOW: Results from the computational model are a negligible factor in the decision associated with the question being answered.

MEDIUM: Results from the computational model are a moderate factor in the decision associated with the question being answered.

HIGH: Results from the computational model are a significant factor in the decision associated with the question being answered.

Patient consequence

LOW: A poor decision would not adversely affect patient safety or health, but might result in nuisance to the physician or has other negligible impacts.

MEDIUM: A poor decision would result in minor patient injury and potentially requiring physician intervention or has other moderate impacts.

HIGH: A poor decision would result in severe patient injury or death or has other significant impacts.


Risk assessment matrix1
Risk Assessment Matrix in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.


Risk assessment matrix2
Risk Assessment Matrix in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

COU4

COU1: Worst case

determination

COU2: Absolute evaluation

COU3

COU1,2


Cam elements of computational models
CAM – Elements of Computational Models in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Verification

  • Code (Column B) – 4

    • Used commercially available validated FEA software

  • Solution (Column C) – 4

    • Mesh convergence study was performed

      • Numerical effects are determined to be small on all important quantity of interests at conditions/ geometries directly relevant to the context of use

    • All inputs and outputs based on independently reputable source


Cam elements of computational models1
CAM – Elements of Computational Models in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Validation: Computational Model

  • System Configuration (Column D) – 2

    • Used mean/nominal geometry (no LMC/MMC)

    • Major and minor features captured

    • Two sizes considered

  • Governing Equations (Column E) – 4

    • Used nonlinear material (constitutive) model for UHMWPE

      • Key physics (press-fit, resistance against force) was captured

      • Material model did not need re-calibration/tuning

  • System Properties (Column F) – 1

    • Nominal physical properties that are representative of the comparator from literature

    • Sensitivity analysis on material properties was not performed

  • Boundary Conditions (Column G) – 3

    • Load applied through assumed contact patch on spine, rather than directly modeling the femoral component - Representative but simplified BCs with non-quantified effect on QOI


Cam how well is the comparator understood
CAM – How Well Is The Comparator Understood? in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Validation: Evidence-Based Comparator

  • System Configuration (Column H) – 3

    • Prescribed location

    • Geometries matched to machine tolerance (production parts)

    • Signal to noise ratio is high

  • System Properties (Column I) – 3

    • Off-the-shelf parts were tested

    • Environmental effects on the material are known (testing speed was modified, environment was kept the same for both groups: in air).

  • Boundary Conditions (Column J) – 3

    • No sensitivity analysis was performed.

    • Known (recorded) loading (perturbations) was applied and boundary condition variability (e.g. posterior slope) is known.

  • Sample Size (Column K) – 3

    • Statistically relevant sample size (n = 5)

    • Component size, a key parameter for lift-off, variation was considered.


Cam how appropriate is cm to comparator
CAM – How Appropriate is CM to Comparator? in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Validation: Model-to-Comparator

  • Discrepancy (Column L) – 4

    • Equivalent input parameters, equivalent quantity of interest


Cam how appropriate is cm to comparator1
CAM – How Appropriate is CM to Comparator? in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Validation: Model-to-Comparator

  • Discrepancy (Column L) – 4

    • Equivalent input parameters, equivalent quantity of interest

CAM – How Rigorously Are Outputs Compared?

Validation: Qualitative or Quantitative

  • Comparison (Column M) – 3

    • Quantitative comparison, with single set of input parameters, without predictive accuracy or uncertainties available

    • No quantitative comparison with broad range of cases


Cam how v v activities relates to cou
CAM – How V&V activities relates to COU? in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Validation: V&V to COU

  • Applicability (Column N) – 3

    • Validation activities embody relevant characteristics of the CoU sufficient overlap between the validation domain and the CoU space)


What can we conclude
What can we conclude? in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

COU4

COU3

COU1,2


Overall v v flow4
Overall V&V Flow in the development and evaluation of medical devices, the consistent application of V&V must be applied to establish model credibility.

Assess Model Risk

Establish Credibility Requirements

Establish Work plan for VV

Define

COU

Purpose

Is the plan achievable?

NO

If the plan is not achievable, you will need to redefine the scope, purpose and context of use of the CM&S, which will effect model risk, credibility requirements and the work plan.

YES

Execute pre-defined M&S and V&V plan

Is the

CM&S Credible for COU?

NO

YES

Document M&S and VV Plan and Findings


Public Meeting - FDA/NIH/NSF Workshop on Computer Models and Validation for Medical Devices, June 11-12, 2013

http://www.fda.gov/MedicalDevices/NewsEvents/WorkshopsConferences/ucm346375.htm

Additional resources on RAM and CAM


For more information please contact
For More Information Please Contact: Validation for Medical Devices, June 11-12, 2013

  • Tina Morrison, PhD [email protected]

  • Advisor of Computational Modeling

  • Office of Device Evaluation, FDA

  • OR

  • Michael Liebschner, PhD [email protected]

  • Pre-ORS Symposium Chair

  • Baylor College of Medicine; Exponent Failure Analysis


ad