Clinical assessment part ii william sacks phd md
Sponsored Links
This presentation is the property of its rightful owner.
1 / 36

Clinical Assessment – Part II William Sacks, PhD, MD PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

COMPUTERIZED THERMAL IMAGING, INC . Breast Cancer System BCS 2100 P010035 Radiological Devices Advisory Panel December 10, 2002. Clinical Assessment – Part II William Sacks, PhD, MD. Safety of the BCS has two aspects: Adverse events Accuracy of BCS output .

Download Presentation

Clinical Assessment – Part II William Sacks, PhD, MD

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


COMPUTERIZED THERMAL IMAGING, INC.

Breast Cancer System BCS 2100P010035

Radiological Devices Advisory Panel

December 10, 2002

Clinical Assessment – Part II

William Sacks, PhD, MD

Sacks - Clinical Assessment


Safety of the BCS has two aspects:

Adverse events

Accuracy of

BCS output

Sacks - Clinical Assessment


There were only 4 minor adverse events, two of which were thought not to be device related, during a clinical trial involving 2407 subjects.

Sacks - Clinical Assessment


From the point of view of the BCS output:

Safety is more closely

related to sensitivity

(cancers), or FNR, but

also FPR

Effectiveness is more

closely related to

specificity (benign masses)

Sacks - Clinical Assessment


POINTS TO BE COVERED

What BCS 2100 (BCS) is and

is not intended to do

How BCS does it

What the clinical trial

demonstrated

Labeling issues

Sacks - Clinical Assessment


CLINICAL SUBMISSIONS

PMA

Amendment 4

Amendment 5

Amendment 7

Sacks - Clinical Assessment


After reviewing the PMA, the FDA sent a letter to CTI listing a number of deficiencies. CTI’s response was Amendment 4.

Sacks - Clinical Assessment


In Amendment 4, for their conclusions concerning BCS effectiveness (p. 11), CTIretrospectively selected from the PMA data one of two analytical indices (Se/Sp, not ROC) and two of three lesion types (masses, and arch. dist., not microcalcifications).

Sacks - Clinical Assessment


In Amendment 4 the revised labeling further deleted architectural distortion, and referred to masses alone.

Sacks - Clinical Assessment


The FDA sent another deficiency letter to CTI. The response was Amendment 5.

Sacks - Clinical Assessment


Amendment 5 was offered as a test of BCS in additional subjects, because Amendment 4 had contained retrospective selections. CTI called the additional dataset PPMA (Post-PMA).

Sacks - Clinical Assessment


Amendment 5 confined its analysis of the PPMA data to the newly chosen analytical index (Se/Sp) in the newly chosen subgroup (masses).

Sacks - Clinical Assessment


In addition to presenting data on a new set of subjects (PPMA), Amendment 5 also contained analysis of the combined datasets from Amendment 4 and the PPMA.

Sacks - Clinical Assessment


Because of the retrospective selections in Amendment 4, FDA asked CTI to justify combining that data with the PPMA. CTI’s response was Amendment 7.

Sacks - Clinical Assessment


In Amendment 7 CTI applied a Bonferroni correction in an attempt to compensate for

retrospective selection of

data in Amendment 4 and

the smallness of the

additional (PPMA) sample

in Amendment 5.

Sacks - Clinical Assessment


Two overriding issues:

Adequacy of data

Interpretation of

data, i.e., do the

data demonstrate S&E

of BCS?

Sacks - Clinical Assessment


Adequacy of data

Can data from Amendment 4 contribute to judgment of S&E, when it consists of retrospective selection of mammographic masses, because the Se was too low in microcalcifications and in the tested population as a whole?

Sacks - Clinical Assessment


Adequacy of data (cont.)

Is a Bonferroni

correction applicable in

this context?

Sacks - Clinical Assessment


Adequacy of data (cont.)

Are the data from

the PPMA alone (in

Amendment 5) adequate to

judge S&E?

Sacks - Clinical Assessment


Interpretation of the data

It is noteworthy for the following discussion that no formal hypotheses were explicitly put forward for testing, either in the PMA or in the subsequent amendments.

Sacks - Clinical Assessment


To qualify as a testable hypothesis there must be a quantitative criterion, whereby either

a point estimate may imply

rejection, or

a confidence interval may

entail exclusion.

Sacks - Clinical Assessment


There were two implicit hypotheses:

that the ROC area for BCS and

mammography combined would exceed that

of mammography alone (with statistical

significance)

[Protocol, Section 6.0 Statistical Analysis]

that the point estimate for sensitivity

would be at least 99.3% (in at least

75% of simulations with the data)

[PMA, Section 5.8.7.2, p. 489].

Sacks - Clinical Assessment


The protocol otherwise contained only non-quantitative statements of what CTI hoped to achieve in the clinical trial. For example,

The objective of the study is to determine if the (BCS 2100), when used in conjunction with clinical examination and/or diagnostic mammography, increases the ability of physicians to differentiate benign from malignant, or suspicious, breast abnormalities. [Protocol, Section 3.1 Study Objective]

Sacks - Clinical Assessment


In the original PMA submission, the comparison of ROC areas failed to achieve statistical significance, except as an artifact of too few points in the mammography-alone curve, so it was not pursued in any of the amendments.

Sacks - Clinical Assessment


In addition, the sensitivities failed to achieve a level of 99.3% with 75% confidence, in any of the datasets.

Sacks - Clinical Assessment


Mammography

alone

PMA (n=187)

(Se 97.1%,

Sp 14.3%)

PPMA

(n=15)

(Se 93.3%, Sp 25.4%)

Combined

(n=105)

(Se 99.0%, Sp 19.2%)

Am. 4

(n=90)

(Se 100%,

Sp 18.0%)

100%

99.7%

98.8%

Reference Se 99.3%

96.7%

95.6%

94.1%

Chance Line

SENSITIVITY

SPECIFICITY

0%

72.1%

Sacks - Clinical Assessment


Potential safety and effectiveness in U.S. population:

Sacks - Clinical Assessment


Percent of U.S. biopsies potentially obviated by the BCS if used on all eligible women:

1.3 million U.S. women biopsied each year,

of which ~45% (585,000) have

mammographic masses,

of which ~80% (468,000) are benign,

of which 15% to 20% (70,000 to 94,000)

would be BCS(-) and saved a biopsy.

Thus ~5% to 7% of 1.3 million U.S. biopsies

would be obviated,

and that’s if BCS were used on all 585,000 women

with mammographic masses on their way to biopsy.

In addition to saving biopsies on these benign

masses, approximately 1% to 6% of the

malignant masses, and 0.5% to 3% of all breast

cancers, might be delayed in diagnosis.

(

Sacks - Clinical Assessment


POINTS TO BE COVERED

What BCS 2100 (BCS) is and

is not intended to do

How BCS does it

What the clinical trial

demonstrated

Labeling issues

Sacks - Clinical Assessment


Labeling issues

Size of mass

Depth of mass

Sacks - Clinical Assessment


Size of mass

Effect of small lesion size on device sensitivity difficult to evaluate, since only 2/105 cancers < 5mm.

Sacks - Clinical Assessment


Size of mass (cont.)

(Amendment 4, p.8, Table A4.4,

Amendment 5, p.23, Table A5.14)

Sacks - Clinical Assessment


Depth of mass

With the chosen IOS threshold (20.59) there was no definite effect of lesion depth on BCS result, but...

Sacks - Clinical Assessment


Depth of mass (cont.)

...the effect of lesion depth was difficult to evaluate, since depth is not easily gauged on mammography.

Sacks - Clinical Assessment


Conclusions

Sacks - Clinical Assessment


In summary

Only 4 out of 2407 subjects had an adverse event,

all minor.

There were no explicit quantitative hypotheses.

There were two implicit quantitative hypotheses.

Neither hypothesis was fulfilled.

Most data were selected retrospectively.

Bonferroni correction is not applicable in this

context.

Using the trial results, if the BCS were in general

use in the U.S., it would obviate 5% to 7% of the

1.3 million biopsies per year.

Approximately 1% to 6% of these obviated biopsies

would turn out to be malignant, and their

diagnoses would thus be delayed.

Sacks - Clinical Assessment


  • Login