Clinical assessment part ii william sacks phd md
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

Clinical Assessment – Part II William Sacks, PhD, MD PowerPoint PPT Presentation


  • 83 Views
  • Uploaded on
  • Presentation posted in: General

COMPUTERIZED THERMAL IMAGING, INC . Breast Cancer System BCS 2100 P010035 Radiological Devices Advisory Panel December 10, 2002. Clinical Assessment – Part II William Sacks, PhD, MD. Safety of the BCS has two aspects: Adverse events Accuracy of BCS output .

Download Presentation

Clinical Assessment – Part II William Sacks, PhD, MD

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Clinical assessment part ii william sacks phd md

COMPUTERIZED THERMAL IMAGING, INC.

Breast Cancer System BCS 2100P010035

Radiological Devices Advisory Panel

December 10, 2002

Clinical Assessment – Part II

William Sacks, PhD, MD

Sacks - Clinical Assessment


Safety of the bcs has two aspects adverse events accuracy of bcs output

Safety of the BCS has two aspects:

Adverse events

Accuracy of

BCS output

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

There were only 4 minor adverse events, two of which were thought not to be device related, during a clinical trial involving 2407 subjects.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

From the point of view of the BCS output:

Safety is more closely

related to sensitivity

(cancers), or FNR, but

also FPR

Effectiveness is more

closely related to

specificity (benign masses)

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

POINTS TO BE COVERED

What BCS 2100 (BCS) is and

is not intended to do

How BCS does it

What the clinical trial

demonstrated

Labeling issues

Sacks - Clinical Assessment


Clinical submissions pma amendment 4 amendment 5 amendment 7

CLINICAL SUBMISSIONS

PMA

Amendment 4

Amendment 5

Amendment 7

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

After reviewing the PMA, the FDA sent a letter to CTI listing a number of deficiencies. CTI’s response was Amendment 4.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

In Amendment 4, for their conclusions concerning BCS effectiveness (p. 11), CTIretrospectively selected from the PMA data one of two analytical indices (Se/Sp, not ROC) and two of three lesion types (masses, and arch. dist., not microcalcifications).

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

In Amendment 4 the revised labeling further deleted architectural distortion, and referred to masses alone.

Sacks - Clinical Assessment


The fda sent another deficiency letter to cti the response was amendment 5

The FDA sent another deficiency letter to CTI. The response was Amendment 5.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Amendment 5 was offered as a test of BCS in additional subjects, because Amendment 4 had contained retrospective selections. CTI called the additional dataset PPMA (Post-PMA).

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Amendment 5 confined its analysis of the PPMA data to the newly chosen analytical index (Se/Sp) in the newly chosen subgroup (masses).

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

In addition to presenting data on a new set of subjects (PPMA), Amendment 5 also contained analysis of the combined datasets from Amendment 4 and the PPMA.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Because of the retrospective selections in Amendment 4, FDA asked CTI to justify combining that data with the PPMA. CTI’s response was Amendment 7.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

In Amendment 7 CTI applied a Bonferroni correction in an attempt to compensate for

retrospective selection of

data in Amendment 4 and

the smallness of the

additional (PPMA) sample

in Amendment 5.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Two overriding issues:

Adequacy of data

Interpretation of

data, i.e., do the

data demonstrate S&E

of BCS?

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Adequacy of data

Can data from Amendment 4 contribute to judgment of S&E, when it consists of retrospective selection of mammographic masses, because the Se was too low in microcalcifications and in the tested population as a whole?

Sacks - Clinical Assessment


Adequacy of data cont is a bonferroni correction applicable in this context

Adequacy of data (cont.)

Is a Bonferroni

correction applicable in

this context?

Sacks - Clinical Assessment


Adequacy of data cont are the data from the ppma alone in amendment 5 adequate to judge s e

Adequacy of data (cont.)

Are the data from

the PPMA alone (in

Amendment 5) adequate to

judge S&E?

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Interpretation of the data

It is noteworthy for the following discussion that no formal hypotheses were explicitly put forward for testing, either in the PMA or in the subsequent amendments.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

To qualify as a testable hypothesis there must be a quantitative criterion, whereby either

a point estimate may imply

rejection, or

a confidence interval may

entail exclusion.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

There were two implicit hypotheses:

that the ROC area for BCS and

mammography combined would exceed that

of mammography alone (with statistical

significance)

[Protocol, Section 6.0 Statistical Analysis]

that the point estimate for sensitivity

would be at least 99.3% (in at least

75% of simulations with the data)

[PMA, Section 5.8.7.2, p. 489].

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

The protocol otherwise contained only non-quantitative statements of what CTI hoped to achieve in the clinical trial. For example,

The objective of the study is to determine if the (BCS 2100), when used in conjunction with clinical examination and/or diagnostic mammography, increases the ability of physicians to differentiate benign from malignant, or suspicious, breast abnormalities. [Protocol, Section 3.1 Study Objective]

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

In the original PMA submission, the comparison of ROC areas failed to achieve statistical significance, except as an artifact of too few points in the mammography-alone curve, so it was not pursued in any of the amendments.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

In addition, the sensitivities failed to achieve a level of 99.3% with 75% confidence, in any of the datasets.

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Mammography

alone

PMA (n=187)

(Se 97.1%,

Sp 14.3%)

PPMA

(n=15)

(Se 93.3%, Sp 25.4%)

Combined

(n=105)

(Se 99.0%, Sp 19.2%)

Am. 4

(n=90)

(Se 100%,

Sp 18.0%)

100%

99.7%

98.8%

Reference Se 99.3%

96.7%

95.6%

94.1%

Chance Line

SENSITIVITY

SPECIFICITY

0%

72.1%

Sacks - Clinical Assessment


Potential safety and effectiveness in u s population

Potential safety and effectiveness in U.S. population:

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Percent of U.S. biopsies potentially obviated by the BCS if used on all eligible women:

1.3 million U.S. women biopsied each year,

of which ~45% (585,000) have

mammographic masses,

of which ~80% (468,000) are benign,

of which 15% to 20% (70,000 to 94,000)

would be BCS(-) and saved a biopsy.

Thus ~5% to 7% of 1.3 million U.S. biopsies

would be obviated,

and that’s if BCS were used on all 585,000 women

with mammographic masses on their way to biopsy.

In addition to saving biopsies on these benign

masses, approximately 1% to 6% of the

malignant masses, and 0.5% to 3% of all breast

cancers, might be delayed in diagnosis.

(

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

POINTS TO BE COVERED

What BCS 2100 (BCS) is and

is not intended to do

How BCS does it

What the clinical trial

demonstrated

Labeling issues

Sacks - Clinical Assessment


Labeling issues size of mass depth of mass

Labeling issues

Size of mass

Depth of mass

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Size of mass

Effect of small lesion size on device sensitivity difficult to evaluate, since only 2/105 cancers < 5mm.

Sacks - Clinical Assessment


Size of mass cont

Size of mass (cont.)

(Amendment 4, p.8, Table A4.4,

Amendment 5, p.23, Table A5.14)

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Depth of mass

With the chosen IOS threshold (20.59) there was no definite effect of lesion depth on BCS result, but...

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

Depth of mass (cont.)

...the effect of lesion depth was difficult to evaluate, since depth is not easily gauged on mammography.

Sacks - Clinical Assessment


Conclusions

Conclusions

Sacks - Clinical Assessment


Clinical assessment part ii william sacks phd md

In summary

Only 4 out of 2407 subjects had an adverse event,

all minor.

There were no explicit quantitative hypotheses.

There were two implicit quantitative hypotheses.

Neither hypothesis was fulfilled.

Most data were selected retrospectively.

Bonferroni correction is not applicable in this

context.

Using the trial results, if the BCS were in general

use in the U.S., it would obviate 5% to 7% of the

1.3 million biopsies per year.

Approximately 1% to 6% of these obviated biopsies

would turn out to be malignant, and their

diagnoses would thus be delayed.

Sacks - Clinical Assessment


  • Login