Clinical assessment part ii william sacks phd md
Download
1 / 36

Clinical Assessment – Part II William Sacks, PhD, MD - PowerPoint PPT Presentation


  • 121 Views
  • Uploaded on

COMPUTERIZED THERMAL IMAGING, INC . Breast Cancer System BCS 2100 P010035 Radiological Devices Advisory Panel December 10, 2002. Clinical Assessment – Part II William Sacks, PhD, MD. Safety of the BCS has two aspects: Adverse events Accuracy of BCS output .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Clinical Assessment – Part II William Sacks, PhD, MD' - marlo


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Clinical assessment part ii william sacks phd md

COMPUTERIZED THERMAL IMAGING, INC.

Breast Cancer System BCS 2100P010035

Radiological Devices Advisory Panel

December 10, 2002

Clinical Assessment – Part II

William Sacks, PhD, MD

Sacks - Clinical Assessment


Safety of the bcs has two aspects adverse events accuracy of bcs output

Safety of the BCS has two aspects:

Adverse events

Accuracy of

BCS output

Sacks - Clinical Assessment


There were only 4 minor adverse events, two of which were thought not to be device related, during a clinical trial involving 2407 subjects.

Sacks - Clinical Assessment


From the point of view of the BCS output:

Safety is more closely

related to sensitivity

(cancers), or FNR, but

also FPR

Effectiveness is more

closely related to

specificity (benign masses)

Sacks - Clinical Assessment


POINTS TO BE COVERED

What BCS 2100 (BCS) is and

is not intended to do

How BCS does it

What the clinical trial

demonstrated

Labeling issues

Sacks - Clinical Assessment


Clinical submissions pma amendment 4 amendment 5 amendment 7

CLINICAL SUBMISSIONS

PMA

Amendment 4

Amendment 5

Amendment 7

Sacks - Clinical Assessment


After reviewing the PMA, the FDA sent a letter to CTI listing a number of deficiencies. CTI’s response was Amendment 4.

Sacks - Clinical Assessment


In Amendment 4, listing a number of deficiencies. CTI’s response was for their conclusions concerning BCS effectiveness (p. 11), CTIretrospectively selected from the PMA data one of two analytical indices (Se/Sp, not ROC) and two of three lesion types (masses, and arch. dist., not microcalcifications).

Sacks - Clinical Assessment


In Amendment 4 the revised labeling further deleted architectural distortion, and referred to masses alone.

Sacks - Clinical Assessment


The fda sent another deficiency letter to cti the response was amendment 5

The FDA sent another deficiency letter to CTI. The response was Amendment 5.

Sacks - Clinical Assessment


Amendment 5 was offered as a test of BCS in was additional subjects, because Amendment 4 had contained retrospective selections. CTI called the additional dataset PPMA (Post-PMA).

Sacks - Clinical Assessment


Amendment 5 confined its analysis of the PPMA data to the newly chosen analytical index (Se/Sp) in the newly chosen subgroup (masses).

Sacks - Clinical Assessment


In addition to presenting data on a new set of subjects (PPMA), Amendment 5 also contained analysis of the combined datasets from Amendment 4 and the PPMA.

Sacks - Clinical Assessment


Because of the retrospective selections in Amendment 4, FDA asked CTI to justify combining that data with the PPMA. CTI’s response was Amendment 7.

Sacks - Clinical Assessment


In Amendment 7 CTI applied a Bonferroni correction in an attempt to compensate for

retrospective selection of

data in Amendment 4 and

the smallness of the

additional (PPMA) sample

in Amendment 5.

Sacks - Clinical Assessment


Two overriding issues: attempt to compensate for

Adequacy of data

Interpretation of

data, i.e., do the

data demonstrate S&E

of BCS?

Sacks - Clinical Assessment


Adequacy attempt to compensate for of data

Can data from Amendment 4 contribute to judgment of S&E, when it consists of retrospective selection of mammographic masses, because the Se was too low in microcalcifications and in the tested population as a whole?

Sacks - Clinical Assessment


Adequacy of data cont is a bonferroni correction applicable in this context

Adequacy attempt to compensate for of data (cont.)

Is a Bonferroni

correction applicable in

this context?

Sacks - Clinical Assessment


Adequacy of data cont are the data from the ppma alone in amendment 5 adequate to judge s e

Adequacy attempt to compensate for of data (cont.)

Are the data from

the PPMA alone (in

Amendment 5) adequate to

judge S&E?

Sacks - Clinical Assessment


Interpretation attempt to compensate for of the data

It is noteworthy for the following discussion that no formal hypotheses were explicitly put forward for testing, either in the PMA or in the subsequent amendments.

Sacks - Clinical Assessment


To qualify as a testable hypothesis there must be a attempt to compensate forquantitative criterion, whereby either

a point estimate may imply

rejection, or

a confidence interval may

entail exclusion.

Sacks - Clinical Assessment


There were two attempt to compensate forimplicit hypotheses:

that the ROC area for BCS and

mammography combined would exceed that

of mammography alone (with statistical

significance)

[Protocol, Section 6.0 Statistical Analysis]

that the point estimate for sensitivity

would be at least 99.3% (in at least

75% of simulations with the data)

[PMA, Section 5.8.7.2, p. 489].

Sacks - Clinical Assessment


The protocol otherwise contained only non-quantitative statements of what CTI hoped to achieve in the clinical trial. For example,

The objective of the study is to determine if the (BCS 2100), when used in conjunction with clinical examination and/or diagnostic mammography, increases the ability of physicians to differentiate benign from malignant, or suspicious, breast abnormalities. [Protocol, Section 3.1 Study Objective]

Sacks - Clinical Assessment


In the original PMA submission, the comparison of statements of what CTI hoped to achieve in the clinical trial. For example,ROC areas failed to achieve statistical significance, except as an artifact of too few points in the mammography-alone curve, so it was not pursued in any of the amendments.

Sacks - Clinical Assessment


In addition, the statements of what CTI hoped to achieve in the clinical trial. For example,sensitivities failed to achieve a level of 99.3% with 75% confidence, in any of the datasets.

Sacks - Clinical Assessment


Mammography statements of what CTI hoped to achieve in the clinical trial. For example,

alone

PMA (n=187)

(Se 97.1%,

Sp 14.3%)

PPMA

(n=15)

(Se 93.3%, Sp 25.4%)

Combined

(n=105)

(Se 99.0%, Sp 19.2%)

Am. 4

(n=90)

(Se 100%,

Sp 18.0%)

100%

99.7%

98.8%

Reference Se 99.3%

96.7%

95.6%

94.1%

Chance Line

SENSITIVITY

SPECIFICITY

0%

72.1%

Sacks - Clinical Assessment


Potential safety and effectiveness in u s population

Potential safety and effectiveness in U.S. population: statements of what CTI hoped to achieve in the clinical trial. For example,

Sacks - Clinical Assessment


Percent of U.S. biopsies potentially obviated by the BCS statements of what CTI hoped to achieve in the clinical trial. For example,if used on all eligible women:

1.3 million U.S. women biopsied each year,

of which ~45% (585,000) have

mammographic masses,

of which ~80% (468,000) are benign,

of which 15% to 20% (70,000 to 94,000)

would be BCS(-) and saved a biopsy.

Thus ~5% to 7% of 1.3 million U.S. biopsies

would be obviated,

and that’s if BCS were used on all 585,000 women

with mammographic masses on their way to biopsy.

In addition to saving biopsies on these benign

masses, approximately 1% to 6% of the

malignant masses, and 0.5% to 3% of all breast

cancers, might be delayed in diagnosis.

(

Sacks - Clinical Assessment


POINTS TO BE COVERED statements of what CTI hoped to achieve in the clinical trial. For example,

What BCS 2100 (BCS) is and

is not intended to do

How BCS does it

What the clinical trial

demonstrated

Labeling issues

Sacks - Clinical Assessment


Labeling issues size of mass depth of mass

Labeling issues statements of what CTI hoped to achieve in the clinical trial. For example,

Size of mass

Depth of mass

Sacks - Clinical Assessment


Size of mass statements of what CTI hoped to achieve in the clinical trial. For example,

Effect of small lesion size on device sensitivity difficult to evaluate, since only 2/105 cancers < 5mm.

Sacks - Clinical Assessment


Size of mass cont

Size of mass statements of what CTI hoped to achieve in the clinical trial. For example, (cont.)

(Amendment 4, p.8, Table A4.4,

Amendment 5, p.23, Table A5.14)

Sacks - Clinical Assessment


Depth of mass statements of what CTI hoped to achieve in the clinical trial. For example,

With the chosen IOS threshold (20.59) there was no definite effect of lesion depth on BCS result, but...

Sacks - Clinical Assessment


Depth of mass statements of what CTI hoped to achieve in the clinical trial. For example,(cont.)

...the effect of lesion depth was difficult to evaluate, since depth is not easily gauged on mammography.

Sacks - Clinical Assessment


Conclusions

Conclusions statements of what CTI hoped to achieve in the clinical trial. For example,

Sacks - Clinical Assessment


In summary statements of what CTI hoped to achieve in the clinical trial. For example,

Only 4 out of 2407 subjects had an adverse event,

all minor.

There were no explicit quantitative hypotheses.

There were two implicit quantitative hypotheses.

Neither hypothesis was fulfilled.

Most data were selected retrospectively.

Bonferroni correction is not applicable in this

context.

Using the trial results, if the BCS were in general

use in the U.S., it would obviate 5% to 7% of the

1.3 million biopsies per year.

Approximately 1% to 6% of these obviated biopsies

would turn out to be malignant, and their

diagnoses would thus be delayed.

Sacks - Clinical Assessment


ad