1 / 25

Computerized Thermal Imaging Breast Cancer System 2100 (CTI BCS2100)

Computerized Thermal Imaging Breast Cancer System 2100 (CTI BCS2100). Radiological Devices Advisory Panel December 10, 2002 Statistical Presentation Harry F. Bushar, PhD. Outline of Statistical Presentation. Clinical Study Protocol Objective Design Population Demographics

jayme-ruiz
Download Presentation

Computerized Thermal Imaging Breast Cancer System 2100 (CTI BCS2100)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computerized Thermal Imaging Breast Cancer System 2100 (CTI BCS2100) Radiological Devices Advisory Panel December 10, 2002 Statistical Presentation Harry F. Bushar, PhD Harry Bushar

  2. Outline of Statistical Presentation • Clinical Study Protocol • Objective • Design • Population • Demographics • Evaluation (Effectiveness + Safety) • PMA Clinical Study • Effectiveness + Safety • Amendment 4 Clinical Study • Effectiveness • Amendment 5 Clinical Study • Effectiveness • Amendment 7 Adjustment • Effectiveness • Statistical Conclusions Harry Bushar

  3. Clinical Study Protocol • Study Objective • “The objective of the study is to determine if the CTI System, when used in conjunction with clinical examination and/or diagnostic mammography, increases the ability of physicians to differentiate benign from malignant, or suspicious, breast abnormalities.” Harry Bushar

  4. Clinical Study Protocol • Study Design • prospective • blinded-to-histology • multi-center • intended to compare • Level of Suspicion (LOS) (0-5) of malignancy of suspicious breast lesions for clinical examination or diagnostic mammography before BCS to • LOS + (BCS Index of Suspicion (IOS) (0.00 – 100.00) of malignancy of suspicious breast lesions) • biopsy as the “gold standard” for pathology. Harry Bushar

  5. Clinical Study Protocol • Study Population • Original study population = 600 patients with biopsy. • Actual study population = 2,407 patients with biopsy. Harry Bushar

  6. Clinical Study Protocol • Study Demographics • Gender • 2,364 Female • 15 Male • 28 Unknown • Ethnicity • 53% Caucasian • 30% African American • 13% Latino • 2% Asian • 1% Other • 1% Unknown • Age • 12% (< 40) • 55% (40 – 60) • 32% (> 60) • 1% Unknown Harry Bushar

  7. Clinical Study Protocol • Primary Effectiveness on Overall Population • Evaluation • Area Under the ROC* Curve (AUC) to compare results of diagnostic mammography (LOS) without BCS (IOS) and diagnostic mammography with BCS (LOS + IOS). • Sensitivity • Specificity • “The CTI System will be considered effective if its performance in conjunction with diagnostic mammography and/or clinical examination is clinically better than mammography and/or clinical examination alone.” *ROC = Receiver Operating Characteristic Harry Bushar

  8. Clinical Study Protocol • Secondary Effectiveness on Subpopulations • Mammographic Lesion Type • Calcifications • Masses • Distortions • Mammographic Lesion Size • < 0.5 cm • 0.5 – 1.0 cm • > 1.0 cm • Mammographic Lesion Depth • As available Harry Bushar

  9. Clinical Study Protocol • Safety Evaluation • occurrence of adverse events. Harry Bushar

  10. PMA Clinical Study Population • The sponsor acquired BCS images from 2,407 patients at 6 US clinical sites from 12/20/96 through 4/30/01. • The sponsor actually analyzed only those patients with both • mammography, not those with just clinical examination, and • biopsy within 60 days. Harry Bushar

  11. PMA Training Clinical Study • 700 patients, consisting of the first 220 patients + 480 patients randomly selected from among the next 1,912 patients, were used to set the following BCS IOS cut-off: ≥ 20.59 implies a recommendation for biopsy of a given lesion and < 20.59 implies a recommendation for short-interval follow-up of a given lesion. Harry Bushar

  12. PMA Testing Clinical Study • 1,432 patients, enrolled from 12/20/96 through 10/30/00, were initially available to test the effectiveness of the BCS both in the original PMA and in Amendment 4. • 769 patients, with 187 malignant lesions + 688 benign lesions, were actually included in the effectiveness evaluation both in the original PMA and in Amendment 4. • Note that each patient had 1 to 4 lesions. • Note also that the sponsor assumes that lesions within patient are independent. Harry Bushar

  13. PMA Clinical Study Results • Primary Effectiveness by ROC AUC • The sponsor found, after excluding calcifications alone, a statistically significantly greater ROC AUC for (IOS + LOS1) than for mammography LOS1 scores (1-4) alone at significance level = 0.05. • The sponsor found, after again excluding calcifications alone, but expanding mammography LOS1 scores beyond (1-4) to include 2 additional intermediate categories (3.50 & 3.75), no statistically significant difference between ROC AUC for (IOS + LOS2) & (LOS2 alone) at significance level = 0.05. Harry Bushar

  14. Harry Bushar

  15. Harry Bushar

  16. PMA Clinical Study Results • Safety • The following 4 adverse events occurred out of 2,407 subjects (12/20/96 through 4/30/01): • 2 “mild”, possibly-related, resolved adverse events were both associated with patient discomfort during positioning. • 1 “serious” and 1 “mild”, not-likely-related, resolved adverse events were hospitalization for treatment of a pre-existing metabolic disorder and dizziness when sitting up after thermal imaging, respectively. Harry Bushar

  17. Amendment 4 Clinical Results • Overall (187 malignant + 688 benign) • Sensitivity = 97.1% with 95% CI* = (94.1%, 98.8%) • Specificity = 14.3% with 95% CI = (12.1%, 16.6%) *CI = Confidence Interval Harry Bushar

  18. Amendment 4 Clinical Results • Calcifications* (105 malignant + 368 benign) • Sensitivity = 94.8% with 95% CI = (89.6%, 97.8%) • Specificity = 9.2% with 95% CI = ( 6.9%, 12.1%) • Masses (90 malignant + 322 benign) • Sensitivity = 100% with 95% CI = (96.7%, 100%) • Specificity = 18.0% with 95% CI = (14.6%, 21.9%) • Distortions (16 malignant + 15 benign) • Sensitivity = 100% with 95% CI = (82.9%, 100%) • Specificity = 16.7% with 95% CI = ( 3.9%, 40.2%) *The sponsor interpreted these effectiveness clinical results by lesion type to specifically exclude calcifications alone. Harry Bushar

  19. Amendment 4 Interpretation • The sponsor’s initial rejection of overall effectiveness, followed by the sponsor’s differential findings among the 3 lesion type sub-populations clearly indicates exploration: • which does require confirmation, • which must be based on new data. Harry Bushar

  20. PPMA* Amendment 5 Population • Study population = 275 additional patients. • Gender • 274 female • 1 male • Ethnicity • 65% Caucasian • 31% African American • 3% Latino • 1% Asian or Unknown • Age • 13% (< 40) • 60% (40 – 60) • 27% (> 60) *PPMA = Post-PMA Harry Bushar

  21. PPMA Amendment 5 Study • 275 additional patients, enrolled from 11/1/00 through 4/30/01 at 3 of the 6 original US clinical sites, were initially available for confirmation of the effectiveness of the BCS in Amendment 5. • 173 patients, with 43 malignant + 159 benign lesions, were actually included in the Amendment 5 effectiveness evaluation. • Note that each patient had 1 to 3 lesions. • Note also that the sponsor assumes that lesions within patient are independent. Harry Bushar

  22. PPMA Amendment 5 Results • Overall (43 malignant + 159 benign) • Sensitivity = 93.8% with 95% CI = (84.0%, 98.5%) • Specificity = 20.0% with 95% CI = (14.9%, 26.0%) Harry Bushar

  23. PPMA Amendment 5 Results • Calcifications (24 malignant + 87 benign) • Sensitivity = 93.1% with 95% CI = (77.9%, 99.0%) • Specificity = 17.4% with 95% CI = (11.1%, 25.5%) • Masses* (15 malignant + 63 benign) • Sensitivity = 93.3% with 95% CI = (72.1%, 99.7%) • Specificity = 25.4% with 95% CI = (16.6%, 36.0%) • Distortions (8 malignant + 4 benign) • Sensitivity = 75.0% with 95% CI = (40.0%, 95.4%) • Specificity = 25.0% with 95% CI = ( 1.3%, 75.1%) *The sponsor interpreted these effectiveness results by lesion type to specifically include only masses. Harry Bushar

  24. Amendment 7 Adjustment • The sponsor’s attempted Bonferroni adjustment, in response to FDA deficiency 1.a), by • widening sensitivity/specificity CI estimates, which are • based on a simple direct combination of exploratory + confirmatory clinical data (PMA/Amendment 4 + PPMA Amendment 5), to • test a theoretically possible hypothetical set of hypotheses for • 7 (lesion types) or • 63 (lesion types, sizes, & depths), • which hypotheses are not explicitly included in the protocol, • Is not statistically acceptable, because • the sponsor simply estimates sensitivity/specificity CI’s for various sub-populations, • without actually statistically testing any hypotheses. • Therefore, there are no multiple comparisons requiring adjustment. Harry Bushar

  25. Statistical Conclusions • Diagnostic mammography, not just clinical examination, is required for use of BCS. • The sponsor’s primary effectiveness demonstration, using ROC AUC, loses statistical significance, when mammography LOS (1-4) is expanded by just 2 additional intermediate categories (3.50 & 3.75), after excluding calcifications alone. • The sponsor’s initial rejection of overall sensitivity, followed by rejection of calcification-alone sensitivity, indicates exploration: • which requires confirmation, • which requires new data. • The sponsor’s attempted Bonferroni adjustment, by widening sensitivity/specificity CI estimates, is not statistically acceptable. Harry Bushar

More Related