1 / 39

Probe Validation

Probe Validation. Anne Wiktor September 21, 2006. Abnormalities identified by FISH. Familiarization. Pilot Study. Precision. Validation Process. Evaluation Study. Enumeration probe strategy. p53. D17Z1. Abbott Molecular. Normal metaphase. Familiarization. Validation Process.

bina
Download Presentation

Probe Validation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Probe Validation Anne Wiktor September 21, 2006

  2. Abnormalities identified by FISH

  3. Familiarization Pilot Study Precision Validation Process Evaluation Study

  4. Enumeration probe strategy p53 D17Z1 Abbott Molecular Normal metaphase

  5. Familiarization Validation Process

  6. Familiarization Experiment Objectives • Assess probe performance • Assess equipment • Calculate analytic sensitivity and specificity

  7. Familiarization Experiment 5 normal male PHA-stimulated peripheral bloods • For each sample, score: • target loci in 20 metaphases • 50 interphase nuclei • Record signal pattern of each cell • expected pattern 2R2G

  8. Probe evaluation Manufacturer: Vysis Strategy:Enumeration Lot #: 987654 Probe Performance: Large, bright, intact signals, no cross-hybridization Equipment: Standard FISH microscopes and laboratory equipment are suitable

  9. # scored − # false positive • # scored x 100 Familiarization Experiment • Analytical specificity: • metaphase (% cells with signals at expected target) • Analytical sensitivity: • metaphase & interphase (% cells with expected signal pattern)

  10. Familiarization Results Analytical sensitivity: 20/20 = 100% Analytical specificity: 20/20 = 100%

  11. Enumeration Scoring Criteria p53 Orange (R), 17cen Green (G), Overlap/Fusion Yellow (F) Normal 1R1G1F Normal Overlap Normal 2R2G 2R1G Loss of 17cen 1R2G Loss of p53 1R1G signals Loss of p53 and 17cen Do not score overlapping cells Do not score cells with too many or too few signals

  12. Familiarization Results Analytical sensitivity: 246/250 = 98.4%

  13. 2R1G 2R2G Familiarization Experiment signal patterns observed 1R1G 1R2G

  14. Familiarization Pilot Study Validation Process

  15. Pilot Study Objectives • Using previous experience w/ similar FISH strategies: • Establish scoring criteria • Determine number of cells to analyze • From normal samples, determine: • Maximum false-positive nuclei for each signal pattern • Analytical sensitivity • Calculate initial cutoff for normal specimens • Percent cells meeting scoring criteria • Confirm and/or redefine scoring criteria

  16. How do we….establish scoring criteria? • Normal patterns • 2R2G • Abnormal patterns • 1R2G

  17. Pilot Study 5 normal & 5 representative abnormal specimens test intended tissue type • Code and randomize • 2 technologists - each score 100 nuclei • Record all signal patterns

  18. Pilot Study Results

  19. Pilot Study Summary • Analytical sensitivity • (% cells with expected signal pattern): 94.9% • Initial normal cutoff: • Percent cells meeting scoring criteria: 97.8%

  20. Familiarization Pilot Study Validation Process Evaluation Study

  21. Prepare for clinical evaluation study • Write provisional standard operating procedure • Final review of safety of reagents & procedures • Select controls for clinical practice • Verify number of cells to score is appropriate for FISH strategy and clinical application • 100 – 3.0% • 200 – 1.5% • 500 – 0.6% • 6000 – 0.05% 95% CI

  22. Clinical Evaluation Objectives • Simulate clinical practice • Establish normal cutoffs • Define abnormal reference range • Identify new scoring patterns

  23. Clinical Evaluation • Code and randomize • 2 technologists - score cells using pilot study criteria • Record number of cells not meeting criteria • Record signal patterns of atypical cells 25 normal and a set of abnormal specimens; include variants and mosaics

  24. Study design • 25 “normal” controls • 15 abnormal samples • 14 - del(17) • 1 – del(17)/-17

  25. Analysis of results • Normal cutoff: upper boundary of 95% CI for false-positive cells in normal specimens • Abnormal reference range: lowest & highest % abnormal cells of patient specimens • % of patterns not meeting scoring criteria: identify potential new additional signal patterns • Experimental clinical sensitivity: % correctly identified as true positives • Experimental clinical specificity: % correctly identified as true negatives

  26. Clinical Evaluation Results

  27. Normal cutoff based on binomial expansion formula Normal cutoff is based on 1-sided 95% confidence interval for binomial (N,p) given observed number of abnormal cells Beta inversion function (Microsoft Excel)

  28. Beta Inverse Function

  29. Beta Inverse Function =BETAINV(0.95,12,200) =BETAINV(0.95,12,200) CI # false positive + 1 # cells scored

  30. Clinical Evaluation Summary • Normal cutoffs (based on 200 nuclei with 95% confidence interval) • 1R2G = 8.5% (17/200 cells) • 1R1G = 6.5% (13/200 cells) • 2R1G = 4.5% (9/200 cells) • 3R3G = 2.5% (5/200 cells) • All other patterns = 1.5% (3/200 cells)

  31. Cutoffs based on # cells scored

  32. CYTOGENETICS LABORATORY CYTOGENETICS LABORATORY Workload Recording 44 minutes Initial handling Specimen processing FISH process 3 minutes 8 minutes 13 minutes Recording/ reporting Analysis 12 minutes DigitalImaging/ Printing 4 minutes 4 minutes

  33. Familiarization Precision Pilot Study Validation Process Evaluation Study

  34. Precision Objectives • Test the reproducibility of the assay • Confirm that the same result can be obtained consistently and accurately

  35. Precision • Perform & score FISH assay on 10 consecutive days • Precision: mean, SD, & range 1 abnormal with known % of abnormal cells

  36. Precision Testing SD = 2.02 Mean = 40.0 Range = 36 - 44

  37. Familiarization Precision Pilot Study Validation Process Evaluation Study

  38. In clinical practice, validation continues with….

  39. Validation Reference • Wiktor AE, Van Dyke DL, Stupca PJ, Ketterling RP, Dewald G, et al: Preclinical validation of fluorescence in situ hyrbidization assays in clinical practice, Genetics in Medicine 8:16-23, 2006.

More Related