1 / 45

Validation of screening methods (2002/657/EC)

IPH. Validation of screening methods (2002/657/EC). N. Van Wouwe. AFSCA-FAVV. Definition (2002/657/EC). Screening method : used to detect the presence of a substance or class of substances at the level of interest. have the capability for a high sample throughput

vian
Download Presentation

Validation of screening methods (2002/657/EC)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IPH Validation of screening methods (2002/657/EC) N. Van Wouwe AFSCA-FAVV

  2. Definition (2002/657/EC) • Screening method : • used to detect the presence of a substance or class of substances at the level of interest. • have the capability for a high sample throughput => are used to sift large numbers of samples for potential non-compliant results. Exemple: ELISA, plate test, biosensor, receptor test,…

  3. Definition (2002/657/EC) • Minimum criteria to use an analytical method as screening method: • must be validated (traceability) • must have a false compliant rate of <5% (β-error) at the level of interest

  4. Performance characteristics for method validation (screening) Qualitative method: identifies a substance on basis of its chemical, biological or physical propriety (binary response: +/-, absence/presence) Quantitative method: determines the amount or mass fraction of a substance (response: numerical value of appropriate unit) + = determination is mandatory

  5. Validation of screening test • Definition of the scope of the method • Analyte of group of analytes • Range of concentration • List of matrices • Initial validation with the most often used matrice in national monitoring program • Detection capacity (CCβ) • Selectivity/Specificity • Applicability/ Ruggedness/Stability • Precision (only for semi-quantitative method) If possible: different sources of blank material, different technicians, different days on the same spiked sample

  6. Validation of screening test • Targeted test: for 1 compound • validation for this compound • Targeted test: for a family of compounds • validation for 1 representative molecule of the family (antibody) • Wide range test: for more than 50 different molecules • Validation for at least a list of representative compounds • Common pattern of activity on a specific bacteria? • Common way of action (acting target)? • Published reference data on validation available?

  7. Proposition of the CRL for antimicrobials (in milk)

  8. Performance characteristics • Detection capacity • Selectivity/Specificity • Applicability/ Ruggedness/Stability • Precision (only for semi-quantitative method)

  9. Detection capability (CCβ) • The smallest content of the substance that may be detected, identified and/or quantified in a sample with an error probability of β • In case of MRPL, CCβ= lowest concentration at which the method is able to detect truly contaminated sample with a statistical certainty of 1-β • In case of MRL, CCβ= concentration at which the method is able to detect the MRL concentrations with a statistical certainty of 1-β

  10. Detection capability (CCβ) • No permitted limit • Analyse 20 blank materials => CCα = 3x signal/noise Analyse 20 blank materials fortified at CCα => CCβ = CCα + 1.64 x SDRW • Calibration curve procedure (ISO 11843) • Analyse of blank material fortified at 0 MRLP, 0.5 MRLP, 1 MRLP, 1.5 MRLP and 2 MRLP • Plot analytical results (y-axis) vs concentration(x-axis) • CCα = y-intercept (blank) + 2.33 x SDRW • CCβ = CCα + 1.64 x SDRW

  11. Detection capability (CCβ) • No permitted limit • If no quantitative results • Analyse fortified blank samples at and above CCα (n ≥ 20 / concentration level) • CCβ = concentration level where only ≤5% false compliant results remain

  12. Detection capability (CCβ) CCa CCb Blank +1.64xSDRW +2.33xSDblank α=1% β=5% Signal orConcentration

  13. Detection capability (CCβ) • Permitted limit (MRL) • Analyse 20 blank materials fortified at MRL => CCα = MRL + 1.64 x SDRW Analyse 20 blank materials fortified at CCα => CCβ = CCα + 1.64 x SDRW • Calibration curve procedure (ISO 11843) • Analyse of blank materials fortified at 0.5 MRL, 1 MRL, 1.5 MRL and 2 MRL • Plot analytical results (y-axis) vs concentration(x-axis) • CCα = MRL + 1.64 x SDRW • CCβ = CCα + 1.64 x SDRW

  14. Detection capability (CCβ) CCa CCb MRL +1.64xSDMRL +1.64xSDRW α=5% β=5% Signal orConcentration

  15. Performance characteristics • Detection capacity • Selectivity/Specificity • Applicability/ Ruggedness/Stability • Precision (only for semi-quantitative method)

  16. Selectivity/specificity • Specificity: ability of a method to distinguish between analyte being measured and other substances problem of interference? • F(measuring technique, class of compounds, matrices,…)

  17. How to test specificity for qualitative screening method? Analyse 20 different blank samples and 20 positive samples (blind study, same or different days/technicians) Selectivity/specificity Specificity= 100* NA/N- Other parameters: Accuracy= 100* (PA+NA)/(N- + N+) Sensitivity= 100* PA/N+ False positive= 100* FP/(N- + N+) False negative= 100* FN/(N- + N+)

  18. Selectivity/specificity • How to test specificity for semi-quantitative screening methods? • Select potentially interfering substances (metabolites, derivatives,…) • Analyse relevant blank samples (n ≥ 20) • Analyse fortified blank samples with interfering substances at a relevant concentration • Estimate the effect of the interferences • False identification? • Influence in quantification? • Identification of the target analyte is hindered?

  19. Performance characteristics • Detection capacity • Selectivity/Specificity • Applicability/ Ruggedness/Stability • Precision (only for semi-quantitative method)

  20. Applicability • Scope of the method must be define in term of : • Matrix (solid/liquid matrix, type of tissue) • Animal species • To introduce a new matrix • Analyse at least 10 different blank material fortified at level of interest for the new matrix (CCβ) + test of interferences • If 10 positive results => method applicable for the new matrix • If 1 negative result => 10 additional analyses • If 1 negative result=> CCβ must be recalculated for the new matrix

  21. Ruggedness • Ruggedness: the susceptibility of an analytical method to changes in experimental conditions • sample material • analytes • storage condition • environmental condition • sample preparation condition

  22. Ruggedness • How to test ruggedness? (during development) • Identify possible factor that could influence the results (the analyst, solvents, pH, T°, rate of heating,…) • Vary each factor slightly • If one factor is found to influence results of the representative molecule, conduct further experiments => acceptability limits for this factor (in the method protocol) Recommendation of CRL: analyses of 10 blank and 10 spiked samples at the same concentration and with minor change of factor to detect influence on results

  23. Stability • Test are not necessary if stability data already exist (from other lab or from publication) • To include in the validation report • Stability test: • the analyte in solution • the analyte in matrix Aliquots of a fresh solution or sample stored under different conditions (T° and/or storing time)

  24. Performance characteristics • Detection capacity • Selectivity/Specificity • Applicability/ Ruggedness/Stability • Precision (only for semi-quantitative method)

  25. Precision (for quantitative screening) • Precision: the closseness of agreement between independent test results obtained under predetermined conditions • Expressed in terms of imprecision / standard deviation of test results

  26. Precision (for quantitative screening) • How to test precision? • Repeatability test • within-laboratory reproducibility test(or intermediate precision) • Reproducibility test (between laboratories: interlaboratory studies) determination of RSD (%) < Precision criteria

  27. Repeatability 3 concentrations: 1x; 1,5x; 2x MRPL 0,5; 1x; 1,5x MRL 6 replicates/level 3 times same conditions Within-laboratory Reproducibility 3 concentrations: 1x; 1,5x; 2x MRPL 0,5; 1x; 1,5x MRL 6 replicates/level 3 times different conditions (analyst, env. condition,…) Precision (for quantitative screening)

  28. Precision (for quantitative screening) • ANOVA treatment of data => RSDr & RSDRW • Comparison with precision criteria: • Horwitz equation: RSDR(%) = 2(1-0.5logC) • Criteria for repeatability: RSDr = 1/2 to 2/3 RSDR • Criteria for within-lab reproducibility: RSDRW = 2/3 to 1 RSDR ! For concentration < 100 µg/kg, RSDR becomes too high!

  29. Other recommendations • False negative rate <5%: Analyses of 20 negative and 20 positive samples in order to test the screening method (see selectivity). • One QC sample must be added in routine and results must be added to the validation file • Method transfer/Commercial test • Bibliographical survey to compil the evaluation of performance of the test • Collection of data from supplier on validation study • Experimental plan to test skillness of technician to perform the test • Use of QC sample • Participation to proficiency test

  30. Exemple: analyse of PCDD/F by CALUX bioassay • PCDD/F: 17 toxic congeners to analyse in various matrices (TCDD=most toxic dioxin) • Results expressed in TEQ (=Sum (CCixTEFi)i=1-17) • MRL for each matrix (milk, meat, egg, fish oil,…) • MRL expressed in pg TEQ/g fat or ng TEQ/ kg • Reference method: GC-HRMS • Screening method: immunoassay, bioassay,…

  31. Gene expression LIGHT Exemple: analyse of PCDD/F by CALUX bioassay CALUX bioassay= genetically modified cell-based bioassay (luciferase) Amount of light produced is proportional to the toxicity (TEQ) of extracts All substances fixing the Ah receptor

  32. Analyse of PCDD/F by CALUX bioassay • Advantage: • Rapid • Cheaper than GC-HRMS • Time for analyses • Disadvantage: • Various compounds can fix the Ah receptor (PAH, PCB, PHDD/F,…) specificity!!!!

  33. Analyse of PCDD/F by CALUX bioassay : protocol Extraction of fat Clean-up on silica acid + carbon columns Fraction with interfering compounds Fraction with PCBs Fraction with PCDD/F Evaporation Reading plate Dosing plate

  34. Analyse of PCDD/F by CALUX bioassay : validation • Selectivity/specificity • Ruggedness/Stability • Precision • Detection capability

  35. Analyse of PCDD/F by CALUX bioassay : selectivity • Possible interfering compounds? • PAH : mostly in environmental sample • PCB: fractionation during clean-up • Other compounds? (PHDD/F): dependant of the matrix? (matrix effect?) • Results of the selectivity test: • No interferences for feedstuff, milk, egg, fat • Interferences for fish oil CALUX results = 2 x GC-HRMS results

  36. Analyse of PCDD/F by CALUX bioassay : selectivity • Matrix effect for fish oil

  37. Analyse of PCDD/F by CALUX bioassay : ruggedness • What are the critical point in the protocol? • Carbon column (interferences) • Solvent (interferences) • Curve (results) • Evaporation time (recovery) • Age of CALUX cell line (RSD)

  38. Analyse of PCDD/F by CALUX bioassay : ruggedness • Carbon column: amount of carbon used (Rdt PCDD/F= 60%) (Rdt PCDD/F= 80%) DX fraction PCB fraction Not collected fraction (Rdt PCDD/F= 80%)

  39. Analyse of PCDD/F by CALUX bioassay : ruggedness • Evaporation time

  40. Analyse of PCDD/F by CALUX bioassay : ruggedness • Solvent: tested before use on a TCDD solution (antagonist/agonist effect) • Curve: tested with an independant TCDD solution • Age of CALUX cells: new cell every 2 months

  41. Analyse of PCDD/F by CALUX bioassay : precision • Validation protocol

  42. Analyse of PCDD/F by CALUX bioassay : precision • ANOVA results for the TEQ determination of PCDD/F in feedstuff by CALUX bioassay • At MRL (0.75ng TEQ/kg) : XMRL= 0.751 ng TEQ/kg Sr= 0.063 => RSDr= 8.4% SRW=0.073 =>RSDRW= 9.7% • At MRL/2 (0.376ng TEQ/kg) : XMRL/2= 0.464 ng TEQ/kg Sr= 0.051 => RSDr= 11% SRW=0.051 =>RSDRW= 11% • At 2MRL (1.5ng TEQ/kg): X2MRL= 1.571 ng TEQ/kg Sr= 0.107 => RSDr= 6.8% SRW=0.115 =>RSDRW= 7.3% RSD < 30% (2002/70/EC)

  43. Analyse of PCDD/F by CALUX bioassay : detection capacity • CCβfor the TEQ determination of PCDD/F in feedstuff by CALUX bioassay • CCα = MRL + 1.64 x SRW CCα = 0.75 + 1.64 x 0.073 = 0.87 ng TEQ/kg • CCβ = CCα + 2.33 x SRW CCβ = 0.87 + 2.33 x 0.073 = 1.04 ng TEQ/kg 2002/70/EC: false negative rate < 1% ! => At a concentration of 1.04ng TEQ/kg, we are sure that the sample is a positive sample with 99% certainty

  44. Analyse of PCDD/F by CALUX bioassay :confirmatory range ? COMPLIANT SUSPICIOUS NON COMPLIANT CC* MRL CCa CCb -2.33sMRL +1.64sMRL +2.33ssample Signal orConcentration *=1% β=5% α=5%

  45. Analyse of PCDD/F by CALUX bioassay :confirmatory range • Lower limit of the confirmatory rangefor the TEQ determination of PCDD/F in feedstuff by CALUX bioassay • CC*= MRL-2.33 x SDRW CC*= 0.75- 2.33 x 0.073 = 0.58 ng TEQ/kg • Conclusion • Sample lower than 0.58 ng TEQ/kg are negative with 99% certainty (false negative rate < 1%) • Sample above 0.58 ng TEQ/kg must be confirmed by GC-HRMS

More Related