Accreditation & Validation

1. Accreditation & Validation Joris Van Loco Scientific Institute of Public Health Food Section

2. Method Validation • Is method validation • analyzing 6 samples ? • Calculating the bias, repeatability, reproducibility,… of a method ? • Knowing the detection limits of the method ? • knowing the uncertainty associated with a method? • satisfying ISO 17025 assessors?

3. What is Method Validation? • Method validation is the process of proving that an analytical method is acceptable for its intended purpose

4. Why is Method Validation Necessary? • To prove what we claim is true • To increase the value of test results • To justify customer’s trust • To trace criminals Examples • To value goods for trade purposes • To support health care • To check the quality of drinking water

5. New method development Revision of established methods When established methods are implemented in new laboratories Interlaboratory Comparison Single lab validation Full Validation Implementation Validation Method performance parameters are determined using equipment that is: Within specification Working correctly Adequately calibrated Competent operators When and How should Methods be Validated

6. In house validation (Bias), recovery Repeatability Within lab reproducibility Internal QC Control charts Starting data Validation and Quality Control • Long term within lab reproducibility • Proficiency Testing • Bias (trueness) • Collaborative trial • Reproducibility • Bias (trueness)

7. Method Validation • Accuracy • Trueness (CRM) • Recovery (spikes) • Precision • Repeatability • (Within) reproducibility • Selectivity (& Specificity) • Detection capability • LOD, LOQ, CC, CC • Linearity – calibration range • Robustness • Applicability – stability

8. Method ValidationPerformance Characteristics 2002/657/CE S: Screening methods C: Confirmatory methods

9. Linearity • Purpose • to evaluate the linear response of your instrument • How • Evaluating your calibration model • Mandels fitting test • Lack-of-Fit • Residuals • Conclusion • Linear model • <> other (i.e. quadratic) regression model

10. Residual plots (ei) with Statistical tests Lack-of-fit Mandel’s fitting test Linearity

11. Coefficient of correlation (r) • Is NOT a suitable measure for linearity

12. Matrix Effect • Purpose • To evaluate whether you have a concentration dependent systematic error due the matrix • i.e. ion suppression • How • comparison of standard curve with matrix matched standard curve • Conclusion • Standard solutions, spiked extracts or spiked samples for the calibration line.

13. Detection Limits Detection limit DIN 32645 from blanks from calibration data Funk dynamic model IUPAC Coleman recursive formula explicit formula

14. Detection Limits A) DIN 32645 Detection limit by fast estimation: Capability limit Determination limit by fast estimation Factor for fast estimation

15. Detection Limits B) Funk Detection limit dynamic model Determination limit dynamic model

16. Detection Limits C) IUPAC Detection limit

17. Detection limits “How to” • Choose a definition and stick to it • Describe the equation used in the validation file • Problems • statistics <> practical limitations • statistics <> ID-criteria • Practical LOD • Analyzing samples with decreasing concentration • Minimum concentration which fulfills the identification criteria = practical limit of detection • Repeat the experiment • S/N • i.e. LOD=3xS/N

18. Quantitation Limit (LQ) • The quantification limit is the minimum signal (concentration or amount) the can be quantified. • the residual standard deviation (RSD) is included in the definition. • The IUPAC default value for RSDQ= 0.1 (or 10%). LQ=10sQ.

19. a- and b-error • a-error • risk of erroneously rejecting H0 • i.e. risk of the conviction of an innocent • b-error • risk of erroneously accepting H0 • i.e. risk of the non conviction of a criminal

20. Detection CapabilityCase of a permitted limit (MRL) CCa MRL CCb +1.64sMRL +1.64ssample Signal orConcentration a = b = 5% a = 5%

21. Determination of CCa and CCb with ISO 11843 yc CCa CCb MRL

22. Detection CapabilityCase of a permitted limit (MRL)

23. Detection CapabilityCase of no established permitted limit or banned substance Xblank CCa CCb +2.33sblank +1.64ssample Signal orConcentration a = 1%≠b = 5%

24. Presence of Heteroscedasticity Nitroimidazoles in plasma (MNZ-OH) • Residuals plot • “<“ - shape • Plot of S vs conc • Linear relationship between S and concentration • Heteroscedasticity • Impact on CCa and CCb • CCa and CCb are incorrectly calculated • Sblank ↓  CCa ↓ • CCb ↓ or ↑

25. Other examples • Nitroimidazoles in plasma • Nitrofurans in honey • Corticosteroids in liver

26. Weighted regression equations for CCa and CCb • Solved by iteration

27. Conclusion detection capability • Many definitions of detection limits • detection limit (≈ CCa_banned substances) • determination limit (≈ CCb_banned substances) • Quantition limit • Complicated statistics • KISS • demonstrate with real (spiked) samples at low concentration level  practical limit of detection

28. Selectivity/Specificity • Identity: Signal to be attributed to the analyte • GLC (change column/polarity), GC/MS, Infra-red • Selectivity: The ability of the method to determine accurately the analyte of interest in the presence of other components in a sample matrix under the stated conditions of the test. • Specificity is a state of perfect selectivity

29. Selectivity • The procedure to establish selectivity: • Analyze samples and reference materials • Assess the ability of the methods to confirm identity and measure the analyte • Choose the more appropriate method. • Analyze samples • Examine the effect of interferences

30. Selectivity: Verification of the identification criteria (2002/657/EC) • MS – criteria • 3 or 4 identification points • 1 precursor and 2 transition ions • Relative ion intensities • LC – criteria • Relative retention time (RRT): +/- 2.5 % (LC) • UV – criteria • Spectrum match • +/- 3 nm • CCb is concentration at or above the calculated CCb for which the ID criteria are fulfilled in 95% of the cases. • CCa is concentration at or above the calculated CCa for which the ID criteria are fulfilled in 50% of the cases.

31. Ruggedness and Robustness • Intra-laboratory study to check changes due to environmental and/or operating conditions • Usually it is part of method development • Deliberate changes in • Temperature • Reagents ( e.g. different batches) • Extraction time • Composition in the sample • etc

32. Precision – ISO 5725 1-6 (1994) • Expresses the closeness of agreement (dispersion level, relative standard deviation) between a series of measurements from multiple sampling of the same homogeneous sample (independent assays) under prescribed conditions. • Irrespective of whether mean is a correct representation of the true value. • Gives information on random errors • Evaluated at three levels: • repeatability • intermediate precision (within laboratory) • reproducibility (between laboratory)

33. Precision (cont.) – ISO 5725 1-6 (1994) • Repeatability: precision under conditions where the results of independent assays are obtained by the same analytical procedure, on identical samples, in the same lab, by the same operator, using the same equipment and during short interval of time • Intermediate precision: ISO recognizes M-factor different intermediate precision conditions (M = 1, 2 or 3) • M = 1: only 1 of 3 factors (operator, equipment, time) is different • M= 2 or 3: 2 or all 3 factors differ between determinations

34. Precision (cont.) – ISO 5725 1-6 (1994) • Reproducibility: precision under conditions where results obtained: • by same analytical procedure • on identical sample • in different laboratories, different operators, differentequipment • Reproducibility established by interlaboratory study (standardisation of an analytical procedure) Intermediate precision • RepeatabilityReproducibility

35. Evaluation of Precision • 10 samples for each conc.under r,R, within lab R • Standard Deviation • Determination in pairs under r,R, within lab R • Std. Dev. between two single determinations • a-b, the difference between the values, d, the number of pairs sr SR SRw

36. Repeatability (r) and within-lab reproducibility (Rw) ANOVA table for a single factor balanced design with 3 replicate samples on the same day. repeatability (Sr²) and within-lab reproducibility variances (SRw²) Sr² = Srepl² SRw² = Sr² + Sdays² The Srepl²and Sdays² can be obtained from mean squares as (nrepl = 3): Srepl² = MSrepl Sdays² = (MSdays – MSrepl) / 3

37. Repeatability and reproducibility • The value of 2.8? • Variance of difference between 2 replicate measurements is 2s² • Confidence interval at 95% level on the difference is 0 ± 1.96 √2 s  ± 1.96 x 1.41 sr = ± 2.8 sr •  95% probability that difference between duplicate determinations will not exceed 2.8 sr • r = limit of the repeatability r = 2.8 sr • R = limit of the reproducibility R = 2.8 SR

38. Precision criteria 2002/657/CE

39. Horwitz: RSDR(%) = 2(1-0.5logC)

40. Determination of Trueness • Using Certified Reference Materials • Using RM or In-house materials • Using Reference methods • Single sample • Many samples • Via Interlaboratory study

41. Trueness, extraction yield (recovery) and apparent recovery • Trueness means the closeness of agreement between the average value obtained from a large series of test results and an accepted reference value. Trueness is usually expressed as bias • Recovery (extraction yield): yield of a preconcentration or extraction stage of an analytical process for an analyte divided by amount of analyte in the original sample. • Apparent recovery: observed value derived from an analytical procedure by means of a calibration graph divided by reference value.

42. Trueness criteria 2002/657/CE • When no such CRMs are available, it is acceptable that trueness of measurements is assessed through recovery of additions of known amounts of the analyte(s) to a blank matrix. Data corrected with the mean recovery are only acceptable when they fall within the ranges