Efficient Experimental Design – Obtaining Maximum Information from a Minimum of Analysis.

1. Efficient Experimental Design – Obtaining Maximum Information from a Minimum of Analysis. Seán Earley B.Sc., Ph.D. Veterinary Public Health Regulatory Laboratory

2. Contents • Objectives of Presentation • Analysis Requirements • Validation Requirements • Basic Experimental Design Approaches • Multi Factorial/Model Approaches • Youden’sApproach to Ruggedness Studies • InterVAL • Resval

3. Objectives of Presentation • Give a brief summary of considerations in regard to analysis and validation requirements. • Give an introduction/overview of a number of approaches to efficient experimental design*. *Note: these approaches are derived from the requirements of Commission Decision 2002/657/EC – Concerning the Performance of Analytical Methods and the Interpretation of Results - although Veterinary Residues Analysis is the main focus of 2002/657, these approaches can be applied to the validation of methods for other areas of analysis.

4. Analysis Requirements • Define Method: Screening (Qualitative/Semi Quantitative) / Confirmatory? • Scope of Method – Target Analytes / Materials or Matrices to be Analysed • Client Requirements • Legislative Requirements/Criteria • Accreditation Requirements

5. Validation Requirements General Guidelines B. Magnusson and U. Örnemark (eds.) Eurachem Guide: The Fitness for Purpose of Analytical Methods – A Laboratory Guide to Method Validation and Related Topics, (2nd ed. 2014). ISBN 978-91-87461-59-0. Available from http://www.eurachem.org

6. Validation Requirements Legislative Guidelines 2002/657/EC: Commission Decision of 12 August 2002 implementing Council Directive 96/23/EC concerning the performance of analytical methods and the interpretation of results Available from: http://eur-lex.europa.eu

7. Validation Requirements ISO 17025 Guidelines INAB – PS15 – Guide to Method Validation for Quantitative Analysis in Chemical Testing Laboratories (Issue 3, April 2012) Available from: http://www.inab.ie/

8. Validation Requirements GMP Guidelines ICH Guidelines on the Validation of Analytical Procedures Available from: http://www.ich.org

9. Typical Validation Criteria

10. Validation – 2002/657/EC Guidelines • VPHRL LC-MS/MS methods – Screening & Confirmatory - 2002/657/EC • Measurement Uncertainty also required for ISO 17025 Accreditation

11. Validation – 2002/657/EC Guidelines

12. Specificity – Available Efficiencies? • Matrix – Biological / Multi-Species / Inconsistent Sample Matrix (e.g. CFS) • Method Development - Consider 3 or More MRMs (if available) when developing to allow flexibility should problems arise at Validation, e.g. Species Related Matrix Peaks • Typically No Short Cut if establishing a pool of negative material from scratch for Multi Vet. Residue Methods across several species. • Utilise test samples from Cut-Off (CCβ, minimum of 20) experiments for specificity study.

13. Validation – Cut-Off (CCβ) • Minimum of 20 Negative Samples and 20 Spiked Samples to determine Cut-Off if using the CRL Guidelines1 • 20 Negative and 20 Spiked Samples from Repeatability & Reproducibility experiments • The 20 Samples can incorporate several different species* • Provides data for Specificity and Cut-Off (CCβ) *Guidelines for the Validation of Screening Methods for Residues of Veterinary Medicines (initial validation and transfer) produced by the Community Reference Laboratories Residues (CRLs) 20/1/2010

14. Experimental Design: Basic Approach Specificity/Repeatability/Reproducibility/CCβ & CCα / Measurement of Uncertainty can be derived from 4 Sets of Experiments for Multi-Species Methods e.g. Phenicols in Muscle.

15. Experimental Design: Basic Approach Specificity/Repeatability/Reproducibility/CCβ & CCα / Measurement of Uncertainty can be derived from 4 Sets of Experiments for Multi-Species Methods Specificity & Cut-Off (CCβ)

16. Experimental Design: Basic Approach Specificity/Repeatability/Reproducibility/CCβ & CCα / Measurement of Uncertainty can be derived from 4 Sets of Experiments for Multi-Species Methods CCα, Cut-Off (CCβ), MoU

17. Experimental Design: Basic Approach Precision & Accuracy / Reproducibility / Working Range Specificity/Repeatability/Reproducibility/CCβ & CCα / Measurement of Uncertainty can be derived from 4 Sets of Experiments for Multi-Species Methods

18. Streamline by Species Sampling Level Experiments by Species in Relative Proportion to the Sample Numbers taken Annually for the National Residue Control Plan – A Single Cut-Off and CCα across species rather than calculating individually by species! – 112 samples vs. >300 samples. (6 x 40)

19. Multi Factorial /Other Approaches • 2002/657/EC –3.1.1.3 Youden Method for ruggedness testing. • ResVal– Streamlined/Standardised Approach to meet the requirements of 2002/657/EC • 2002/657/EC – 3.1.3 Validation according to alternative models: 2-Level Design – series of factors e.g. storage conditions, samples freshness, different operators – InterVal Software.

20. Youden Approach to Ruggedness Tests • Fractional factorial design - Interactions between the different factors cannot be detected. • Where a factor is found to influence the measurement results significantly, further experiments are required to decide on the acceptability limits of this factor. • Factors that significantly influence the results should be identified clearly in the method protocol.

21. Youden Approach to Ruggedness Tests • The basic idea is not to study one alteration at a time but to introduce several variations at once. As an example, let A, B, C, D, E, F, G denote the nominal values for seven different factors that could influence the results if their nominal values are changed slightly. • Let their alternative values be denoted by the corresponding lower case letters a, b, c, d, e, f and g. This results in 27 or 128 different possible combinations.

22. Youden Approach to Ruggedness Tests • It is possible to choose a subset of eight of these combinations that have a balance between capital and small letters. Eight determinations have to be made, which will use a combination of the chosen factors (A-G). • 1 Experiment with only 8 samples required!

23. Youden Approach - Example Calculations for Ruggedness (For factor A/a): Averages: AA =  (Ai)/4 Aa =  (ai)/4 Differences: Da = A – a =  (Ai) -  (ai) Standard Deviation of the Differences: SDi = √(2 x  (Di2/7))

24. Youden Approach - Example

25. Youden Approach - Example • If the SDi is significantly larger than the standard deviation of the method carried out under within-laboratory reproducibility conditions it is a concluded that all factors together have an effect on the result even if any single factor does not show a significant influence; in this case it can be concluded that the method is not sufficiently robust against the chosen modifications. • In the above case SDi = 6.76% • %CV of Within Lab Reproducibility = 10.3% • Therefore the method above is sufficiently robust for Clenbuterol in Retina as SDi<< WLR.

26. ResVal • ResVal – available from Wageningen University and Research (RIKILT, Contact: Marco Blokland) • 4 Day experimental approach. • Determines all of the criteria required by 2002/657/EC using a pre-defined set of 83 experiments.

27. ResVal Experimental Strategy: VL = Validation Level

28. ResVal • Conventional vs. ResVal Approach

29. ResVal • Microsoft Excel Platform

30. InterVal • InterVAL – software package from QuoData • Prescribes experimental design and performance criteria for a number of standards including 2002/657

31. InterVal • Utilises the ‘Validation According to Alternative Models’ Approach in 2002/657/EC – Section 3.1.3. • Recommended by the EURL, BVL – Berlin Uhlig, Steffen; Gowik, Petra; Radeck, W. (2003): Performance of a matrix-comprehensive in-house validation study by means of an especially designed software. In: AnalyticaChimicaActa 483 (1-2), S. 351–362. • Fractional Factorial Design – Based the use of Leading Factors and the variation of several sample/experimental sub-factors on 2 levels for each leading factor.

32. InterVal • For example, Species is set as the Leading Factor with 7 other ‘sub’ factors to be varied on 2 levels. • Possible ‘Sub’ Factors: Storage Conditions Breeding Sample Condition Operator Breed

33. InterVal • Example of Experiment Plan in InterVal – Uhlig et al. – 4-methylaminoantipyrine (MAA), Ramifenazone and Antipyrine • 3 Leading Factors: Calf, Bovine And Porcine • 8 ‘Sub’ Factors (@ 2 Levels): Muscle/Plasma Extensive/Intensive Breeding Cool/Warm Transport Cond.s Minced/Lyophilised Sample Fresh/Tainted Sample Frozen/Cooled Storage Operator A/B Uhlig, Steffen; Gowik, Petra; Radeck, W. (2003): Performance of a matrix-comprehensive in-house validation study by means of an especially designed software. In: AnalyticaChimicaActa 483 (1-2), S. 351–362.

34. InterVal • Validation Analysis at the following Levels: MAA: 100, 150, 200 and 300µg/kg (MRL = 200µg/kg) Ramifenazone and Antipyrine: 0.25, 0.375,0.50,0.75 and 1.00µg/kg (No MRL) • Various Statistical Analyses of Data and Calculations possible with InterVal: Outlier detection with Grubbs Test and Cochran Test. Calculation of CCα and CCβ (Power Curve for Detection Capability) Calculation of Uncertainty of Measurement at all levels Calculation of Repeatability, Within Lab Reproducibility %CV and % Recovery across all levels Factorial Effects Automated Calibration Plots Automated Power Curve Analysis Uhlig, Steffen; Gowik, Petra; Radeck, W. (2003): Performance of a matrix-comprehensive in-house validation study by means of an especially designed software. In: AnalyticaChimicaActa 483 (1-2), S. 351–362.

35. InterVal – Thiamphenicol in Milk Report (Demo) CCβ = 65.78

36. Acknowledgements • VPHRL Colleagues • Quodata • Eurachem

37. References • Commission Decision 2002/657/EC • Guidelines for the Validation of Screening Methods for Residues of Veterinary Medicines (Initial Validation and Transfer) – Community Reference Laboratories Residues (CRLs) • B. Magnusson and U. Örnemark (eds.) Eurachem Guide: The Fitness for Purpose of Analytical Methods – A Laboratory Guide to Method Validation and Related Topics, (2nd ed. 2014). ISBN 978-91-87461-59-0. Available from http://www.eurachem.org • PS15 – INAB Guide to Method Validation for Quantitative Analysis in Chemical Testing Laboratories. Available from http://www.inab.ie • ICH Guidelines on the Validation of Analytical Procedures. Available from: http://www.ich.org • W.J. Youden; Steiner, E.H.; Statistical Manual of the AOAC – Association of Official Analytical Chemists, AOAC-I, Washington DC: 1975, Pg. 35. • RIKILT – Resval Information: http://www.wur.nl/en/Expertise-Services/Research-Institutes/rikilt/Reference-laboratory/European-Union-Reference-Laboratory/Reference-standards/Software-ResVal.htm • Quodata Quality & Statistics – InterVal – www.quodata.de • Uhlig, Steffen; Gowik, Petra; Radeck, W. (2003): Performance of a matrix-comprehensive in-house validation study by means of an especially designed software. In: AnalyticaChimicaActa 483 (1-2), S. 351–362.