A Step in the Right Direction? t he development of USP chapter <1210>

1. A Step in the Right Direction? the development of USP chapter <1210> Charles Y. Tan, PhD USP Statistics Expert Committee

2. Outline • Introduction of <1210> • Key topics • Accuracy and Precision • Linearity • LOD, LOQ, range • Summary

3. USP <1210> • United States Pharmacopeia • General Chapters • <1210> Statistical Tools for Method Validation • Current status: a draft is published in Pharmacopeial Forum40(5) [Sept-Oct 2014] • Seek public comments

4. Purpose of <1210> • A companion chapter to <1225> Validation of Compendial Procedures • USP <1225> and ICHQ2(R1) • USP <1033> Biological Assay Validation • Statistical tools • TOST, statistical equivalence • Statistical power, experimental design • tolerance intervals, prediction intervals • Risk assessment, Bayesian analysis • AIC for calibration model selection

5. Recent Framework • Life cycle perspective • procedure design • performance qualification / validation • ongoing performance verification • ATP: Analytical Target Profile • Pre-specified acceptance criteria • Assume established • Validation: confirmatory step • Statistical interpretation of “validation”

6. Performance Characteristics • Different statistical treatments • Tier 1: accuracy and precision • Statistical “proof” ATP is met • Equivalence test / TOST • Sample size / power, DOE • Tier 2: linearity, LOD • Relaxed evidential standard, estimation • Sample size / power optional

7. Key topics USP General Chapter <1210> Statistical Tools for Method Validation

8. Accuracy and Precision • Separate Assessment Of Accuracy And Precision • Confidence interval within acceptance criteria from ATP • Combined Validation Of Accuracy And Precision • γ-expectation tolerance interval: 100γ% prediction interval for a future observation,Pr (-λ≤ Y ≤ λ) ≥ γ • γ-content tolerance interval: 100γ% confidence of all future observations • Bayesian tolerance interval

9. Experimental Condition • Yij = μ + Ci + Eij • Ci: experimental condition • combination of ruggedness factors: analyst, equipment, or day • DOE: experience the full domain of operating conditions • As independent as possible • Eij: replication within each condition • One-way analysis (w/ random factor): why?

10. Separate Assessment • Closed form formulas: • Accuracy: classic confidence interval for bias • Precision: confidence interval for total variability under one-way layout (Graybill and Wang) • Power and sample size calculation • Statement of the parameters: bias, variance • Eg. CI of bias: [-0.4%, 1.1%], within ±5% (ATP) • Eg. CI of total variability: ≤2.4%, within 3% (ATP) • Implicit risk level: 95% confidence intervals

11. Combine Accuracy and Precision • Statement of observation(s) • Closed form formulas, but a bit more complicate • 99%-expectation tolerance interval: eg. [-4.3%, 5.0%] within ±10% (ATP) • 99%-content tolerance interval: eg. [-5.9%, 6.6%] within ±15% (ATP) • Bayesian tolerance interval • “the aid of an experienced statistician is recommended” • Simpler Alternative: directly assess the risk with the λgiven in ATP • Pr (-λ≤ deviation from truth ≤ λ|data)

12. Scale of Analysis • Pooling variances is central to stat analysis • Variance estimates with df=2 are highly unstable • Need to pool across samples, levels • Variance at mass or concentration scale/unit • Increase with level • Solutions: • Normalize with constants, eg. Label claim • Normalizing by observed averages makes stat analysis too complicated • Log transformation • %NSD and %RSD

13. Linearity • Internal performance characteristic • External view: accuracy and precision • Transparency => credibility • Appropriateness of standard curve fitting • A model • A range • Better than the alternatives (all models are approximations) • Proportional: model: Y = β1X + ε • Straight line: Y = β0 + β1X + ε • Quadratic model: Y = β0 + β1X + β2X2 + ε

14. Current Practices • Pearson correlation coefficient • Anscombe's quartet • Lack-of-fit F test • independent replicate • Mandel’s F-test, the quality coefficient, and the Mark–Workman test • Test of significance • Evidential standard: low since it gives the benefit of doubt to the model you want • Good precision may be “penalized” with a high false rejection rate • Poor precision is “rewarded” with false confirmation of the simpler and more convenient model

15. Anscombe's Quartet

16. Two New Proposals • Equivalence test, TOST, in concentration units • Define maximum allowable bias due to calibration in ATP • Construct 90% confidence interval for the bias comparing the proposed model to a slightly more flexible model • Closed form formula, complex • Evidential standard: could be high, depend on allowable bias • Akaike Information Criterion, AICc • Compare the AICc of the proposed model to a slightly more flexible model (smaller wins) • Very simple calculations • Evidential standard: most likely among candidates

17. Different Burden of Proof • Hypothesis Testing: Neyman-Pearson • Frame the issue: null versus alternative hypotheses • Goal: reject the null hypothesis • Null hypothesis: protected regardless of amount of data • Decision standard: beyond reasonable doubt • Legal analogy: criminal trial • Information Criteria: Kullback-Leibler • Frame the issue: a set of candidate models • Goal: find the best approximation to the truth • Best: most parsimonious model given the data at hand • Decision standard: most likely among candidates • Legal analogy: civil trial • Stepping-stone or tactical questions: information criteria are apt alternatives to hypothesis tests

18. IUPAC/ISO LOD (RC and RD)

19. IUPAC/ISO LOD

20. LOD: Using Prediction Bounds

21. Range and LOQ • Range • suitable level of precision and accuracy • Both upper and lower limits • LOQ (LLOQ) • acceptable precision and accuracy • lower limit • LOQ versus LOD • Only one is needed for each use • LOQ for quantitative tests • LOD for qualitative limit tests • LOQ calculation in ICHQ2: candidate starting values

22. Summary • A draft of USP <1210> is published, seeking public comments • A step in the right direction? • More than a bag of tools • Implement modern validation concepts with a statistical structural • More tools development needed • More statisticians involvement needed in pharmacopeia and ICH development