Is it potent? Can these results tell me? Statistics for assays

1. Is it potent? Can these results tell me?Statistics for assays Ann Yellowlees PhD Quantics Consulting Limited

2. Contents • Role of statistics in bioassay • Regulations • Estimating Relative Potency • Choice of model for RP estimation • Parallelism • Case study

3. Role of statistics in bioassay Design / optimisation Analysis Validation

4. Why use statistics - regulations • Validation of routine and custom assays: • ICH Q6B • SPECIFICATIONS: TEST PROCEDURES AND ACCEPTANCE CRITERIA FOR BIOTECHNOLOGICAL/BIOLOGICAL PRODUCTS • “Assessment of biological properties constitutes an ....essential step in establishing a complete characterisation profile” • “Appropriate statistical analysis should be applied” • “Methods of analysis, including justification and rationale, should be described fully” • “A relevant, validated potency assay should be part of the specifications for a biotechnological or biological drug substance and/or drug product”

5. Why use statistics - regulations • Stability testing, more defined by ICH: • ICH Q5C • QUALITY OF BIOTECHNOLOGICAL PRODUCTS • “At time of submission, applicants should have validated the methods that comprise the stability-indicating profile and the data should be available for review” • ICH Q1A (R2) • STABILITY TESTING • “An approach for analyzing data of quantitative attribute that is expected to change with time is to determine the time at which the 95% one-sided confidence limit for the mean curve intersects the acceptance criterion”

6. Estimating Relative Potency • Analysis of dose – response data • Choosing the best model for estimating RP • Checks for parallelism • Estimating RP • Calculations: RP and its precision • Improving precision

7. Data types • Response per unit (animal, welI, etc): • Binary • Dead / alive at a given time point • Diseased / disease free at a given time point • Summarised as % or proportion • Continuous • Antibody level • Time of death • Optical density

8. Continuous response • Note: • Log concentration • S shape • Noise level varies

9. Continuous response – means

10. What is Relative Potency? • Potency of the sample in comparison with the reference Mathematically this is the same as:

11. Estimating RP

12. When is it valid to calculate RP? • When the bioassay is a dilution assay “the unknown preparation to be assayed is supposed to contain the same active principle as the standard preparation, but in a different ratio of active and inert compounds”** • Implies RP constant across concentrations • One curve is a horizontal shift of the other • i.e. ‘parallel’ curves ** European Pharmacopoeia 3.1.1

13. Ph. Eur approach: Residual sum of squares (RSS) and the F-test Arbitrary p value Penalises ‘good’ data USP Confidence intervals on differences between parameters Arbitrary confidence level Arbitrary limits on width Penalises ‘bad’ data Others Chi squared test Similar to (1) Check for parallelism

14. EP, USP are guidance only ‘No simple, generally applicable statistical solution exists to overcome these fundamental problems. The appropriate action has to be decided on a case-by-case basis’ USP Workshop 2008 Check for parallelism

15. Estimating RP from data • Choose a model; fit to each material • Check system suitability • reference is behaving as expected • parallel models are appropriate • Calculate RP and 95% confidence interval

16. Choose a model

17. Choose a model

18. Linear model(4 concentrations) Assume: Middle 4 concentrations of interest Parallel when β the same for both materials

19. Four parameter logisticmodel Note: If A = 0 and B = 1 this is a simple logistic model: proportions.

20. Four parameter logisticmodel Parallel: when A, B and scale are the same for both materials

21. Five parameter logisticmodel Parallel: when A, Bscale and asym are the same for both materials

22. Which model to use?

23. Which model to use? ‘Is a 5PL model really necessary or is it a statistical remedy for a bad assay?’ R Capen Chair, USP workshop 2008 • Consider the relevant range of concentrations • How much do you need to know about the ends? • Pros and cons – • Need more data to fit curves • More data = more precision? • Weighting / variability at ends • Formal statistical tests for fit

24. 4 PL parallel model chosen

25. Calculate RP and 95% CI • Parallel model provides an estimate of logeRP • Horizontal distance between the curves • logeRP = 0.233 Back-transform for RP • RP = e0.233 = 1.26 • 95% confidence interval for RP (1.26) • (0.84, 1.90)

26. Assay development / optimisation • Choose statistical model • Design assay • Number of replicates per concentration • Operators, days etc • to achieve required precision for RP • Set suitability criteria for assay • Reference behaviour • Parallelism

27. Model selection for an assay Model must: • Fit the data • Allow RP calculation most of the time • i.e. the curves are parallel • Provide precise estimates of RP

28. Example with 12 plates • 12 development plates run • Wide range of concentrations • 0.001 – 2000 IU/ml • Reference and sample • 3 replicates • 4 statistical models examined • Linear (4), Linear (6), 4PL, 5PL • Parallelism • Precision • Fit

30. Summary: Parallelism test (F) * Denominator = 12

31. Summary: Precision

32. Linear model: 4 points, parallel

33. Summary: Model selection • Linear model based on 4 concentrations • All 12 pairs passed linearity test • All 12 pairs passed parallelism test • Provided the best precision • No apparent bias

34. Improving precision • If the linear model can be justified: • Allows extra replication • Better precision within plate • ? fewer plates required • How low can you go? 2 doses: test for linearity cannot be done 3 doses: test for linearity has low power

35. Summary • System software provides most of the required statistics per plate • When do you need a statistician? • Choosing model • “Appropriate statistical analysis should be applied” • “Methods of analysis, including justification and rationale, should be described fully” • Designing and validating assay • Assessing sources of variation • Simulation • Setting suitability criteria • “A relevant, validated potency assay should be part of the specifications for a biotechnological or biological drug substance and/or drug product”

36. Thank you • BioOutsource • The invitation • The data • Quantics staff • Kelly Fleetwood, Catriona Keerie • Analysis and graphics