Statistical Aspects to Setting Specifications on Biological Products

1. Statistical Aspects to Setting Specifications on Biological Products Robert Capen, Ph.D. Associate Director, Scientific Staff Department of Vaccine Biometrics Research Merck & Co., Inc.

3. Introduction A specification is a list of tests, references to analytical procedures, and appropriate acceptance criteria with numerical limits, ranges, or other criteria for the tests described, which establishes the set of criteria to which a drug substance or drug product or materials at other stages of their manufacture should conform to be considered acceptable for its intended use. FR, 63, No. 110, 6/98, Notices

4. Introduction Acceptance Criteria: Numerical limits, ranges, or other suitable measures for acceptance which the drug substance or drug product or materials at other stages of their manufacture should meet to conform with the specification of the results of analytical procedures. FR, 63, No. 110, 6/98, Notices

5. Introduction Expiry limits represent a quality commitment over the entire shelf life of the product. They are a commitment by the manufacturer to its customers that the measured characteristic (e.g., potency) will fall within the prescribed limits, whether the product is measured on the day of its manufacture or on the day it expires. Dillard (2002)

6. Introduction Release limits are generated by the manufacturer to ensure compliance to the expiry limits. They apply only at the time of manufacture. They are means to adjust for the uncertainties caused by product instability and measurement variation so that, with a high degree of confidence, any batch that meets its release requirements will also conform to its expiry requirements over the shelf life of the product. Dillard (2002)

7. Introduction An action limit is an internal (in-house) value used to assess the consistency of the process at less critical steps. These limits are the responsibility of the manufacturer. FR, 63, No. 110, 6/98, Notices

8. Introduction Control or Tolerance Limits provide the boundaries between which results generated under normal operating conditions can be expected to fall. They reflect the range of common cause or chance variation expected in the data when the manufacturing process and analytical method are operating in a consistent (i.e., predictable) manner.

9. Introduction An Out-of-Trend (OOT) result is a stability result that does not follow the expected trend, either in comparison with other stability batches or with respect to results collected during a stability study. PhRMA CMC Statistics and Stability Expert Teams, Identification of Out-of-Trend Stability Results, Pharmaceutical Technology, April 2003

10. Introduction The Producer�s Risk (PR) is the risk associated in claiming that the characteristic being tested is OOT (or OOS) due to chance variation alone. �Investigating a �good� lot� The Consumer�s Risk (CR) is the risk associated in claiming that the characteristic being tested meets its acceptance criterion due to chance variation alone. �Releasing a �bad� lot�

11. Introduction Justification of the Specification Linked to the manufacturing process Account for instability in the drug substance or drug product Linked to preclinical and clinical studies Linked to analytical procedures FR, 64, No. 159, 8/99, Notices

12. Introduction

13. Action Limits Statistically derived as tolerance limits Assesses consistency of manufacturing process NOT quality of product Must account for pertinent sources of manufacturing and analytical variability Beware of multiplicity

14. Action Limits

15. Action Limits Choice of k Na�ve: k = 1.96, 2.576, 3 corresponding to approximately a 5%, 1% and 0.3% PR Tolerance factors: k=k(a, p, n) 1 � a = Confidence level 1 � p = Coverage level n = Sample size

16. Action Limits

17. Action Limits Exact values for k have to be obtained through numerical integration Odeh and Owen (1980) Partially reproduced in Hahn and Meeker (1991) Approximate values for k also exist and are simple to program into Excel Wald and Wolfowitz (1946) Gardiner and Hull (1966)

18. Action Limits

19. Action Limits

20. Action Limits

21. Action Limits

22. Action Limits s2 is an estimate of and can, for the balanced case (ni = n for all i), be expressed as

23. Action Limits

24. Action Limits

25. Action Limits

26. Action Limits Thomas and Hultquist (1978) show that . This leads to the reasonable estimate for var (y) of

27. Action Limits The corresponding df are given by

28. Action Limits For more complicated (balanced) design structures consult Beckman and Tietjen (1989). In practice, for either balanced or unbalanced data, a reasonable approximate approach involves estimating var (y) through a variance-component analysis and choosing for ? a value of 3 or 4.

29. Release Limits As mentioned previously release limits are in-house acceptance criteria derived so that if a lot passes these criteria it will satisfy its shelf-life criteria with high confidence.

30. Release Limits

31. Release Limits Is there anything missing? A term for lot-to-lot variability? No. The decision about the disposition of a particular lot is specific to that lot. The basic unit for decisions on quality (release, reject, retest, recall) is always the �lot.� A term accounting for variation in the slopes? Possibly.

32. Release Limits

33. Release Limits Why not just �pass� if the potency at time 0 is > L instead of > R? Does not account for assay variability. The consumer is not protected against the release of a non-efficacious lot. A time 0 potency result between L and R could just be assay variability working in our favor. To be �sure� need to have time 0 result > R. For a lot that is put on stability, is the value derived for R appropriate at other time points? Or is looking at data on a �point-by-point� basis the right thing to be doing to begin with?

34. Release Limits

35. Release Limits The degradation over the shelf life of the product is simply -bT where b is the estimated �average� slope May need to include multiple degradation steps Frozen/lyophilized storage to refrigerated liquid Sealing/packaging/inspection at RT Handling at physician�s clinic Marketing needs � RT stability claim Both assay and manufacturing sources of variability must be accounted for

36. Release Limits

37. Release Limits The �extra� multiplier (z0.05) is a result of how Wei formulated the problem. k depends on T and the number of replicates at each time point. If n replicates are obtained only at time 0 and time T, then T2k = 2/n

38. Release Limits

39. Release Limits

40. Release Limits

41. Release Limits

42. Release Limits

43. Release Limits

44. Release Limits Wei�s approach appears to be too conservative. It requires the 95% lower confidence limit to be above R to have high confidence that the true potency is above L. ADG�s approach agrees with the no-degradation derivation discussed earlier.

45. Final Thoughts

46. Final Thoughts For highly variable assays/processes, the release limits may be so large as to be unrealistic Inside the action limits At a level that would require the manufacture to target an unobtainable potency level Replication is the short-term solution to this problem. Long-term�process has to improve

47. Final Thoughts

48. Final Thoughts

49. Final Thoughts Statistically derived limits are only as good as the data that were used to generate them. Plan to re-evaluate them, especially if set early in the development of a new product.

50. Suggested Readings ICH Guideline �Q6A Specifications: Test Procedures and Acceptance Criteria for New Drug Substances and New Drug Products: Chemical Substances� ICH Guideline �Q6B Specifications: Test Procedures and Acceptance Criteria for Biotechnological/Biological Products� Federal Register, Vol. 63, No. 110, June 1998, Notices (note: this is essentially the same as Q6B) Federal Register, Vol. 64, No. 159, August 1999, Notices Boddy, A.W., et al, (1995), An Approach for Widening the Bioequivalence Acceptance Limits in the Case of Highly Variable Drugs, Pharm. Res., Vol. 12, No. 12. Hayakawa, T. (1997), Global Perspective on Specifications for Biotechnology Products - Perspective from Japan, Development of Specifications for Biotechnology Pharmaceutical Products. Dev. Biol. Stand. Vol 91, Brown, F. and Fernandez, J. (eds).

51. Suggested Readings Baffi, R. (1997), The Role of Assay Validation in Specification Development, Development of Specifications for Biotechnology Pharmaceutical Products. Dev. Biol. Stand. Vol 91, Brown, F. and Fernandez, J. (eds). Geigert, J. (1997), Appropriate Specifications at the IND Stage, Development of Specifications for Biotechnology Pharmaceutical Products. Dev. Biol. Stand. Vol 91, Brown, F. and Fernandez, J. (eds). PhRMA CMC Statistics and Stability Expert Teams (2003), Identification of Out-of-Trend Stability Results: A Review of the Potential Regulatory Issue and Various Approaches, Pharmaceutical Technology, April, Pages 38 � 52. Dillard, R.F. (2002), �Statistical Approaches to Specification Setting with Application to Bioassay,� The Design and Analysis of Potency Assays for Biotechnology Products (Brown, F. and Mire-Sluis, A., eds), Developments in Biologicals, Basel Karger, vol 107, pages 117-127.

52. Suggested Readings Hahn, G. and Meeker, W. (1991), Statistical Intervals: A Guide for practitioners, John Wiley & Sons, Inc., New York, N.Y. Wald, A. and Wolfowitz, J. (1946), Tolerance Limits for a Normal Distribution, Annals of Mathematical Statistics, 17. Gardiner, D. and Hull, N. (1966), An Approximation to Two-Sided Tolerance Limits for Normal Populations, Technometrics, Vol. 8, No. 1. Owen, D. B. (1968), A Survey of Properties and Applications of the Noncentral t-Distribution, Technometrics, Vol. 10, No. 3. Odeh, R. E. and Owen, D. B. (1980), Tables for Normal Tolerance Limits, Sampling Plans and Screening, New York: Marcel Dekker, Inc. Beckman, R. and Tietjen, G. (1989), Two-Sided Tolerance Limits for Balanced Random-Effects ANOVA Models, Technometrics, Vol. 31, No. 2.

53. Suggested Readings Thomas, J. and Hultquist, R. (1978), Interval Estimation for the Unbalanced Case of the One-Way Random Effects Model, Annals of Statistics, 3. Burdick, R. and Graybill, F. (1992), Confidence Intervals on Variance Components, Statistics: textbooks and monographs, Vol. 127., Edited by Owen, D.B., Marcel Dekker, Inc., New York, NY. Faulkenberry, G. and Weeks, D. (1968), Sample Size Determination for Tolerance Limits, Technometrics, Vol. 12. Quesenberry, C. (1993), The Effect of Sample Size on Estimated Limits for X and X Control Charts, Journal of Qual. Tech., Vol. 25, No. 4. Burr, I. (1976), Statistical Quality Control Methods, Vol. 16 of the series: Statistics: textbooks and monographs, Owen, D. (ed)., Marcel Dekker, New York, N.Y.

54. Suggested Readings Satterthwaite, F.E. (1946) �An Approximate Distribution of Estimates of Variance Components,� Biometrics Bulletin, 2, 110-114. Mandel, J. (1984) �Fitting Straight Lines When Both Variables are Subject to Error,� J. of Quality Technology, vol. 16, no. 1, 1 � 14. Allen, Paul V., Dukes, Gary R., and Gerger, Mark E. (1991), Determination of Release Limits: A General Methodology, Pharm. Res., 8(9), 1210 - 1213. Wei, Greg C. G. (1998), Simple Methods for Determination of the Release Limits for Drug Products, J. of Biopharmaceutical Stat., 8(1), 103-114. Graybill, F.A. (1976) Theory and Application of the Linear Model, North Situate MA: Duxbury Press

55. Back-up Slides

56. Action Limits To illustrate the difference between the na�ve interval and a tolerance interval, a simulation study was performed as follows: 10000 independent samples of size n = 5 and 25 were drawn from a normal population with mean = 100 and standard deviation = 10. (5000 samples of size n = 120.) The na�ve interval was calculated for each sample. A [95/95] tolerance interval was calculated for each sample for n = 5; the [95/95] tolerance interval was calculated for each sample for n = 25.

57. Action Limits

58. Action Limits

59. Action Limits

60. Action Limits

61. Action Limits

62. Action Limits Normality is critical assumption Distribution-free tolerance intervals can be used if assumption violated Based on order statistics Generally require quite large sample sizes to achieve typical coverage and confidence levels Hahn and Meeker (1991)

63. Action Limits When one analytical method is to be replaced by another, it is often necessary to translate the existing action limits into action limits for the new method. Often there is a linear relationship between the two methods. Taking this into account is necessary in the derivation of the new action limits.

64. Action Limits In many cases, the linear relationship has to be estimated using errors-in-variables regression. Mandel (1984) When the �x� method has �small� measurement error relative to the �y� method or when Berkson�s condition holds, then usual linear regression techniques can be used.

65. Action Limits

66. Action Limits

67. Action Limits

68. Action Limits When significant measurement error variability exists in both methods, a reasonable approach would be to use the estimates of a, � and se from the errors-in-variables regression in the derivation given on the previous slide.