A Bayesian Approach to Parallelism Testing in Bioassay

1. A Bayesian Approach toParallelism Testing in Bioassay Steven Novick, GlaxoSmithKline Harry Yang, MedImmune LLC

2. Manuscript co-author John Peterson, Director of statistics, GlaxoSmithKline

4. Warm up exercise

6. When are two lines parallel? • Parallel: Being everywhere equidistant and not intersecting • Slope • Horizontal shift places one line on top of the other.

7. When are two curves parallel? • Parallel: Being everywhere equidistant and not intersecting? • Can you tell by checking model parameters? • Horizontal shift places one curve on top of the other.

8. Where is parallelism important? • Gottschalk and Dunn (2005) • Determine if biological response(s) to two substances are similar • Determine if two different biological environments will give similar dose–response curves to the same substance. • Compound screening • Assay development / optimization • Bioassay standard curve

9. Screening for compound similar to “gold-standard” • E.g., seeking new HIV compound with AZT-like efficacy, but different viral-mutation profile. • Desire for dose-response curves to be parallel.

10. Change to assay procedure • E.g., change from fresh to frozen cells. • Want to provide same assay signal window. • Desire for control curves to be parallel. Fresh Frozen Day 0 Day 5 Day 10

11. Assess validity of bioassayused for relative potency • Dilution must be parallel to original. • Callahan and Sajjadi (2003) Original Dilution

12. Replacing biological materials used in standard curve • E.g., ELISA (enzyme-linked immunosorbent) assay to measure protein expression. • Recombinant proteins used to make standard curve. • Testing clinical sample • New lot of recombinant proteins for standards. • Check curves are parallel • Calibrate new curve to match the old curve.

13. Potency often determined relative to a reference standard such as ratio of EC50 • Only meaningful if test sample behaves as a dilution or concentration of reference standard • Testing parallelism is required by revised USP Chapter <111> and European Pharmacopeia

14. Linear: Two lots of Protein “A” Estimated Concentrations Lot 2 is 1.4-fold higher than Lot 1 Log10 Signal =0.14

15. If the lines are parallel… • Shift “Lot 2” line to the left by a calibration constant . •  is log relative potency of Lot 2. Draft USP <1034> 2010

16. Testing for Parallelism in bioassay

17. Typical experimental design Serial dilutions of each lot Several replicates Fit on single plate (no plate effect) Log10 Signal

18. Tests for parallel curvesLinear model • Hauck et al, 2005; Gottschalk and Dunn, 2005 H0: b1 = b2 H1: b1 ≠ b2 ANOVA: T-test F goodness of fit test 2goodness of fit test May lack power with small sample size Might be too powerful for large sample size

19. A better idea • Callahan and Sajjadi 2003; Hauck et al. 2005 • Slopes are equivalent • H0: | b1 − b2|≥  • H1: | b1 − b2|< 

20. Nonlinear: Two lots of protein “B” Estimated Concentrations Lot 2 is 1.6-fold higher than Lot 1 =0.21

21. Tests for parallel curves4-parameter logistic (FPL) model • Jonkman and Sidik 2009 • F-test goodness of fit statistic • H0: A1 = A2andB1 = B2andD1 = D2 • H1: At least one parameter not equal May lack power with small sample size Might be too powerful for large sample size

22. Calahan and Sajjadi 2003; Hauck et al. 2005; Jonkman and Sidik (2009) • Equivalence test for each parameter = intersection-union test • H0: |1 / 2| 1or|1 / 2| 2 • H1: 1 <|1 / 2| < 2 i = Ai, Bi, Di ,i=1,2 Equiv. of params does not provide assurance of parallelism (except for linear) May lack power with small sample size Forces a hyper-rectangular acceptance region

23. Our proposal “Parallel Equivalence”

24. Definition of Parallel • Two curves and are parallel if there exists a real number ρ such that for all x.

25. Definition of Parallel Equivalence • Two curves and are parallel equivalent if there exists a real number ρ such that for all x  [xL, xU].

26. It follows that two curves are parallel equivalent if there exists a real number ρ such that • It also follows that

27. Are these two lines parallel enough when xL < x < xU ? < ?

28. Linear-model solution • Linear model: Just check the endpoints

29. Parallel equivalence = slope equivalence wlog Same as testing: | b1 − b2|< 

30. Parallel Equivalence • FPL model: • No closed-form solution. • Simple two-dimensional minimax procedure.

31. Are these two curves parallel enough when xL < x < xU ? < ?

32. Testing for parallel equivalence H0: H1: • Proposed metric (Bayesian posterior probability):

33. Computing the Bayesian posterior probability • For each curve, assume Data distribution: i =1, 2 = reference or sample j = 1, 2, …, N = observations Prior distribution: Posterior distribution proportional to:

34. Draw a random sample of the i of size K from the posterior distribution (e.g., using WinBugs). • The posterior probability is estimated by the proportion (out of K) that the posterior distribution of:

35. = 0.14 (100.14=1.4-fold shift) • = 0.07  90% probability to call parallel equivalent

36. = 0.33 (100.33=2.15-fold shift) • = 0.52  90% probability to call parallel equivalent

37. Simulation: FPL ModelBased on protein “B” data

38. Simulation: FPL model • Similar to Protein “B” protein-chip data. • Concentrations (9-point curve + 0): • 0, 102(=100), 102.5625, 103.125, …, 106.5(=3,200,000) • Three replicates • xL = 3.5=log10(3162) & xU = 5=log10(100,000) • δ = 0.2. •  = 0.02, 0.04, 0.11, and 0.21 (%CV = 5, 10, 25, 50) • For each Monte Carlo run, I computed:

39. 5,000 Monte Carlo Replicates

40. Example data (CV=10%) Diff: 0 0 0.15 0.15 0.20 0.30

41. Diff: 0 0 0.15 0.15 0.20 0.30

43. Summary • Straight-forward and simple test method to assess parallelism. • Yields the log-relative potency factor. • Easily extended.

44. Extensions • Instead of f(, x), could use • f(, x) /  x = instantaneous slope • f-1(, y) = estimated concentrations

45. What’s next? • Head-to-head comparison with existing methods • Choosing test level and , possibly based on ROC curve? – Harry Yang paper • Guidance for prior distribution of 1 and 2.

46. References • Callahan, J. D. and Sajjadi, N. C. (2003), “Testing the Null Hypothesis for a Specified Difference - The Right Way to Test for Parallelism”, Bioprocessing Journal Mar/Apr 1-6. • Gottschalk P.J. and Dunn J.R. (2005), “Measuring Parallelism, Linearity, and Relative Potency in Bioassay and Immunoassay Data”, Journal of Biopharmaceutical Statistics, 15: 3, 437 - 463. • Hauck W.W., Capen R.C., Callahan J.D., Muth J.E.D., Hsu H., Lansky D., Sajjadi N.C., Seaver S.S., Singer R.R. and Weisman D. (2005), “Assessing parallelism prior to determining relative potency”, Journal of pharmaceutical science and technology, 59, 127-137. • Jonkman J and Sidik K (2009), “Equivalence Testing for Parallelism in the Four- Parameter Logistic Model”, Journal of Biopharmaceutical Statistics, 19: 5, 818 - 837.

47. Thank you! Questions?