1 / 62

Peter Gilbert Vaccine Infectious Disease Institute

Evaluating an Immunological Surrogate of Protection. Peter Gilbert Vaccine Infectious Disease Institute Fred Hutchinson Cancer Research Center and University of Washington December 14, 2009. Prototype Preventive Vaccine Efficacy Trial. Primary Objective

hop
Download Presentation

Peter Gilbert Vaccine Infectious Disease Institute

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluating an Immunological Surrogate of Protection Peter Gilbert Vaccine Infectious Disease Institute Fred Hutchinson Cancer Research Center and University of Washington December 14, 2009

  2. Prototype Preventive Vaccine Efficacy Trial • Primary Objective • Assess VE: Vaccine Efficacy to prevent pathogen-specific disease • Secondary Objective • Assess vaccine-induced immune responses as “correlates of protection” Randomize Vaccine Placebo Receive inoculations Measure immune response Follow for clinical endpoint (pathogen-specific disease)

  3. Importance of an Immune Correlate • Finding an immune correlate is a central goal of vaccine research • One of the 14 ‘Grand Challenges of Global Health’ of the NIH & Gates Foundation (for HIV, TB, Malaria) • Immune correlates useful for: • Affording insight into mechanisms of vaccine-protection • Guiding iterative development of vaccines between basic and clinical research • Shortening trials and reducing costs • Guiding regulatory decisions • Guiding immunization policy • Bridging efficacy of a vaccine observed in an efficacy trial to a new setting

  4. Considerable Uncertainty About Immune Correlates for Most Licensed Vaccines

  5. But─ What Exactly is an Immune Correlate? • Normative for the scientific literature to not define what is meant • Confusion in the meaning of immune correlate (or “correlate of protection”, “correlate of protective immunity”) • This talk will: • Clarify 3 distinct concepts of “immune correlate,” with nomenclature and definitions • Summarize a general framework for evaluating the 3 levels of immune correlates • Suggest how lab investigations and clinical trials can be designed to improve the development of immune correlates

  6. Take Home Points • Consider adopting nomenclature and definitions proposed in Qin, Gilbert, McElrath, Corey, Self (2007, JID) • “correlate of risk”, “specific surrogate of protection”, “general surrogate of protection” • Jerald Sadoff and Janet Wittes “encourage researchers in vaccinology to adopt this useful heirarchy” (2007, JID) • Opportunity to improve immune correlates assessment by augmenting standard efficacy trial designs • Baseline Immunogenicity Predictors: Measure participant characteristics that predict the immune responses of interest • Closeout Placebo Vaccination: Vaccinate placebo recipients at the end of follow-up and measure their immune responses to vaccine

  7. Three Tiers of Immune Correlates Evaluation* *Proposed in Qin, Gilbert, McElrath, Corey, Self (2007, JID)

  8. Outline • Introduction to Three Tiers of Surrogate Endpoint Evaluation • Tier 1: Correlate of Risk (CoR) • Tier 2: Specific Surrogate of Protection (Specific SoP) • Statistical Surrogate (Prentice, 1989) • Principal Surrogate (Frangakis and Rubin, 2002) • Tier 3: General Surrogate of Protection (General SoP) • Summary and Conclusions

  9. Tier 1: “Correlate of Risk” (CoR) • Definition: A correlate of risk (CoR) is an immunologic measurement that predicts the rate of the clinical endpoint in some population • Example: Individuals with higher antibody titers have a lower rate of pathogen-specific disease • In an observational study • In the vaccine group of a Phase III trial • In the placebo group of a Phase III trial

  10. Example 1. U.S. VaxGen Phase III Trial: CD4 Blocking Level a CoR for HIV Infection in Vaccine Group* In vaccine group assess CD4 blocking level as a predictor of HIV infection (Cox model) Antibody data measured on 239 infected/163 uninfected p = .026 *Gilbert et al., 2005,JID

  11. Example 2. 1943 Influenza Vaccine Field Trial (Salk, Menke, and Francis, 1945) • N = 1,776 men in an Army Service Unit at the Univ. of Michigan • Evaluated a trivalent vaccine containing Weiss Strain A and PR8 Strain A • Men assigned vaccine or placebo based on alphabetical ordering of names • Inoculations completed in 7 days • Strain-specific Ab titers to Weiss Strain A and to PR8 Strain A measured 2 weeks post-inoculation • Every 10th vaccinee and 5th placebo • Clinical endpoint = Hospitalization due to respiratory illness with a specific influenza strain found in throat culture • Follow-up: Oct 25 1943 to Jan 1 1944

  12. Example 2. 1943 Influenza Vaccine Field Trial Distributions of Weiss Strain A Log Ab Titer

  13. Example 2. Ab Titer a CoR for Weiss Strain A-Specific Hospitalization (Similarly for PR8 Strain A) • CoR Estimation • Inverse correlation between Weiss strain A Ab titers and clinical risk in the placebo group and in the vaccine group

  14. Key Consideration for Evaluating a CoR (1) • Two-phase designs are typically used • Phase I: Stratify all vaccinees by infection status and covariate strata S • Phase II: Take a random sample of vaccinees within each cell, and measure the immune response of interest • There are many ways to do the sampling and the analysis • Most published case-control studies use an approach that ignores valuable information in the data • There are approaches that leverage all the information (and are not overly complicated)- important to implement these Covariate level 1 2 3 4 5 6 Infected Not infected

  15. Key Consideration for Evaluating a CoR (1) • Optimizing the two-phase design • Recommend to use an approximately optimal analysis method* that leverages information in subject characteristics predictive of the immune response and of the clinical endpoint • Provides tighter confidence intervals and greater power than methods in standard use • This analysis method can be used in combination with intelligent/efficient sampling design • Sample all cases • Over-sample controls with unusual immune responses • Under-sample controls for which, based on their characteristics, the anticipated variability of the immune response is low *Breslow et al. (2009, American Journal of Epidemiology)

  16. Planning a Two-Phase CoR Study for the RV144 Thailand Trial Estimated VE = 31%; 95% CI 1% to 52%; p=0.04 • Collaborative effort underway to evaluate a suite of immunological measurements as potential CoRs of HIV infection rate in the vaccine arm

  17. Key Consideration for Evaluating a CoR (2) • Understand the components of variability of the immunological measurement • Biological variability between and within subject; technical measurement error of the assay • Protection-irrelevant variability in the immunological measurement attenuates statistical power

  18. Power to Assess a CoR with 5 Controls per Case in RV144 (195 Controls, 39 Cases)

  19. Outline • Introduction: Three Tiers of Surrogate Endpoint Evaluation • Tier 1: Correlate of Risk (CoR) • Tier 2: Specific Surrogate of Protection (Specific SoP) • Statistical Surrogate (Prentice, 1989) • Principal Surrogate (Frangakis and Rubin, 2002) • Tier 3: General Surrogate of Protection (Principal SoP) • Summary and Conclusions

  20. Definition: A specific surrogate of protection (specific SoP) is an immunological correlate of risk such that vaccine effects on it reliably predict vaccine efficacy, for the same settingas the efficacy trial E.g., groups with no difference in immune response vaccine vs placebo have VE = 0; and groups with the greatest difference have the largest VE > 0 2 detailed definitions of a specific SoP- next slides A specific SoP can be used to reliably predict VE for identical or similar settings as the vaccine trial Tier 2: Specific SoP

  21. An immune response can be a strong CoR but fail totally to be a specific SoP VaxGen Trial: Example of a CoR worthless for predicting VE CD4 blocking level a “mere marker” of infection risk Vaccine recipients with lowest (highest) antibody titers had immune systems less (more) able to naturally ward of infection Follow-up studies that validate this explanation: Vaccine recipient sera did not neutralize 28 primary HIV-1 isolates sampled from VaxGen infected subjects (Montefiori et al., submitted) No observed “sieve effect” (no HIV sequence differences between infected vaccine and infected placebo) A CoR May Not Be a Specific SoP

  22. A Valid Surrogate Fully Mediates the Vaccine Effect on the Clinical Endpoint Vaccination Surrogate True Clinical Endpoint Outcome Infection/Disease • Assurance about surrogate validity requires a comprehensive understanding of: • The biological processes leading to the clinical endpoint • The effects of vaccination on the biomarker and the clinical endpoint

  23. Reason for Surrogate Failure: Protection-Irrelevant Assay Variability • Suppose S* is a valid surrogate endpoint, but the assay measures a noisy version of S*, S • S may be unreliable as a surrogate endpoint • Key Point: • Limitations of the immunological assay an important cause for surrogate failure • Defining assay components of variability a key part of developing SoPs

  24. Validating a Surrogate of Protection a Hard Problem “But the parts of the world are all so related and linked together that I think it is impossible to know one without the other and without the whole.” −Blaise Pascal [1670, The Pensees, Lafuma Edition, Number 199]

  25. Two Definitions of a Specific SoP • Statistical SoP • Prentice (1989, Statistics in Medicine) definition of a surrogate endpoint • Based on observed associations • Principal SoP • Builds on Frangakis and Rubin’s (2002, Biometrics) definition of a surrogate endpoint • Based on potential outcomes framework for causal inference

  26. Prentice (1989, Stats Med) Definition and Operational Criteria for a Statistical SoP • Definition: A statistical SoP is an immunologic measurement satisfying: • Vaccination impacts the immunological marker • The immunological marker is a CoR in each of the vaccine and placebo groups • The relationship between the immunological marker and the clinical endpoint rate is the same in the vaccine and placebo groups • I.e., after accounting for the marker, vaccine/placebo assignment contains no information about clinical risk • Interpretation: All of the vaccine effect on the clinical endpoint is mediated through the marker

  27. Example 2.1943 Influenza Vaccine Field Trial (Salk, Menke, and Francis, 1945) • 98% retention of study participants • Incidence of Hospitalization with Weiss Strain A • Placebos: 75 of 888 (8.45%) • Vaccinees: 20 of 888 (2.25%) Estimated VE = (1 – 2.25/8.45)×100% = 73%

  28. Evaluate Weiss Strain A Ab Titer as a Specific SoP • Check Criteria 1 and 2 • Log Weiss Strain A Ab titers significantly greater in vaccine group than placebo group • Log Weiss Strain A Ab titer strongly inversely correlated with case rate in each study group [shown earlier] • Criteria 1 and 2 hold • Check Criterion 3 • Assess consistency between CoR models in the vaccine and placebo groups

  29. Criterion 3 Holds: Relationship Between Log Ab Titer and Influenza Risk the Same in the Vaccine and Placebo Groups Weiss Strain A

  30. What about PR8 Strain A Ab titers? Evaluate them as a CoR and a SoP for hospitalization with PR8 Strain A-specific influenza infection

  31. Incidence of Hospitalization with PR8 Strain A Placebos: 73 of 888 (8.22%) Vaccinees: 20 of 888 (2.25%) Estimated VE = (1 – 2.25/8.22)×100% = 73% (incidentally the same Estimated VE as for Weiss Strain A) VE for PR8 Strain A

  32. Criterion 3 Fails: Relationship Between Log Ab Titer and Influenza Risk Different in the Vaccine and Placebo Groups

  33. Why Are PR8 Strain A Titers an Imperfect SoP? • Protection from PR8 Strain A only partly described by PR8 Ab titer • A possible explanation is that antibodies are protective, but the measurements reflect something else besides protective responses (e.g., measurement error)

  34. Why Are PR8 Strain A Titers an Imperfect SoP? • Protection from PR8 Strain A only partly described by PR8 Ab titer • Another possible explanation is that there are other protective immune responses that were not measured • E.g., cell-mediated immune responses • Another is that PR8 Strain A has different protective determinants than Weiss Strain A • Yet another is that PR8 Ab titer is a valid SoP, but there was residual confounding in the regression assessment (next)

  35. Challenge with the Statistical SoPApproach (1) • For efficacy trials of pathogen-naïve participants, the immune response of interest will be “non-response”/zero for (almost) all placebo recipients • No variation of the marker in the placebo arm implies: • CoR model in placebo group cannot be evaluated (Criterion 2) • Conceptually difficult to check “full mediation” (Criterion 3)

  36. Challenge with the Statistical SoPApproach (2) • Checking “full mediation” entails checking, for each immune response level s, equal clinical risk between the groups {Vaccinees w/ marker level s} vs {Placebos w/ marker level s} • However, S is measured after randomization • Therefore this comparison is subject to confounding, potentially misleading about the biomarker’s value as a SoP

  37. Assumptions Needed for the Statistical SoP Approach to be Valid • Assumes all subject characteristics predictive of both the biomarker and the clinical endpoint are included in the regression model • Unverifiable assumption, similar to “no unmeasured confounders” • Need biological understanding to make the assumption plausible • Assumes all subject characteristics predictive of the clinical endpoint prior to and after the measurement of the biomarker are included in the regression model • Must include all clinical risk factors in the regression model

  38. Alternative Approach to Evaluating a Specific SoP These limitations motivate research into an alternative approach to surrogate endpoint evaluation Principal surrogate framework which leverages augmented data collection from vaccine efficacy trials

  39. Outline • Introduction: Three Tiers of Surrogate Endpoint Evaluation • Tier 1: Correlate of Risk (CoR) • Tier 2: Specific Surrogate of Protection (Specific SoP) • Statistical Surrogate (Prentice, 1989) • Principal Surrogate (Frangakis and Rubin, 2002) • Tier 3: General Surrogate of Protection (Principal SoP) • Summary and Conclusions

  40. Definition of a Principal SoP • Consider the case where all placebo recipients have S = 0 • Define VE(s) = 1 – • Interpretation:Percent reduction in clinical risk for groups of vaccinees with Ab titer s compared to if they had not been vaccinated • Definition: A Principal SoP is an immunologic measurement satisfying • VE(negative response s = 0) = 0 [Necessity] • VE(high enough response s) > 0 [Sufficiency] Risk of infection for Vaccinees with Ab titer s to Vaccine Risk of infection for Placebos with Ab titer s to Vaccine

  41. The Principal SoP Framework Provides a Way to Compare “Surrogate Value” of Different Markers to Predict VE VE(s) s Immunological measurement percentile v

  42. Challenge to Evaluating a Principal SoP: The Immune Responses to Vaccine are Missing for Placebos • Accurately filling in the unknown immune responses is needed to evaluate a principal SoP • Approaches to filling in the missing data (suggested by Dean Follmann, 2006, Biometrics): • Baseline immunogenicity predictors (BIP) • Close-out placebo vaccination (CPV) • Literature on statistical methods for evaluating a principal SoP • Follmann (2006, Biometrics)* Gilbert and Hudgens (2008, Biometrics)* • Joffe and Greene (2008, Biometrics) Qin, Gilbert, Follmann, Li (2008, AOAS)* • Gilbert, Qin, Self (2009a, b, Stats Med)* Gallop, Small, et al. (2009, Stats Med) • Li, Taylor, and Elliott (2009, Biometrics) Wolfson and Gilbert (in press, Biometrics)* • Hudgens and Gilbert (in press, Biometrics)* *Method uses BIP and/or CPV

  43. Schematic of Closeout Placebo Vaccination (CPV) & Baseline Immunogenicity Predictor (BIP) Trial Designs* S=S(1) Vx  - X BIP Approach 1 1 +  CPV Approach S(1) Vx Sc  - 1 1 +  X BIP Approach *Proposed by Follmann (2006, Biometrics)

  44. Baseline Immunogenicity Predictor (BIP) Approach Evaluate correlation of Xand S in vaccine group X Control Predict S(1) from vaccine group model and Xin controls X

  45. Application of CPV & BIP to Thai Trial RV144 • In discussions with Thai trial leadership to implement CPV approach • Vaccinate ~400 uninfected placebo recipients • Will enable application of CPV and BIP methods to evaluate various antibody measurements and T cell measurements as specific SoPs

  46. Illustration with 1943 Influenza Trial: BIP Approach • S = log Ab titer to Weiss strain A 2 weeks post-vaccination • X = Same measurement at baseline • X and S are correlated (natural immune response predicts vaccine-induced immune response) • Would like to evaluate S as a specific SoP using X as the BIP

  47. Illustration with 1943 Influenza Trial: BIP Approach S = log Ab titer to Weiss strain A 2 weeks post-vaccination X = Same measurement at baseline X and S are correlated (natural immune response predicts vaccine-induced immune response) Would like to evaluate S as a specific SoP using X as the BIP Unfortunately, we do not have the data on X Consider a “poorer analysis” using the fact that S is measured for subjects in both the vaccine and control groups

  48. Illustration with 1943 Influenza Trial: BIP Approach S = log Ab titer to Weiss strain A 2 weeks post-vaccination Inverse ranking approach to filling in S for placebos Assume any two placebos with log Ab titers s1Plac < s2Plac have s1 > s2 This assumption allows construction of a complete dataset of S’s for all subjects for whom the Ab titer was measured Assume Inverse Ranking* *Supported by studies including Gorse et al. (2004, JID)

  49. Estimation of VE(s) • For each s = {32, 128, 256, 512, 1024, 2048, 4096, 8192} (the observed values for vaccinees) estimate VE(s) by Est. VE(s) = 1 - • Will also estimate the case rates by fitted values from logistic regression models Case Rate for Vaccinees with Ab titer s Case Rate for Placebos with imputed Ab titer s

  50. Ab Titers Have Excellent Principal Surrogate Value

More Related