The public health value of vaccines beyond efficacy: methods, measures and outcomes.

BACKGROUND: Assessments of vaccine efficacy and safety capture only the minimum information needed for regulatory approval, rather than the full public health value of vaccines. Vaccine efficacy provides a measure of proportionate disease reduction, is usually limited to etiologically confirmed disease, and focuses on the direct protection of the vaccinated individual. Herein, we propose a broader scope of methods, measures and outcomes to evaluate the effectiveness and public health impact to be considered for evidence-informed policymaking in both pre- and post-licensure stages. DISCUSSION: Pre-licensure: Regulatory concerns dictate an individually randomised clinical trial. However, some circumstances (such as the West African Ebola epidemic) may require novel designs that could be considered valid for licensure by regulatory agencies. In addition, protocol-defined analytic plans for these studies should include clinical as well as etiologically confirmed endpoints (e.g. all cause hospitalisations, pneumonias, acute gastroenteritis and others as appropriate to the vaccine target), and should include vaccine-preventable disease incidence and 'number needed to vaccinate' as outcomes. Post-licensure: There is a central role for phase IV cluster randomised clinical trials that allows for estimation of population-level vaccine impact, including indirect, total and overall effects. Dynamic models should be prioritised over static models as the constant force of infection assumed in static models will usually underestimate the effectiveness and cost-effectiveness of the immunisation programme by underestimating indirect effects. The economic impact of vaccinations should incorporate health and non-health benefits of vaccination in both the vaccinated and unvaccinated populations, thus allowing for estimation of the net social value of vaccination. CONCLUSIONS: The full benefits of vaccination reach beyond direct prevention of etiologically confirmed disease and often extend across the life course of a vaccinated person, prevent outcomes in the wider community, stabilise health systems, promote health equity, and benefit local and national economies. The degree to which vaccinations provide broad public health benefits is stronger than for other preventive and curative interventions.

The contribution of neighbours to an individual's risk of typhoid outcome.

An individual's risk of infection from an infectious agent can depend on both the individual's own risk and protective factors and those of individuals in the same community. We hypothesize that an individual's exposure to an infectious agent is associated with the risks of infection of those living nearby, whether their risks are modified by pharmaceutical interventions or by other factors, because of the potential for transmission from them. For example, unvaccinated individuals living in a highly vaccinated community can benefit from indirect protection, or living near more children in a typhoid-endemic region (where children are at highest risk) might result in more exposure to typhoid. We tested this hypothesis using data from a cluster-randomized typhoid vaccine trial. We first estimated each individual's relative risk of confirmed typhoid outcome using their vaccination status and age. We defined a new covariate, potential exposure, to be the sum of the relative risks of all who live within 100 m of each person. We found that potential exposure was significantly associated with an individual's typhoid outcome, and adjusting for potential exposure affected estimates of vaccine efficacy. We suggest that it is useful and feasible to adjust for spatially heterogeneous distributions of individual-level risk factors, but further work is required to develop and test such approaches.

Household transmissibility of avian influenza A (H7N9) virus, China, February to May 2013 and October 2013 to March 2014.

To study human-to-human transmissibility of the avian influenza A (H7N9) virus in China, household contact information was collected for 125 index cases during the spring wave (February to May 2013), and for 187 index cases during the winter wave (October 2013 to March 2014). Using a statistical model, we found evidence for human-to-human transmission, but such transmission is not sustainable. Under plausible assumptions about the natural history of disease and the relative transmission frequencies in settings other than household, we estimate the household secondary attack rate (SAR) among humans to be 1.4% (95% CI: 0.8 to 2.3), and the basic reproductive number R0 to be 0.08 (95% CI: 0.05 to 0.13). The estimates range from 1.3% to 2.2% for SAR and from 0.07 to 0.12 for R0 with reasonable changes in the assumptions. There was no significant change in the human-to-human transmissibility of the virus between the two waves, although a minor increase was observed in the winter wave. No sex or age difference in the risk of infection from a human source was found. Human-to-human transmissibility of H7N9 continues to be limited, but it needs to be closely monitored for potential increase via genetic reassortment or mutation.

Assessing the impact of travel restrictions on international spread of the 2014 West African Ebola epidemic.

The quick spread of an Ebola outbreak in West Africa has led a number of countries and airline companies to issue travel bans to the affected areas. Considering data up to 31 Aug 2014, we assess the impact of the resulting traffic reductions with detailed numerical simulations of the international spread of the epidemic. Traffic reductions are shown to delay by only a few weeks the risk that the outbreak extends to new countries.

El Tor cholera with severe disease: a new threat to Asia and beyond.

During epidemics of cholera in two rural sites (Bakerganj and Mathbaria), a much higher proportion of patients came for treatment with severe dehydration than was seen in previous years. V. cholerae O1 isolated from these patients was found to be El Tor in its phenotype, but its cholera toxin (CT) was determined to be that of classical biotype. Whether the observed higher proportion of severe dehydration produced by the El Tor biotype was due to a shift from El Tor to classical CT or due to other factors is not clear. However, if cholera due to strains with increased severity spread to other areas where treatment facilities are limited, there are likely to be many more cholera deaths.

Using validation sets for outcomes and exposure to infection in vaccine field studies.

Methods of adjusting for bias in estimates due to mismeasured or missing covariates and outcomes through the use of validation sets have been developed in many types of health studies. These methods can be employed for the efficient design and analysis of vaccine studies as well. On the one hand, nonspecific case definitions can lead to attenuated efficacy and effectiveness estimates, but confirmation by culture or a quick test of the infectious agent is also expensive and difficult. On the other hand, data on exposure to infection can influence estimates of vaccine efficacy, but good data on exposure are difficult to obtain. In this paper, the authors show how use of small validation sets can correct the bias of the estimates obtained from a large main study while maintaining efficiency. They illustrate the approach for outcomes using the example of influenza vaccine efficacy and effectiveness trials and illustrate the approach for exposure to infection using the example of a human immunodeficiency virus vaccine trial. The authors discuss challenges posed by infectious diseases in the use of currently available methods. Development of these efficient designs and methods of analysis for vaccine field studies will improve estimation of vaccine efficacy for both susceptibility and infectiousness, as well as estimation of indirect and overall effects of vaccination in community trials.

Nonparametric maximum likelihood estimation for competing risks survival data subject to interval censoring and truncation.

We derive the nonparametric maximum likelihood estimate (NPMLE) of the cumulative incidence functions for competing risks survival data subject to interval censoring and truncation. Since the cumulative incidence function NPMLEs give rise to an estimate of the survival distribution which can be undefined over a potentially larger set of regions than the NPMLE of the survival function obtained ignoring failure type, we consider an alternative pseudolikelihood estimator. The methods are then applied to data from a cohort of injecting drug users in Thailand susceptible to infection from HIV-1 subtypes B and E.

Modeling markers of disease progression by a hidden Markov process: application to characterizing CD4 cell decline.

Multistate models have been increasingly used to model natural history of many diseases as well as to characterize the follow-up of patients under varied clinical protocols. This modeling allows describing disease evolution, estimating the transition rates, and evaluating the therapy effects on progression. In many cases, the staging is defined on the basis of a discretization of the values of continuous markers (CD4 cell count for HIV application) that are subject to great variability due mainly to short time-scale noise (intraindividual variability) and measurement errors. This led us to formulate a Bayesian hierarchical model where, at a first level, a disease process (Markov model on the true states, which are unobserved) is introduced and, at a second level, the measurement process making the link between the true states and the observed marker values is modeled. This hierarchical formulation allows joint estimation of the parameters of both processes. Estimation of the quantities of interest is performed via stochastic algorithms of the family of Markov chain Monte Carlo methods. The flexibility of this approach is illustrated by analyzing the CD4 data on HIV patients of the Concorde clinical trial.

Estimation of the efficacy of live, attenuated influenza vaccine from a two-year, multi-center vaccine trial: implications for influenza epidemic control.

The authors provide an analysis of data from a two-year (1996-1998), multicenter (ten US cities), double-blinded, placebo-controlled influenza vaccine trial in children. The vaccine was the trivalent cold-adapted influenza vaccine. Estimates are made of the vaccine efficacy for susceptibility to culture-confirmed influenza (VE(S)) while taking inter-center variability in the risk of infection into account. Our overall estimate of VE(S) against influenza is 0.92 (95% confidence interval (CI) 0.89-0.94). In addition, for the second year, although the vaccine contained antigen for A/Wuhan-like (H3N2), the estimated VE(S) for epidemic variant A/Sydney-like (H3N2) was 0.89 (95% CI 0.81-0.94). Thus, the vaccine showed a high degree of protection against a variant not closely matched to the vaccine antigen. With regard to natural immunity, an influenza A infection in the first year reduces the estimated risk of an influenza A infection in the second year by a factor of 0.88 (95% CI 0.21-0.98). When comparing year 1 to year 2, there is a negative correlation of -0.50 in the center-specific attack rates in the placebo groups. This is consistent with the theory that natural immunity provides overall community protection to children. The authors argue that mass vaccination of 70% of the children with the cold-adapted influenza vaccine could provide substantial protection to the community at large.

Design and interpretation of vaccine field studies.

There are many different effects to consider when evaluating vaccines in the field. In this review, we have covered some of the various measures and issues related to study design and interpretation of the different measures. We emphasize that in designing and understanding vaccine studies, it is necessary to be specific about what the effect of interest is and about the assumptions underlying the interpretation of the results. Halloran et al. (81) present design, analysis, and interpretation of vaccine studies in more detail.

A Markov model for measuring vaccine efficacy for both susceptibility to infection and reduction in infectiousness for prophylactic HIV vaccines.

We use a discrete-time non-homogeneous Markov chain to model data from augmented human immunodeficiency virus (HIV) vaccine trials. For this design, the study population consists of primary participants some of whom have steady sexual partners who are also enrolled to augment the trial. The state space consists of the infection status of primary participants without steady partners and the infection status of both persons in the steady partnerships. The transition probabilities are functions of the two parameters: vaccine efficacy for susceptibility (VES) and infectiousness (VEI). We use likelihood methods to estimate VES and VEI from time-to-event data. We then use stochastic simulations to explore the bias and precision of the estimators under various plausible conditions for HIV vaccine trials. We show that both the VES and VEI are estimable with reasonable precision for the conditions that may exist for planned HIV vaccine trials. We show that exams conducted every six months will likely provide sufficient information to estimate the VE parameters accurately, and that there is little gain in precision for more frequent exams. Finally, we show that joint estimation of the VES and VEI will likely be feasible in a currently planned HIV vaccine trial among injecting drug users in Bangkok, Thailand, if one augments the information about the primary participants in the trial with information about their steady sexual partners.

Efficiency of estimating vaccine efficacy for susceptibility and infectiousness: randomization by individual versus household.

In designing vaccine efficacy studies based on the secondary attack rate (SAR) or transmission probability in which both vaccine efficacy for susceptibility, VE(S), and vaccine efficacy for infectiousness, VE(I), are estimated, the allocation of vaccine and placebo within transmission units has an important influence on the efficiency of the study. We compared the following randomization schemes that result in different allocations of vaccine and placebo within two-member households: (1) randomization by individual for a mixed allocation, (2) randomization by transmission unit for concordant allocation, and (3) randomization of only one individual in each transmission unit to either vaccine or placebo. There is a complex interaction among the VE(S), VE(I), and the SAR that determines which allocation of vaccine and placebo within households provides the most information. In general, individual randomization with a mixed allocation of vaccine and placebo is better for estimating both VE(S) and VE(I) than is randomizing by household. However, for estimation of VE(I), at very low SARs and low VE(S), randomization by household is slightly more efficient than randomization by individual.

Semiparametric methods for multiple exposure mismeasurement and a bivariate outcome in HIV vaccine trials.

Exposure to infection information is important for estimating vaccine efficacy, but it is difficult to collect and prone to missingness and mismeasurement. We discuss study designs that collect detailed exposure information from only a small subset of participants while collecting crude exposure information from all participants and treat estimation of vaccine efficacy in the missing data/measurement error framework. We extend the discordant partner design for HIV vaccine trials of Golm, Halloran, and Longini (1998, Statistics in Medicine, 17, 2335-2352.) to the more complex augmented trial design of Longini, Datta, and Halloran (1996, Journal of Acquired Immune Deficiency Syndromes and Human Retrovirology 13, 440-447) and Datta, Halloran, and Longini (1998, Statistics in Medicine 17, 185-200). The model for this design includes three exposure covariates and both univariate and bivariate outcomes. We adapt recently developed semiparametric missing data methods of Reilly and Pepe (1995, Biometrika 82, 299 314), Carroll and Wand (1991, Journal of the Royal Statistical Society, Series B 53, 573-585), and Pepe and Fleming (1991, Journal of the American Statistical Association 86, 108-113) to the augmented vaccine trial design. We demonstrate with simulated HIV vaccine trial data the improvements in bias and efficiency when combining the different levels of exposure information to estimate vaccine efficacy for reducing both susceptibility and infectiousness. We show that the semiparametric methods estimate both efficacy parameters without bias when the good exposure information is either missing completely at random or missing at random. The pseudolikelihood method of Carroll and Wand (1991) and Pepe and Fleming (1991) was the more efficient of the two semiparametric methods.

Semi-parametric models for mismeasured exposure information in vaccine trials.

Exposure to infection information is important for estimating vaccine efficacy, but it is difficult to collect and inherently prone to missingness and mismeasurement. It is, therefore, generally not feasible to collect good exposure information on all participants in a large vaccine trial. We discuss study designs that collect detailed exposure information for only a small subset of trial participants, while collecting crude exposure information on all participants, and treat estimation of vaccine efficacy in the missing data/measurement error framework. We demonstrate with the example of an HIV vaccine trial the improvements in bias and efficiency when we combine the different levels of exposure information to estimate vaccine efficacy for reducing both susceptibility and infectiousness. We compare the performance of recently developed semi-parametric missing data methods of Pepe and Fleming and Carroll and Wand, Robins, Hsieh and Newey, and Reilly and Pepe.

Optimal vaccine trial design when estimating vaccine efficacy for susceptibility and infectiousness from multiple populations.

Vaccination can have important indirect effects on the spread of an infectious agent by reducing the level of infectiousness of vaccinees who become infected. To estimate the effect of vaccination on infectiousness, one typically requires data on the contacts between susceptible and infected vaccinated and unvaccinated people. As an alternative, we propose a trial design that involves multiple independent and interchangeable populations. By varying the fraction of susceptible people vaccinated across populations, we obtain an estimate of the reduction infectiousness that depends only on incidence data from the vaccine and control groups of the multiple populations. One can also obtain from these data an estimate of the reduction of susceptibility to infection. We propose a vaccination strategy that is a trade-off between optimal estimation of vaccine efficacy for susceptibility and of vaccine efficacy for infectiousness. We show that the optimal choice depends on the anticipated efficacy of the vaccine as well as the basic reproduction number of the underlying infectious disease process. Smaller vaccination fractions appear desirable when vaccine efficacy is likely high and the basic reproduction number is not large. This strategy avoids the potential for too few infections to occur to estimate vaccine efficacy parameters reliably.

Estimation of vaccine efficacy in the presence of waning: application to cholera vaccines.

The authors present a nonparametric method for estimating vaccine efficacy as a smooth function of time from vaccine trials. Use of the method requires a minimum of assumptions. Estimation is based on the smoothed case hazard rate ratio comparing the vaccinated with the unvaccinated. The estimation procedure allows investigators to assess time-varying changes in vaccine-induced protection, such as those produced by waning and boosting. The authors use the method to reanalyze data from a vaccine trial of two cholera vaccines in rural Bangladesh. This analysis reveals the differential protection and waning effects for the vaccines as a function of biotype and age.

PIP: Vaccine efficacy (VE) is typically estimated by the equation VE = 1 minus relative risk (RR), where RR is based on a comparison of vaccinated and unvaccinated groups. However, since vaccine effects do not follow a simplified model such as an exponential decline in protection, estimation of a rate ratio for time-to-event data is difficult. This paper presents a method for nonparametrically estimating VE(t) = 1 - RR(t) from time-to-event data when the protective effects of a vaccine can wane or boost over time. The method, based on smoothing scaled residuals from a proportional hazards model, is then applied to a reanalysis of data from a trial in rural Bangladesh of two cholera vaccines. The placebo and vaccine curves should be roughly parallel for all time if there are no time-varying effects. Application to the data from Bangladesh confirmed this method provides reliable estimation and analysis of field data. The reanalysis revealed the differential protection and waning effects for the vaccines as a function of biotype and age.

Augmented HIV vaccine trial design for estimating reduction in infectiousness and protective efficacy.

It is important to design HIV vaccine trials to estimate the efficacy of a vaccine in reducing infectiousness in addition to the protective efficacy. Currently planned phase III HIV vaccine field trials in which at-risk individuals are randomized and followed over time do not permit estimation or testing of the vaccine's effect on reducing infectiousness of vaccinees who become infected. We suggest an augmentation of these field trials that recruits steady sexual partners of the primary participants into the trial as far as they are willing to participate. This study design would allow estimation of the efficacy of the vaccine on reducing infectiousness as well as the protective efficacy. We compare the classical design that does not include partners to two different types of augmented design. In the first type of augmentation, called the non-randomized partner design, the steady sexual partners are not randomized to vaccine or placebo. In the second type of augmentation, called the randomized partner design, the steady sexual partners are also randomized to vaccine or placebo. We present a probability model based on infection status at the end of the trial that provides maximum likelihood estimates of the protective efficacy of the vaccine, VES, and the efficacy of the vaccine on reducing infectiousness, VEI. Wald statistics are used for one degree of freedom tests on VES and VEI. With the augmented design, a likelihood ratio test is used to test whether the vaccine has any effect at all. The randomized partner design has more power and provides narrower confidence intervals than does the non-randomized partner design.

Study designs for evaluating different efficacy and effectiveness aspects of vaccines.

Vaccine efficacy and effectiveness (VE) are generally measured as 1 minus some measure of relative risk (RR) in the vaccinated group compared with the unvaccinated group (VE = 1 - RR). In designing a study to evaluate vaccination, the type of effect and the question of interest determine the appropriate choice of comparison population and parameter. Possible questions of interest include that of the biologic effect of vaccination on susceptibility, on infectiousness, or on progression to disease in individuals. The indirect effects, total effects, and overall public health benefits of widespread vaccination of individuals within the context of a vaccination program might also be of primary concern. The change in behavior induced by belief in the protective effects of vaccination might influence the estimates of these effects or might itself be of interest. In this paper, the authors present a framework of study designs that relates the scientific question of interest to the choice of comparison groups, the unit of observation, the level of information available for analysis, and the parameter of effect.

Measuring vaccine efficacy for both susceptibility to infection and reduction in infectiousness for prophylactic HIV-1 vaccines.

Current Phase III trials are designed to assess only a vaccine candidate's ability to reduce susceptibility to infection or disease, that is, vaccine efficacy for susceptibility (VES). Human immunodeficiency virus (HIV) vaccination, however, may reduce the level of infectiousness of vaccinees who become infected, producing an important indirect reduction in HIV transmission even if the vaccine confers only modest protection against infection. We propose two approaches for augmenting the information of a classic trial for estimating protective efficacy that enable the additional estimation of the vaccine's effect on infectiousness, that is, vaccine efficacy for infectiousness (VEI). In the first augmentation, steady sexual partners of trial participants are recruited but not randomized to vaccine or placebo. Their infection status is monitored throughout the trial. In the second augmentation, the sexual partners are randomized. Through computer simulations and analytic methods, we investigate the feasibility and statistical properties of the augmented designs. Phase III prophylactic HIV-1 vaccines trials are currently being planned. Employment of the augmented designs described in this paper would not only provide estimation of VEI but also increase the precision of the VES estimator and the power to reject the null hypothesis of no vaccine effect.