Comparison of Analytical Procedures in Method Transfer and Bridging Experiments.

Comparison of two analytical procedures is the primary objective of a method transfer or when replacing an old procedure with a new one in a single lab. Guidance for comparing two analytical procedures is provided in USP <1010> based on separate tests for accuracy and precision. Determination of criteria is somewhat problematic for these comparisons because of the interdependence of accuracy and precision. In this paper, a total error approach is proposed that requires a single criterion based on an allowable out-of-specification (OOS) rate at the receiving lab. This approach overcomes the difficulty of allocating acceptance criteria between precision and bias. Computations can be performed with any simulation software. Numerical examples are provided for four experimental designs that are typical in a method transfer study. Finally, recommendations are provided to help the user set criteria that provide an acceptable probability of passing for practical sample sizes.

The Goldilocks Challenge-Controlling Uncertainty When Setting Product Specifications.

Product specifications are ideally based on knowledge of patient needs or requirements of subsequent manufacturing steps. However, in most applications, knowledge of patient needs is neither precise nor comprehensive enough to fully define specifications. The prevailing practice is to base specifications on process experience, setting limits to assure consistency of future results with initial results representative of clinical material. Developers of new medicines are often required to set initial product specifications and other limits when only small amounts of process experience have been accumulated. Product developers and health authority reviewers share the mandate to protect patients from harm and assure the effectiveness of medical products, which motivates a tendency to set limits very tight. But although tighter limits give the impression of tighter control, limits alone accomplish no reduction in the variation that exists in established processes and test methods. Limits that are too tight do not represent the natural variability of the process and test methods. Unnaturally tight limits will result in a high number of excursions beyond the limits, potentially causing discards, supply disruptions, and higher cost of goods sold. In this article, we demonstrate how to deliberately control the probability of having intervals that are too tight during the early manufacturing process.

PDA Biosimilars Workshop Report (September 27-28, 2018)-Getting It Right the First Time for Biosimilar Marketing Applications.

This workshop report summarizes the presentations, the breakout session outcomes, and the speaker panel discussions from the PDA Biosimilars Workshop held September 27-28, 2018, in Washington, DC. This format was deliberately selected for the workshop with the expectation of delivering a post-workshop paper on current best practices and existing challenges for sponsors. The event, co-chaired by Dr. Stephan Krause (AstraZeneca Biologics) and Dr. Emanuela Lacana (CDER/FDA), was attended by 140 agency and industry representatives. The workshop was separated into three major sessions P1: Regulatory Perspective, P2: Challenges in Biosimilar Development, and P3: Demonstrating Analytical Similarity. Each of the three sessions started with agency and industry presentations. Participants then split into two concurrent roundtable discussion groups to hear the answers to questions that had been provided to all participants one week prior to the event. The sessions were recorded. This paper provides consolidated answers to specific case studies for current challenges to sponsors and agencies. In addition, the panel discussion notes following each breakout roundtable session, as well as brief talk summaries of all speakers, are provided. The first session explored the challenges encountered with submission of biosimilar marketing applications from the perspectives of regulatory agencies. Expectations for a successful submission of the chemistry, manufacturing, and controls (CMC) information were described. The second session addressed high-level technical challenges and how to avoid pitfalls frequently encountered during biosimilar candidate development, including data quality expectations, creation of the final control strategy, and strategic choices necessary for candidate selection and development. Both regulatory perspectives and industry experience were shared. The last session explored the use of statistical tools to provide meaningful contributions to the demonstration of analytical similarity. The presentations highlighted common issues and practical challenges that arise during the application of statistical tools.LAY ABSTRACT: Significant challenges are still-remaining for sponsors and agencies to successfully develop and license Biosimilars. A Biosimilars Workshop was therefore held on 27-28 September 2018 in Washington, DC, to find practical solutions to the remaining challenges. The workshop planning committee with members from industry and agencies prepared specific case studies focused on some of most difficult situations. The workshop was separated into three major sessions (P1 - Regulatory Perspective; P2 - Challenges in Biosimilar Development; P3 - Demonstrating Analytical Similarity) and each session attempted to provide practical solutions to the relevant case studies. This first session explored the challenges encountered with submission of biosimilar marketing applications from the regulatory agencies' perspectives. Expectations for a successful submission of the CMC information were described. The second session addressed high-level technical challenges frequently encountered during biosimilar candidate development, including data quality expectations, the creation of the final control strategy, and strategic choices necessary for candidate selection and development. The last session explored the use of statistical tools to provide meaningful contributions to the demonstration of analytical similarity and practical challenges that arise during the application of statistical tools.

Simple Approach to Calculate Random Effects Model Tolerance Intervals to Set Release and Shelf-Life Specification Limits of Pharmaceutical Products.

Tolerance intervals are used to statistically derive the acceptance limits to which drugs must conform upon manufacture (release) and throughout shelf-life. The single measurement per lot in release data and repeated measurements per lot longitudinally for stability data have to be considered in the calculation. Methods for the one-way random effects model by Hoffman and Kringle (HK) for two-sided intervals and Hoffman (H) for one-sided limits are extended to a random-intercept, fixed-slope model in this paper. The performance of HK and H was evaluated via simulation by varying the following factors: (a) magnitude of stability trend over time, (b) sample size, (c) percentage of lot-to-lot contribution to total variation, (d) targeted proportion, and (e) data inclusion. The performance metrics are average width (for two-sided) or average limit (for one-sided) and attained confidence level. HK and H maintained nominal confidence levels as originally developed, but H is too conservative (i.e., achieved confidence level exceeds the nominal level) in some situations. The HK method adapted for an attribute that changes over time performed comparably to the more computationally intensive generalized pivotal quantity and Bayesian posterior predictive methods. Mathematical formulas and example calculations as implemented using R statistical software functions are provided to assist practitioners in implementing the methods. The calculations for the proposed approach can also be easily performed in a spreadsheet given basic regression output from a statistical software package. Microsoft Excel spreadsheets are available from the authors upon request.LAY ABSTRACT: Tolerance intervals (a measure of what can be expected from the manufacturing process) calculated from attribute measurements of drug product lots are one of the factors considered when establishing acceptance limits to ensure drug product quality. The methods often used to calculate tolerance intervals when there are multiple measurements per lot and the attribute changes over time are either lacking in statistical rigor or statistically rigorous but computationally intensive to implement. The latter type requires simulations that have to be programmed using specialized statistical software, because closed-form mathematical formulas are not available. As a consequence, some quality practitioners and applied statisticians involved in setting acceptance limits may be hindered in using such computationally intensive methods. This paper aims to address this need by proposing an approach that is statistically rigorous yet simple enough to implement using spreadsheets. The approach builds upon previously published works developed for attributes that do not change over time and adapts the cited works for attributes that change over time. The proposed approach is demonstrated to have good statistical properties and compares favorably against the more computationally intensive alternative methods. The paper provides closed-form mathematical formulas, example data, and illustrative calculations as implemented in programmed R functions to facilitate implementation by practitioners. Alternatively, the calculations can be performed without requiring complex programming/simulation using Microsoft Excel spreadsheets that can be requested from the authors.

Precision of the reportable value-Statistical optimization of the number of replicates.

In pharmaceutical analysis, the precision of the reportable value, i.e. the result which is to be compared to the specification limit(s), is relevant for the suitability of the analytical procedure. Using the variance contributions determined in precision studies addressing the levels injection/system precision, repeatability, and intermediate precision, the number of the corresponding replications for analysis/injection, sample preparation, and series/runs can be varied to improve the precision of the mean (reportable) value (Ermer, Agut, J.Chromatogr. A, 1353 (2014) 71-77). However, this calculation will provide only information on the gain for the precision of the calculated reportable value itself. These so-called point estimators have uncertainty associated with them which can be quantified using statistical confidence intervals. Commonly used statistical equations only allow one to calculate confidence intervals for the intermediate precision of the reportable value, which requires that the routine replication strategy must be defined before starting the precision study. In this paper, statistical models are presented that allow optimizing efficiently the replication strategy with respect to the confidence interval of the precision based on the Satterthwaite approximation posterior, i.e. using the results from the precision study without prior knowledge, as for the point estimate. It is further proposed to simplify the model by including only significant variance contributions larger than 20% of the total variation. The advantage of this minimizing the level of nesting is that the upper precision bound will tighten as the level of nesting decreases. This is important as 90% upper confidence bounds are often up to 2 or 3 times the point estimate, even for a larger number of four runs in the precision study. Four models each have been developed both for a 2-fold balanced nested design representing a complete intermediate precision study, and for a 1-fold balanced nested design using injection/system precision from an independent source. An Excel spreadsheet that performs all the calculations in this paper as well as the appropriate model selection is available from the authors. Due to the usually rather low number of series/runs in precision studies, the uncertainty of the reportable value precision is often dominated by the factor runs. For a statistical evaluation of the precision of the reportable value (in case of three precision levels), the authors recommend a minimum of six runs, two preparations per run, and two injections/analyses per preparation, in order to provide sufficient precision of the variance estimates. However, a risk-based approach is recommended for the decision to apply a statistical evaluation of the precision of the reportable value. In case of low patient risk such as for an assay of a well-characterized drug substance with tightly controlled manufacturing and analytical variability dominating the specification range, a point estimator will usually be adequate to demonstrate the suitability of the analytical procedure.

Statistical Approaches to Assess Biosimilarity from Analytical Data.

Protein therapeutics have unique critical quality attributes (CQAs) that define their purity, potency, and safety. The analytical methods used to assess CQAs must be able to distinguish clinically meaningful differences in comparator products, and the most important CQAs should be evaluated with the most statistical rigor. High-risk CQA measurements assess the most important attributes that directly impact the clinical mechanism of action or have known implications for safety, while the moderate- to low-risk characteristics may have a lower direct impact and thereby may have a broader range to establish similarity. Statistical equivalence testing is applied for high-risk CQA measurements to establish the degree of similarity (e.g., highly similar fingerprint, highly similar, or similar) of selected attributes. Notably, some high-risk CQAs (e.g., primary sequence or disulfide bonding) are qualitative (e.g., the same as the originator or not the same) and therefore not amenable to equivalence testing. For biosimilars, an important step is the acquisition of a sufficient number of unique originator drug product lots to measure the variability in the originator drug manufacturing process and provide sufficient statistical power for the analytical data comparisons. Together, these analytical evaluations, along with PK/PD and safety data (immunogenicity), provide the data necessary to determine if the totality of the evidence warrants a designation of biosimilarity and subsequent licensure for marketing in the USA. In this paper, a case study approach is used to provide examples of analytical similarity exercises and the appropriateness of statistical approaches for the example data.

On Statistical Approaches for Demonstrating Analytical Similarity in the Presence of Correlation.

Analytical similarity is the foundation for demonstration of biosimilarity between a proposed product and a reference product. For this assessment, currently the U.S. Food and Drug Administration (FDA) recommends a tiered system in which quality attributes are categorized into three tiers commensurate with their risk and approaches of varying statistical rigor are subsequently used for the three-tier quality attributes. Key to the analyses of Tiers 1 and 2 quality attributes is the establishment of equivalence acceptance criterion and quality range. For particular licensure applications, the FDA has provided advice on statistical methods for demonstration of analytical similarity. For example, for Tier 1 assessment, an equivalence test can be used based on an equivalence margin of 1.5 σR, where σR is the reference product variability estimated by the sample standard deviation SR from a sample of reference lots. The quality range for demonstrating Tier 2 analytical similarity is of the form X̄R ± K × σR where the constant K is appropriately justified. To demonstrate Tier 2 analytical similarity, a large percentage (e.g., 90%) of test product must fall in the quality range. In this paper, through both theoretical derivations and simulations, we show that when the reference drug product lots are correlated, the sample standard deviation SR underestimates the true reference product variability σR As a result, substituting SR for σR in the Tier 1 equivalence acceptance criterion and the Tier 2 quality range inappropriately reduces the statistical power and the ability to declare analytical similarity. Also explored is the impact of correlation among drug product lots on Type I error rate and power. Three methods based on generalized pivotal quantities are introduced, and their performance is compared against a two-one-sided tests (TOST) approach. Finally, strategies to mitigate risk of correlation among the reference products lots are discussed. LAY ABSTRACT: A biosimilar is a generic version of the original biological drug product. A key component of a biosimilar development is the demonstration of analytical similarity between the biosimilar and the reference product. Such demonstration relies on application of statistical methods to establish a similarity margin and appropriate test for equivalence between the two products. This paper discusses statistical issues with demonstration of analytical similarity and provides alternate approaches to potentially mitigate these problems.

Establishment of an equivalence acceptance criterion for accelerated stability studies.

In this article, the use of statistical equivalence testing for providing evidence of process comparability in an accelerated stability study is advocated over the use of a test of differences. The objective of such a study is to demonstrate comparability by showing that the stability profiles under nonrecommended storage conditions of two processes are equivalent. Because it is difficult at accelerated conditions to find a direct link to product specifications, and hence product safety and efficacy, an equivalence acceptance criterion is proposed that is based on the statistical concept of effect size. As with all statistical tests of equivalence, it is important to collect input from appropriate subject-matter experts when defining the acceptance criterion.

Assessment of individual agreements with repeated measurements based on generalized confidence intervals.

Individual agreement between two measurement systems is determined using the total deviation index (TDI) or the coverage probability (CP) criteria as proposed by Lin (2000) and Lin et al. (2002). We used a variance component model as proposed by Choudhary (2007). Using the bootstrap approach, Choudhary (2007), and generalized confidence intervals, we construct bounds on TDI and CP. A simulation study was conducted to assess whether the bounds maintain the stated type I error probability of the test. We also present a computational example to demonstrate the statistical methods described in the paper.

Assessing equivalence of two assays using sensitivity and specificity.

The equivalence of two assays is determined using the sensitivity and specificity relative to a gold standard. The equivalence-testing criterion is based on a misclassification rate proposed by Burdick et al. (2005) and the intersection-union test (IUT) method proposed by Berger (1982). Using a variance components model and IUT methods, we construct bounds for the sensitivity and specificity relative to the gold standard assay based on generalized confidence intervals. We conduct a simulation study to assess whether the bounds maintain the stated test size. We present a computational example to demonstrate the method described in the paper.

Alternative confidence intervals for the assessment of bioequivalence in four-period cross-over designs.

Bioequivalence studies are conducted to demonstrate equivalence in the bioavailability of the active ingredient in different formulations. The U.S. Food and Drug Administration (FDA) requires pharmaceutical companies to show bioequivalence between different formulations or generic companies to show bioequivalence between generic drugs and brand drugs before approval. A recent FDA guidance on bioequivalence proposes criteria for assessment of population bioequivalence (PBE) and individual bioequivalence (IBE) in a four-period cross-over design. In this paper, computer simulation is used to compare modified large sample (MLS) upper bounds with those proposed by the FDA to test for both PBE and IBE. The comparison criteria are the ability to maintain the stated test size and the simulated power of tests based on these bounds.