Evaluating the Clinical Utility of Epstein-Barr Virus Antibodies as Biomarkers in Multiple Sclerosis: A Systematic Review.

BACKGROUND: EBV is a necessary but not sufficient factor in the pathophysiology of multiple sclerosis (MS). EBV antibodies to the nuclear antigen (EBNA1) and viral capsid antigen (VCA) rise rapidly prior to MS disease manifestations, and their absence has clinical utility with a high negative predictive value. It remains unclear whether EBV levels act as prognostic, monitoring, or pharmacodynamic/response biomarkers. Substantial literature on this topic exists but has not been systematically reviewed. We hypothesized that EBV levels against EBNA1 and VCA are potential prognostic and monitoring biomarkers in MS, and that patient population, MS clinical phenotype, and EBV assay method may play important roles in explaining variation among study outcomes. METHODS: We systematically searched PubMed and EMBASE from inception to April 1, 2022. After removal of duplicates, records were screened by abstract. Remaining full-text articles were reviewed. Clinical and MRI data were extracted from full-text articles for comparison and synthesis. RESULTS: Searches yielded 696 unique results; 285 were reviewed in full, and 36 met criteria for data extraction. Heterogeneity in sample population, clinical outcome measures, assay methods and statistical analyses precluded a meta-analysis. EBV levels were not consistently associated with clinical disease markers including conversion from CIS to RRMS, neurological disability, or disease phenotype. Studies using repeated-measures design suggest that EBNA1 levels may temporarily reflect inflammatory disease activity as assessed by gadolinium-enhancing Magnetic Resonance Imaging (MRI) lesions. Limited data also suggest a decrease in EBV levels following initiation of certain disease-modifying therapies. CONCLUSION: Heterogeneous methodology limited generalization and meta-analysis. EBV antibody levels are unlikely to represent prognostic biomarkers in MS. The areas of highest ongoing promise relate to diagnostic exclusion and pharmacodynamic/disease response. Use of EBV antibodies as biomarkers in clinical practice remains additionally limited by lack of methodological precision, reliability, and validation.

Estimation of heterogeneity variance based on a generalized Q statistic in meta-analysis of log-odds-ratio.

For estimation of heterogeneity variance τ 2 in meta-analysis of log-odds-ratio, we derive new mean- and median-unbiased point estimators and new interval estimators based on a generalized Q statistic, Q F , in which the weights depend on only the studies' effective sample sizes. We compare them with familiar estimators based on the inverse-variance-weights version of Q , Q IV . In an extensive simulation, we studied the bias (including median bias) of the point estimators and the coverage (including left and right coverage error) of the confidence intervals. Most estimators add 0.5 to each cell of the 2 × 2 table when one cell contains a zero count; we include a version that always adds 0.5 . The results show that: two of the new point estimators and two of the familiar point estimators are almost unbiased when the total sample size n ≥ 250 and the probability in the Control arm ( p iC ) is 0.1, and when n ≥ 100 and p iC is 0.2 or 0.5; for 0.1 ≤ τ 2 ≤ 1 , all estimators have negative bias for small to medium sample sizes, but for larger sample sizes some of the new median-unbiased estimators are almost median-unbiased; choices of interval estimators depend on values of parameters, but one of the new estimators is reasonable when p iC = 0.1 and another, when p iC = 0.2 or p iC = 0.5 ; and lack of balance between left and right coverage errors for small n and/or p iC implies that the available approximations for the distributions of Q IV and Q F are accurate only for larger sample sizes.

On the Q statistic with constant weights in meta-analysis of binary outcomes.

BACKGROUND: Cochran's Q statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value (under an incorrect null distribution) is part of several popular estimators of the between-study variance, [Formula: see text]. Those applications generally do not account for use of the studies' estimated variances in the inverse-variance weights that define Q (more explicitly, [Formula: see text]). Importantly, those weights make approximating the distribution of [Formula: see text] rather complicated. METHODS: As an alternative, we are investigating a Q statistic, [Formula: see text], whose constant weights use only the studies' arm-level sample sizes. For log-odds-ratio (LOR), log-relative-risk (LRR), and risk difference (RD) as the measures of effect, we study, by simulation, approximations to distributions of [Formula: see text] and [Formula: see text], as the basis for tests of heterogeneity. RESULTS: The results show that: for LOR and LRR, a two-moment gamma approximation to the distribution of [Formula: see text] works well for small sample sizes, and an approximation based on an algorithm of Farebrother is recommended for larger sample sizes. For RD, the Farebrother approximation works very well, even for small sample sizes. For [Formula: see text], the standard chi-square approximation provides levels that are much too low for LOR and LRR and too high for RD. The Kulinskaya et al. (Res Synth Methods 2:254-70, 2011) approximation for RD and the Kulinskaya and Dollinger (BMC Med Res Methodol 15:49, 2015) approximation for LOR work well for [Formula: see text] but have some convergence issues for very small sample sizes combined with small probabilities. CONCLUSIONS: The performance of the standard [Formula: see text] approximation is inadequate for all three binary effect measures. Instead, we recommend a test of heterogeneity based on [Formula: see text] and provide practical guidelines for choosing an appropriate test at the .05 level for all three effect measures.

Associations Between Natural Language Processing-Enriched Social Determinants of Health and Suicide Death Among US Veterans.

Importance: Social determinants of health (SDOHs) are known to be associated with increased risk of suicidal behaviors, but few studies use SDOHs from unstructured electronic health record notes. Objective: To investigate associations between veterans' death by suicide and recent SDOHs, identified using structured and unstructured data. Design, Setting, and Participants: This nested case-control study included veterans who received care under the US Veterans Health Administration from October 1, 2010, to September 30, 2015. A natural language processing (NLP) system was developed to extract SDOHs from unstructured clinical notes. Structured data yielded 6 SDOHs (ie, social or familial problems, employment or financial problems, housing instability, legal problems, violence, and nonspecific psychosocial needs), NLP on unstructured data yielded 8 SDOHs (social isolation, job or financial insecurity, housing instability, legal problems, barriers to care, violence, transition of care, and food insecurity), and combining them yielded 9 SDOHs. Data were analyzed in May 2022. Exposures: Occurrence of SDOHs over a maximum span of 2 years compared with no occurrence of SDOH. Main Outcomes and Measures: Cases of suicide death were matched with 4 controls on birth year, cohort entry date, sex, and duration of follow-up. Suicide was ascertained by National Death Index, and patients were followed up for up to 2 years after cohort entry with a study end date of September 30, 2015. Adjusted odds ratios (aORs) and 95% CIs were estimated using conditional logistic regression. Results: Of 6 122 785 veterans, 8821 committed suicide during 23 725 382 person-years of follow-up (incidence rate 37.18 per 100 000 person-years). These 8821 veterans were matched with 35 284 control participants. The cohort was mostly male (42 540 [96.45%]) and White (34 930 [79.20%]), with 6227 (14.12%) Black veterans. The mean (SD) age was 58.64 (17.41) years. Across the 5 common SDOHs, NLP-extracted SDOH, on average, retained 49.92% of structured SDOHs and covered 80.03% of all SDOH occurrences. SDOHs, obtained by structured data and/or NLP, were significantly associated with increased risk of suicide. The 3 SDOHs with the largest effect sizes were legal problems (aOR, 2.66; 95% CI, 2.46-2.89), violence (aOR, 2.12; 95% CI, 1.98-2.27), and nonspecific psychosocial needs (aOR, 2.07; 95% CI, 1.92-2.23), when obtained by combining structured data and NLP. Conclusions and Relevance: In this study, NLP-extracted SDOHs, with and without structured SDOHs, were associated with increased risk of suicide among veterans, suggesting the potential utility of NLP in public health studies.

The stratified win statistics (win ratio, win odds, and net benefit).

The win odds and the net benefit are related directly to each other and indirectly, through ties, to the win ratio. These three win statistics test the same null hypothesis of equal win probabilities between two groups. They provide similar p-values and powers, because the Z-values of their statistical tests are approximately equal. Thus, they can complement one another to show the strength of a treatment effect. In this article, we show that the estimated variances of the win statistics are also directly related regardless of ties or indirectly related through ties. Since its introduction in 2018, the stratified win ratio has been applied in designs and analyses of clinical trials, including Phase III and Phase IV studies. This article generalizes the stratified method to the win odds and the net benefit. As a result, the relations of the three win statistics and the approximate equivalence of their statistical tests also hold for the stratified win statistics.

Win statistics (win ratio, win odds, and net benefit) can complement one another to show the strength of the treatment effect on time-to-event outcomes.

Conventional analyses of a composite of multiple time-to-event outcomes use the time to the first event. However, the first event may not be the most important outcome. To address this limitation, generalized pairwise comparisons and win statistics (win ratio, win odds, and net benefit) have become popular and have been applied to clinical trial practice. However, win ratio, win odds, and net benefit have typically been used separately. In this article, we examine the use of these three win statistics jointly for time-to-event outcomes. First, we explain the relation of point estimates and variances among the three win statistics, and the relation between the net benefit and the Mann-Whitney U statistic. Then we explain that the three win statistics are based on the same win proportions, and they test the same null hypothesis of equal win probabilities in two groups. We show theoretically that the Z-values of the corresponding statistical tests are approximately equal; therefore, the three win statistics provide very similar p-values and statistical powers. Finally, using simulation studies and data from a clinical trial, we demonstrate that, when there is no (or little) censoring, the three win statistics can complement one another to show the strength of the treatment effect. However, when the amount of censoring is not small, and without adjustment for censoring, the win odds and the net benefit may have an advantage for interpreting the treatment effect; with adjustment (e.g., IPCW adjustment) for censoring, the three win statistics can complement one another to show the strength of the treatment effect. For calculations we use the R package WINS, available on the CRAN (Comprehensive R Archive Network).

The win odds: statistical inference and regression.

Generalized pairwise comparisons and win statistics (i.e., win ratio, win odds and net benefit) are advantageous in analyzing and interpreting a composite of multiple outcomes in clinical trials. An important limitation of these statistics is their inability to adjust for covariates other than by stratified analysis. Because the win ratio does not account for ties, the win odds, a modification that includes ties, has attracted attention. We review and combine information on the win odds to articulate the statistical inferences for the win odds. We also show alternative variance estimators based on the exact permutation and bootstrap as well as statistical inference via the probabilistic index. Finally, we extend multiple-covariate regression probabilistic index models to the win odds with a univariate outcome. As an illustration we apply the regression models to the data in the CHARM trial.

On the Q statistic with constant weights for standardized mean difference.

Cochran's Q statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value is also used in several popular estimators of the between-study variance, τ 2 . Those applications generally have not considered the implications of its use of estimated variances in the inverse-variance weights. Importantly, those weights make approximating the distribution of Q (more explicitly, Q IV ) rather complicated. As an alternative, we investigate a new Q statistic, Q F , whose constant weights use only the studies' effective sample sizes. For the standardized mean difference as the measure of effect, we study, by simulation, approximations to distributions of Q IV and Q F , as the basis for tests of heterogeneity and for new point and interval estimators of τ 2 . These include new DerSimonian-Kacker-type moment estimators based on the first moment of Q F , and novel median-unbiased estimators. The results show that: an approximation based on an algorithm of Farebrother follows both the null and the alternative distributions of Q F reasonably well, whereas the usual chi-squared approximation for the null distribution of Q IV and the Biggerstaff-Jackson approximation to its alternative distribution are poor; in estimating τ 2 , our moment estimator based on Q F is almost unbiased, the Mandel - Paule estimator has some negative bias in some situations, and the DerSimonian-Laird and restricted maximum likelihood estimators have considerable negative bias; and all 95% interval estimators have coverage that is too high when τ 2 = 0 , but otherwise the Q-profile interval performs very well.

Exploring consequences of simulation design for apparent performance of methods of meta-analysis.

Contemporary statistical publications rely on simulation to evaluate performance of new methods and compare them with established methods. In the context of random-effects meta-analysis of log-odds-ratios, we investigate how choices in generating data affect such conclusions. The choices we study include the overall log-odds-ratio, the distribution of probabilities in the control arm, and the distribution of study-level sample sizes. We retain the customary normal distribution of study-level effects. To examine the impact of the components of simulations, we assess the performance of the best available inverse-variance-weighted two-stage method, a two-stage method with constant sample-size-based weights, and two generalized linear mixed models. The results show no important differences between fixed and random sample sizes. In contrast, we found differences among data-generation models in estimation of heterogeneity variance and overall log-odds-ratio. This sensitivity to design poses challenges for use of simulation in choosing methods of meta-analysis.

A Q statistic with constant weights for assessing heterogeneity in meta-analysis.

The conventional Q statistic, using estimated inverse-variance (IV) weights, underlies a variety of problems in random-effects meta-analysis. In previous work on standardized mean difference and log-odds-ratio, we found superior performance with an estimator of the overall effect whose weights use only group-level sample sizes. The Q statistic with those weights has the form proposed by DerSimonian and Kacker. The distribution of this Q and the Q with IV weights must generally be approximated. We investigate approximations for those distributions, as a basis for testing and estimating the between-study variance (τ2 ). A simulation study, with mean difference as the effect measure, provides a framework for assessing accuracy of the approximations, level and power of the tests, and bias in estimating τ2 . Two examples illustrate estimation of τ2 and the overall mean difference. Use of Q with sample-size-based weights and its exact distribution (available for mean difference and evaluated by Farebrother's algorithm) provides precise levels even for very small and unbalanced sample sizes. The corresponding estimator of τ2 is almost unbiased for 10 or more small studies. This performance compares favorably with the extremely liberal behavior of the standard tests of heterogeneity and the largely biased estimators based on inverse-variance weights.

Adjusting win statistics for dependent censoring.

For composite outcomes whose components can be prioritized on clinical importance, the win ratio, the net benefit and the win odds apply that order in comparing patients pairwise to produce wins and subsequently win proportions. Because these three statistics are derived using the same win proportions and they test the same hypothesis of equal win probabilities in the two treatment groups, we refer to them as win statistics. These methods, particularly the win ratio and the net benefit, have received increasing attention in methodological research and in design and analysis of clinical trials. For time-to-event outcomes, however, censoring may introduce bias. Previous work has shown that inverse-probability-of-censoring weighting (IPCW) can correct the win ratio for bias from independent censoring. The present article uses the IPCW approach to adjust win statistics for dependent censoring that can be predicted by baseline covariates and/or time-dependent covariates (producing the CovIPCW-adjusted win statistics). Theoretically and with examples and simulations, we show that the CovIPCW-adjusted win statistics are unbiased estimators of treatment effect in the presence of dependent censoring.

Estimation in meta-analyses of response ratios.

BACKGROUND: For outcomes that studies report as the means in the treatment and control groups, some medical applications and nearly half of meta-analyses in ecology express the effect as the ratio of means (RoM), also called the response ratio (RR), analyzed in the logarithmic scale as the log-response-ratio, LRR. METHODS: In random-effects meta-analysis of LRR, with normal and lognormal data, we studied the performance of estimators of the between-study variance, τ2, (measured by bias and coverage) in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect in the log scale, λ. We obtained additional empirical evidence from two examples. RESULTS: The results of our extensive simulations showed several challenges in using LRR as an effect measure. Point estimators of τ2 had considerable bias or were unreliable, and interval estimators of τ2 seldom had the intended 95% coverage for small to moderate-sized samples (n<40). Results for estimating λ differed between lognormal and normal data. CONCLUSIONS: For lognormal data, we can recommend only SSW, a weighted average in which a study's weight is proportional to its effective sample size, (when n≥40) and its companion interval (when n≥10). Normal data posed greater challenges. When the means were far enough from 0 (more than one standard deviation, 4 in our simulations), SSW was practically unbiased, and its companion interval was the only option.

The inverse-probability-of-censoring weighting (IPCW) adjusted win ratio statistic: an unbiased estimator in the presence of independent censoring.

The win ratio method has received much attention in methodological research, ad hoc analyses, and designs of prospective studies. As the primary analysis, it supported the approval of tafamidis for the treatment of cardiomyopathy to reduce cardiovascular mortality and cardiovascular-related hospitalization. However, its dependence on censoring is a potential shortcoming. In this article, we propose the inverse-probability-of-censoring weighting (IPCW) adjusted win ratio statistic (i.e., the IPCW-adjusted win ratio statistic) to overcome censoring issues. We consider independent censoring, common censoring across endpoints, and right censoring. We develop an asymptotic variance estimator for the logarithm of the IPCW-adjusted win ratio statistic and evaluate it via simulation. Our simulation studies show that, as the amount of censoring increases, the unadjusted win proportions may decrease greatly. Consequently, the bias of the unadjusted win ratio estimate may increase greatly, producing either an overestimate or an underestimate. We demonstrate theoretically and through simulation that the IPCW-adjusted win ratio statistic gives an unbiased estimate of treatment effect.

Methods for estimating between-study variance and overall effect in meta-analysis of odds ratios.

In random-effects meta-analysis the between-study variance ( τ2 ) has a key role in assessing heterogeneity of study-level estimates and combining them to estimate an overall effect. For odds ratios the most common methods suffer from bias in estimating τ2 and the overall effect and produce confidence intervals with below-nominal coverage. An improved approximation to the moments of Cochran's Q statistic, suggested by Kulinskaya and Dollinger (KD), yields new point and interval estimators of τ2 and of the overall log-odds-ratio. Another, simpler approach (SSW) uses weights based only on study-level sample sizes to estimate the overall effect. In extensive simulations we compare our proposed estimators with established point and interval estimators for τ2 and point and interval estimators for the overall log-odds-ratio (including the Hartung-Knapp-Sidik-Jonkman interval). Additional simulations included three estimators based on generalized linear mixed models and the Mantel-Haenszel fixed-effect estimator. Results of our simulations show that no single point estimator of τ2 can be recommended exclusively, but Mandel-Paule and KD provide better choices for small and large numbers of studies, respectively. The KD estimator provides reliable coverage of τ2 . Inverse-variance-weighted estimators of the overall effect are substantially biased, as are the Mantel-Haenszel odds ratio and the estimators from the generalized linear mixed models. The SSW estimator of the overall effect and a related confidence interval provide reliable point and interval estimation of the overall log-odds-ratio.

The win ratio: Impact of censoring and follow-up time and use with nonproportional hazards.

The win ratio has been studied methodologically and applied in data analysis and in designing clinical trials. Researchers have pointed out that the results depend on follow-up time and censoring time, which are sometimes used interchangeably. In this article, we distinguish between follow-up time and censoring time, show theoretically the impact of censoring on the win ratio, and illustrate the impact of follow-up time. We then point out that, if the treatment has long-term benefit from a more important but less frequent endpoint (eg, death), the win ratio can show that benefit by following patients longer, avoiding masking by more frequent but less important outcomes, which occurs in conventional time-to-first-event analyses. For the situation of nonproportional hazards, we demonstrate that the win ratio can be a good alternative to methods such as landmark survival rate, restricted mean survival time, and weighted log-rank tests.

Estimation in meta-analyses of mean difference and standardized mean difference.

Methods for random-effects meta-analysis require an estimate of the between-study variance, τ2 . The performance of estimators of τ2 (measured by bias and coverage) affects their usefulness in assessing heterogeneity of study-level effects and also the performance of related estimators of the overall effect. However, as we show, the performance of the methods varies widely among effect measures. For the effect measures mean difference (MD) and standardized MD (SMD), we use improved effect-measure-specific approximations to the expected value of Q for both MD and SMD to introduce two new methods of point estimation of τ2 for MD (Welch-type and corrected DerSimonian-Laird) and one WT interval method. We also introduce one point estimator and one interval estimator for τ2 in SMD. Extensive simulations compare our methods with four point estimators of τ2 (the popular methods of DerSimonian-Laird, restricted maximum likelihood, and Mandel and Paule, and the less-familiar method of Jackson) and four interval estimators for τ2 (profile likelihood, Q-profile, Biggerstaff and Jackson, and Jackson). We also study related point and interval estimators of the overall effect, including an estimator whose weights use only study-level sample sizes. We provide measure-specific recommendations from our comprehensive simulation study and discuss an example.

Pitfalls of using the risk ratio in meta-analysis.

For meta-analysis of studies that report outcomes as binomial proportions, the most popular measure of effect is the odds ratio (OR), usually analyzed as log(OR). Many meta-analyses use the risk ratio (RR) and its logarithm because of its simpler interpretation. Although log(OR) and log(RR) are both unbounded, use of log(RR) must ensure that estimates are compatible with study-level event rates in the interval (0, 1). These complications pose a particular challenge for random-effects models, both in applications and in generating data for simulations. As background, we review the conventional random-effects model and then binomial generalized linear mixed models (GLMMs) with the logit link function, which do not have these complications. We then focus on log-binomial models and explore implications of using them; theoretical calculations and simulation show evidence of biases. The main competitors to the binomial GLMMs use the beta-binomial (BB) distribution, either in BB regression or by maximizing a BB likelihood; a simulation produces mixed results. Two examples and an examination of Cochrane meta-analyses that used RR suggest bias in the results from the conventional inverse-variance-weighted approach. Finally, we comment on other measures of effect that have range restrictions, including risk difference, and outline further research.