Optimal two-phase sampling for comparing correlated areas under the ROC curves of two screening tests in the presence of verification bias.

The accuracy of a screening test is often measured by the area under the receiver characteristic (ROC) curve (AUC) of a screening test. Two-phase designs have been widely used in diagnostic studies for estimating one single AUC and comparing two AUCs where the screening test results are measured for a large sample (Phase one sample) while the disease status is only verified for a subset of Phase one sample (Phase two sample) by a gold standard. In this paper, we consider the optimal two-phase sampling design for comparing the performance of two ordinal screening tests in classifying disease status. Specifically, we derive an analytical variance formula for the AUC difference estimator and use it to find the optimal sampling probabilities that minimize the variance formula for the AUC difference estimator. According to the proposed optimal two-phase design, the strata with the levels of two tests far apart from each other should be over-sampled while the strata with the levels of two tests close to each other should be under-sampled. Simulation results indicate that two-phase sampling under optimal allocation (OA) achieves a substantial amount of variance reduction, compared with two-phase sampling under proportional allocation (PA). Furthermore, in comparison with a one-phase random sampling, two-phase sampling under OA or PA has a clear advantage in reducing the variance of AUC difference estimator when the variances of the two screening test results in the disease population differ greatly from their counterparts in non-disease population.

Sample size calculation for comparing two screening tests when the gold standard is missing at random.

Sample size formulas have been proposed for comparing two sensitivities (specificities) in the presence of verification bias under a paired design. However, the existing sample size formulas involve lengthy calculations of derivatives and are too complicated to implement. In this paper, we propose alternative sample size formulas for each of three existing tests, two Wald tests and one weighted McNemar's test. The proposed sample size formulas are more intuitive and simpler to implement than their existing counterparts. Furthermore, by comparing the sample sizes calculated based on the three tests, we can show that the three tests have similar sample sizes even though the weighted McNemar's test only use the data from discordant pairs whereas the two Wald tests also use the additional data from accordant pairs.

Simple methods for comparing two predictive values with incomplete data.

Statistical methods have been well developed for comparing the predictive values of two binary diagnostic tests under a paired design. However, existing methods do not make allowance for incomplete data. Although maximum likelihood based method can be used to deal with incomplete data, it requires iterative algorithm for implementation. A simple and easily implemented statistical method is therefore needed. Simple methods exist for comparing two sensitivities or specificities with incomplete data but such simple methods are not available for comparing two predictive values with incomplete data. In this paper, we propose two simple methods for comparing two predictive values with incomplete data. The test statistics derived by these two methods are simple to compute, only involving some minor modification of the existing weighted generalized score statistics with complete data. Simulation results demonstrate that the proposed methods are more efficient than the ad-hoc method that only uses the subjects wit complete data. As an illustration, the proposed methods are applied to an observational study comparing two non-invasive methods in detecting endometriosis.

Joint comparison of the predictive values of multiple binary diagnostic tests: an extension of McNemar's test.

Positive and negative predictive values are important measures of the clinical accuracy of a diagnostic test. Various test statistics have been proposed to compare positive predictive values or negative predictive values of two binary diagnostic tests separately. However, such separate comparisons do not present a complete picture of the relative accuracy of the two diagnostic tests. In this paper, we propose an extension of McNemar's test for the joint comparison of predictive values of multiple diagnostic tests. The proposed extended McNemar's test is intuitive and simple to compute, only involving cell counts of discordant pairs from multiple 2×2 tables. Furthermore, we also propose a re-formulation of an existing Wald test statistic so that it can be implemented more easily than its original form. Simulations demonstrate that the proposed extended McNemar's test statistic preserves type one error much better than the existing Wald test statistic. Thus, we believe that the proposed extended McNemar's test statistic is the preferred statistic to simultaneously compare the predictive values of multiple binary diagnostic tests.

Weighted generalized score test for comparing predictive values in the presence of verification bias.

Positive and negative predictive values of a diagnostic test are two important measures of test accuracy, which are more relevant in clinical settings than sensitivity and specificity. Statistical methods have been well-developed to compare the predictive values of two binary diagnostic tests when test results and disease status fully observed for all study patients. In practice, however, it is common that only a subset of study patients have the disease status verified due to ethical or cost considerations. Methods applied directly to the verified subjects may lead to biased results. A bias-corrected method has been developed to compare two predictive values in the presence of verification bias. However, the complexity of the existing method and the computational difficulty in implementing it has restricted its use. A simple and easily implemented statistical method is therefore needed. In this paper, we propose a weighted generalized score (WGS) test statistic for comparing two predictive values in the presence of verification bias. The proposed WGS test statistic is intuitive and simple to compute, only involving some minor modification of the WGS test statistic when disease status is verified for each study patient. Simulations demonstrate that the proposed WGS test statistic preserves type I error much better than the existing Wald statistic. The method is illustrated with data from a study of methods for the diagnosis of coronary artery disease.

Weighted McNemar's test for the comparison of two screening tests in the presence of verification bias.

Statistical methods have been well-developed for comparing two binary screening tests in the presence of verification bias. However, the complexity of existing methods and the computational difficulty in implementing them have restricted their use. A simple and easily implemented statistical method is therefore needed. In this paper, we propose a weighted McNemar's test statistic for comparing two sensitivities(specificities). The proposed test statistics are intuitive and simple to compute, only involving some minor modification of a McNemar's test statistic using the estimated verification probabilities for discordant pairs. Simulations demonstrate that the proposed weighted McNemar's test statistics preserve type I error as well as or better than the existing statistics. Furthermore, unlike the existing methods, the proposed weighted McNemar's test statistics can still be applied even when none of the accordant pairs are verified.

Assess predictive values of a binary diagnostic test under a nested case-control design.

Predictive values of a binary diagnostic test are often evaluated under a random sample design. When the disease is rare, however, such a design might not be as efficient as a nested case-control design where the cases are oversampled from a large existing cohort. Under a nested case-control design, direct proportion estimators of predictive values are biased because cases are oversampled. Consistent estimates of predictive values can be easily obtained by inverse probability weighting (IPW) method. The only difficulty with these IPW estimators has been the absence of expressions for their variances. To fill this gap, in the current paper, we obtain the asymptotic variance formulas for the IPW estimators of predictive values. Unlike their counterparts from weighted logistic regression, our variance formulas take into account the variance of the estimated weights in the IPW estimators of predictive values. We further use the proposed variance formulas to examine the gain in efficiency under a nested case-control design compared with a simple random sampling design. Our results clearly show that when the disease is rare, a nested case-control design can achieve a substantial amount of variance reduction by oversampling cases, compared with a random sample design. Finally, we compare via simulation the accuracy of the proposed variance formulas with the existing methods and illustrate the proposed method by a real data example evaluating the accuracy of D-dimer test.

Nonparametric inference of the area under ROC curve under two-phase cluster sampling.

Nonparametric inference of the area under ROC curve (AUC) has been well developed either in the presence of verification bias or clustering. However, current nonparametric methods are not able to handle cases where both verification bias and clustering are present. Such a case arises when a two-phase study design is applied to a cohort of subjects (verification bias) where each subject might have multiple test results (clustering). In such cases, the inference of AUC must account for both verification bias and intra-cluster correlation. In the present paper, we propose an IPW AUC estimator that corrects for verification bias and derive a variance formula to account for intra-cluster correlations between disease status and test results. Results of a simulation study indicate that the method that assumes independence underestimates the true variance of the IPW AUC estimator in the presence of intra-cluster correlations. The proposed method, on the other hand, provides a consistent variance estimate for the IPW AUC estimator by appropriately accounting for correlations between true disease statuses and between test results.

Simple confidence interval and region formulas for comparing diagnostic likelihood ratios under a paired design.

A population-based paired design is often used for comparing the diagnostic likelihood ratios of two binary diagnostic tests. However, a case-control paired design, which involves the application of both diagnostic tests to two independent samples, is a good alternative study design especially when the disease is rare. Existing methods for comparing two diagnostic likelihood ratios have been mainly focused on the population-based paired design with little attention paid to the case-control paired design. In this paper, we derive a confidence interval formula for the relative diagnostic likelihood ratio (the ratio of two diagnostic likelihood ratios), which can be used for the comparison of two positive or negative diagnostic likelihood ratios separately. We also derive a confidence region formula for the two relative positive and negative diagnostic likelihood ratios, which allows simultaneous comparison of two positive and negative diagnostic likelihood ratios. The proposed confidence interval and region formulas are simple to compute and can be used for both population-based paired design and case-control paired designs. Simulation studies are used to assess the finite sample performance of the confidence interval and region formulas. The proposed methods are applied to a real data set on coronary artery disease and two diagnostic tests.

Optimal two-phase sampling for estimating the area under the receiver operating characteristic curve.

Statistical methods are well developed for estimating the area under the receiver operating characteristic curve (AUC) based on a random sample where the gold standard is available for every subject in the sample, or a two-phase sample where the gold standard is ascertained only at the second phase for a subset of subjects sampled using fixed sampling probabilities. However, the methods based on a two-phase sample do not attempt to optimize the sampling probabilities to minimize the variance of AUC estimator. In this paper, we consider the optimal two-phase sampling design for evaluating the performance of an ordinal test in classifying disease status. We derived the analytic variance formula for the AUC estimator and used it to obtain the optimal sampling probabilities. The efficiency of the two-phase sampling under the optimal sampling probabilities (OA) is evaluated by a simulation study, which indicates that two-phase sampling under OA achieves a substantial amount of variance reduction with an over-sample of subjects with low and high ordinal levels, compared with two-phase sampling under proportional allocation (PA). Furthermore, in comparison with an one-phase random sampling, two-phase sampling under OA or PA have a clear advantage in reducing the variance of AUC estimator when the variance of diagnostic test results in the disease population is small relative to its counterpart in nondisease population. Finally, we applied the optimal two-phase sampling design to a real-world example to evaluate the performance of a questionnaire score in screening for childhood asthma.

True verification probabilities should not be used in estimating the area under receiver operating characteristic curve.

In medical research, a two-phase study is often used for the estimation of the area under the receiver operating characteristic curve (AUC) of a diagnostic test. However, such a design introduces verification bias. One of the methods to correct verification bias is inverse probability weighting (IPW). Since the probability a subject is selected into phase 2 of the study for disease verification is known, both true and estimated verification probabilities can be used to form an IPW estimator for AUC. In this article, we derive explicit variance formula for both IPW AUC estimators and show that the IPW AUC estimator using the true values of verification probabilities even when they are known are less efficient than its counterpart using the estimated values. Our simulation results show that the efficiency loss can be substantial especially when the variance of test result in disease population is small relative to its counterpart in nondiseased population.

Power calculation of adjusted McNemar's test based on clustered data of varying cluster size.

McNemars test is often used to compare two proportions estimated from paired observations. When the observations are sampled in clusters, adjustment is needed to ensure that the size of McNemars test does not exceed the nominal level. Eliasziw and Donner (1991) developed an adjustment to McNemars test that involves first estimating the correlation between discordant pairs within a cluster, then using the estimate of the correlation to adjust the usual McNemar's test statistic. Gönen (2004) derived two approximations for calculating the power and sample size for the adjusted McNemar's test. He reported that the accuracy of the two approximations is compromised for large value of intracluster correlation and small value of proportion of discordant pairs; the error of the approximation can be higher than 10 per cent. In this paper, we extend his power formula, developed under fixed cluster size assumption, to accommodate the case where the cluster sizes are not constant. We show via simulations that the theoretical powers calculated from our proposed power formula are close to their empirical counterparts under a variety of settings. More significantly, in the case of fixed cluster size, our reduced power formula provides a more accurate power approximation than Gönen's power formula regardless of the values of intracluster correlation and the proportion of discordant pairs.

Heat stress risk profiles for three non-woven coveralls.

The ACGIH® Threshold Limit Value® (TLV®) is used to limit heat stress exposures so that most workers can maintain thermal equilibrium. That is, the TLV was set to an upper limit of Sustainable exposures for most people. This article addresses the ability of the TLV to differentiate between Sustainable and Unsustainable heat exposures for four clothing ensembles over a range of environmental factors and metabolic rates (M). The four clothing ensembles (woven clothing, and particle barrier, water barrier and vapor barrier coveralls) represented a wide range of evaporative resistances. Two progressive heat stress studies provided data on 480 trials with 1440 pairs of Sustainable and Unsustainable exposures for the clothing over three levels of relative humidity (rh) (20, 50 and 70%), three levels of metabolic rate (115, 180, and 254 Wm-2) using 29 participants. The exposure metric was the difference between the observed wet bulb globe temperature (WBGT) and the TLV. Risk was characterized by odds ratios (ORs), Receiver Operating Characteristic (ROC) curves, and dose-response curves for the four ensembles. Conditional logistic regression models provided information on ORs. Logistic regressions were used to determine ROC curves with area under the curve (AUC), model the dose-response curve, and estimate offsets from woven clothing. The ORs were about 2.5 per 1°C-WBGT for woven clothing, particle barrier, and water barrier and for vapor barrier at 50% rh. When using the published Clothing Adjustment Values (CAVs, also known as Clothing Adjustment Factors, CAFs) or the offsets that included different values for vapor barrier based on rh, the AUC for all clothing was 0.86. When the fixed CAVs of the TLV were used, the AUC was 0.81. In conclusion, (1) ORs and the shapes of the dose-response curves for the nonwoven coveralls were similar to woven clothing, and (2) CAVs provided a robust way to account for the risk of nonwoven clothing. The robust nature of CAV extended to the exclusion of different adjustments for vapor barrier by rh.

Ability to Discriminate Between Sustainable and Unsustainable Heat Stress Exposures-Part 1: WBGT Exposure Limits.

OBJECTIVES: Heat stress exposure limits based on wet-bulb globe temperature (WBGT) were designed to limit exposures to those that could be sustained for an 8-h day using limited data from Lind in the 1960s. In general, Sustainable exposures are heat stress levels at which thermal equilibrium can be achieved, and Unsustainable exposures occur when there is a steady increase in core temperature. This paper addresses the ability of the ACGIH® Threshold Limit Value (TLV®) to differentiate between Sustainable and Unsustainable heat exposures, to propose alternative occupational exposure limits, and ask whether an adjustment for body surface area improves the exposure decision. METHODS: Two progressive heat stress studies provided data on 176 trials with 352 pairs of Sustainable and Unsustainable exposures over a range of relative humidities and metabolic rates using 29 participants wearing woven cotton clothing. To assess the discrimination ability of the TLV, the exposure metric was the difference between the observed WBGT and the TLV adjusted for metabolic rate. Conditional logistic regression models and receiver operating characteristic curves (ROC) along with ROC's area under the curve (AUC) were used. Four alternative models for an occupational exposure limit were also developed and compared to the TLV. RESULTS: For the TLV, the odds ratio (OR) for Unsustainable was 2.5 per 1°C-WBGT [confidence interval (CI) 2.12-2.88]. The AUC for the TLV was 0.85 (CI 0.81-0.89). For the alternative models, the ORs were also about 2.5/°C-WBGT, with AUCs between 0.84 and 0.88, which were significantly different from the TLV's AUC but have little practical difference. CONCLUSIONS: This study (1) confirmed that the TLV is appropriate for heat stress screening; (2) demonstrated the TLV's discrimination accuracy with an ROC AUC of 0.85; and (3) established the OR of 2.5/°C-WBGT for unsustainable exposures. The TLV has high sensitivity, but its specificity is very low, which is protective. There were no important improvements with alternative exposure limits, and there was weak evidence to support metabolic rate normalized to body surface area. In sum, the TLV is protective with an appropriate margin of safety for relatively constant occupational exposures to heat stress.

Ability to Discriminate Between Sustainable and Unsustainable Heat Stress Exposures-Part 2: Physiological Indicators.

OBJECTIVES: There are times when it is not practical to assess heat stress using environmental metrics and metabolic rate, and heat strain may provide an alternative approach. Heat strain indicators have been used for decades as tools for monitoring physiological responses to work in hot environments. Common indicators of heat strain are body core temperature (assessed here as rectal temperature Tre), heart rate (HR), and average skin temperature (Tsk). Data collected from progressive heat stress trials were used to (1) demonstrate if physiological heat strain indicators (PHSIs) at the upper limit of Sustainable heat stress were below generally accepted limits; (2) suggest values for PHSIs that demonstrate a Sustainable level of heat stress; (3) suggest alternative PHSIs; and (4) determine if metabolic rate was an effect modifier. METHODS: Two previous progressive heat stress studies included 176 trials with 352 pairs of Sustainable and Unsustainable exposures over a range of relative humidities and metabolic rates using 29 participants. To assess the discrimination ability of PHSIs, conditional logistic regression and stepwise logistic regression were used to find the best combinations of predictors of Unsustainable exposures. The accuracy of the models was assessed using receiver operating characteristic curves. RESULTS: Current recommendations for physiological heat strain limits were associated with probabilities of Unsustainable greater than 0.5. Screening limits for Sustainable heat stress were Tre of 37.5°C, HR of 105 bpm, and Tsk of 35.8°C. Tsk alone resulted in an area under the curve of 0.85 and the combination of Tsk and HR (area under the curve = 0.88) performed the best. The adjustment for metabolic rate was statistically significant for physiological strain index or ∆Tre-sk as main predictors, but its effect modification was negligible and could be ignored. CONCLUSIONS: Based on the receiver operating characteristic curve, PHSIs (Tre, HR, and Tsk) can accurately predict Unsustainable heat stress exposures. Tsk alone or in combination with HR has a high sensitivity, and makes better discriminations than the other PHSIs under relatively constant exposure (metabolic rate and environment) for an hour or so. Screening limits with high sensitivity, however, have low thresholds that limit utility. To the extent that the observed strain is low, there is good evidence that the exposure is Sustainable.

The ratio of pericardial to subcutaneous adipose tissues is associated with insulin resistance.

OBJECTIVE: To examine the association between pericardial adipose tissue (PAT) and the ratio of PAT to subcutaneous adipose tissue (SAT) with insulin resistance in adults with and without type 1 diabetes (T1D). METHODS: Data for this report came from a substudy of the Coronary Artery Calcification in Type 1 Diabetes cohort (n = 83; 38 with T1D, 45 without T1D). Insulin resistance was measured by hyperinsulinemic-euglycemic clamp. Abdominal computed tomography (CT) was used to measure visceral adipose tissue (VAT) and SAT. PAT was measured from CT scans of the heart. RESULTS: PAT and the ratio of PAT to SAT was higher in males compared to females. After adjustment for demographics, diabetes, blood pressure and lipid factors, BMI, VAT, and log PAT/SAT ratio, log PAT was positively associated with the glucose infusion rate (GIR) in females only (ß = 3.36 ± 1.96, P = 0.097, P for sex interaction = 0.055). Conversely, the log PAT/SAT ratio was significantly associated with decreased GIR in both males and females (ß = -2.08 ± 1.03, P = 0.047, P for sex interaction = 0.768). CONCLUSIONS: A significant association between the PAT/SAT ratio and insulin resistance was found, independent of BMI, VAT, and PAT. These results highlight the importance of considering fat distribution independent of volume.

Sleep, Cognitive impairment, and Alzheimer's disease: A Systematic Review and Meta-Analysis.

Study Objectives: Mounting evidence implicates disturbed sleep or lack of sleep as one of the risk factors for Alzheimer's disease (AD), but the extent of the risk is uncertain. We conducted a broad systematic review and meta-analysis to quantify the effect of sleep problems/disorders on cognitive impairment and AD. Methods: Original published literature assessing any association of sleep problems or disorders with cognitive impairment or AD was identified by searching PubMed, Embase, Web of Science, and the Cochrane library. Effect estimates of individual studies were pooled and relative risks (RR) and 95% confidence intervals (CI) were calculated using random effects models. We also estimated the population attributable risk. Results: Twenty-seven observational studies (n = 69216 participants) that provided 52 RR estimates were included in the meta-analysis. Individuals with sleep problems had a 1.55 (95% CI: 1.25-1.93), 1.65 (95% CI: 1.45-1.86), and 3.78 (95% CI: 2.27-6.30) times higher risk of AD, cognitive impairment, and preclinical AD than individuals without sleep problems, respectively. The overall meta-analysis revealed that individuals with sleep problems had a 1.68 (95% CI: 1.51-1.87) times higher risk for the combined outcome of cognitive impairment and/or AD. Approximately 15% of AD in the population may be attributed to sleep problems. Conclusion: This meta-analysis confirmed the association between sleep and cognitive impairment or AD and, for the first time, consolidated the evidence to provide an "average" magnitude of effect. As sleep problems are of a growing concern in the population, these findings are of interest for potential prevention of AD.

Exertional heat illness and acute injury related to ambient wet bulb globe temperature.

BACKGROUND: The Deepwater Horizon disaster cleanup effort provided an opportunity to examine the effects of ambient thermal conditions on exertional heat illness (EHI) and acute injury (AI). METHODS: The outcomes were daily person-based frequencies of EHI and AI. Exposures were maximum estimated WBGT (WBGTmax) and severity. Previous day's cumulative effect was assessed by introducing previous day's WBGTmax into the model. RESULTS: EHI and AI were higher in workers exposed above a WBGTmax of 20°C (RR 1.40 and RR 1.06/°C, respectively). Exposures above 28°C-WBGTmax on the day of the EHI and/or the day before were associated with higher risk of EHI due to an interaction between previous day's environmental conditions and the current day (RRs from 1.0-10.4). CONCLUSIONS: The risk for EHI and AI were higher with increasing WBGTmax. There was evidence of a cumulative effect from the prior day's WBGTmax for EHI. Am. J. Ind. Med. 59:1169-1176, 2016. © 2016 Wiley Periodicals, Inc.

Incidence rates of dementia, Alzheimer disease, and vascular dementia in the Japanese American population in Seattle, WA: the Kame Project.

There are few studies on the incidence of dementia in representative minority populations in the United States; however, no population-based study has been conducted on Japanese American women. We identified 3045 individuals aged 65+ with at least 1 parent of Japanese descent living in King County, WA in the period 1992 to 1994, of whom 1836 were dementia-free and were examined every 2 years (1994 to 2001) to identify incident cases of all dementias, Alzheimer disease (AD), vascular dementia (VaD), and other dementias. Cox regression was used to examine associations with age, sex, years of education, and apolipoprotein (APOE)-ε4. Among 173 incident cases of dementia, the overall rate was 14.4/1000/y, with rates being slightly higher among women (15.9/1000) than men (12.5/1000). Rates roughly doubled every 5 years for dementia and AD; the age trend for VaD and other dementias was less consistent. Sex was not significantly related to incidence of dementia or its subtypes in adjusted models. There was a trend for an inverse association with increasing years of education. APOE-ε4 was a strong risk factor for all dementias [hazard ratio (HR)=2.89; 95% confidence interval (CI), 1.88-4.46], AD (HR=3.27; 95% CI, 2.03-5.28), and VaD (HR=3.33; 95% CI, 1.34-8.27). This study is the first to report population-based incidence rates for both Japanese American men and women.

Predictors of right ventricular systolic dysfunction in compensated and decompensated heart failure.

Current understanding of the mechanisms of right ventricular (RV) systolic dysfunction in heart failure (HF) is limited. The authors analyzed a limited access dataset from the Evaluation Study of Congestive Heart Failure and Pulmonary Artery Catheterization Effectiveness (ESCAPE) provided by the National Heart, Lung, and Blood Institute (NHLBI). RV systolic function was measured by echocardiography at baseline and at 3-month follow-up using fractional area change. Univariate and multivariate analysis was performed with linear regression. Of 433 patients enrolled in the ESCAPE trial, 190 had RV systolic function measured at baseline (decompensated HF) and 147 had it measured at 3-month follow-up. On both occasions, parameters of congestion were associated with RV systolic function. Interestingly, lower hematocrit level was also associated with better RV systolic function. In multivariate analysis, only wedge pressure remained a statistically significant predictor of RV dysfunction. In summary, cardiac diastolic pressures and corresponding echocardiographic parameters, as well as hematocrit level, predicted RV systolic function in both compensated and decompensated systolic HF.