A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations.

An important quality of meta-analytic models for research synthesis is their ability to account for both within- and between-study variability. Currently available meta-analytic approaches for studies of diagnostic test accuracy work primarily within a fixed-effects framework. In this paper we describe a hierarchical regression model for meta-analysis of studies reporting estimates of test sensitivity and specificity. The model allows more between- and within-study variability than fixed-effect approaches, by allowing both test stringency and test accuracy to vary across studies. It is also possible to examine the effects of study specific covariates. Estimates are computed using Markov Chain Monte Carlo simulation with publicly available software (BUGS). This estimation method allows flexibility in the choice of summary statistics. We demonstrate the advantages of this modelling approach using a recently published meta-analysis comparing three tests used to detect nodal metastasis of cervical cancer.

Effect of training with the american college of radiology breast imaging reporting and data system lexicon on mammographic interpretation skills in developing countries.

RATIONALE AND OBJECTIVES: The authors evaluated the effect of training in the American College of Radiology (ACR) Breast Imaging Reporting and Data System (BI-RADS) lexicon on the interpretive skills of radiologists evaluating screening mammograms in Ukraine. MATERIALS AND METHODS: As part of a program to improve breast cancer detection and treatment in Ukraine, a series of training sessions was given to a group of radiologists across Ukraine to improve their interpretive skills in screening mammography. The training sessions focused on the use of the lexicon and assessment categories developed by the ACR BI-RADS committee. Participants (n = 14) evaluated 30 test screening mammograms before and after the training sessions. The test sets were randomly selected from a larger collection of training sets containing normal, benign, and abnormal mammograms. False-positive, false-negative, true-positive, and true-negative evaluations were determined, and sensitivity, specificity, and positive predictive values were calculated for each participant before and after training. RESULTS: The mean baseline sensitivity, specificity, and positive predictive values were 50%, 77%, and 43%, respectively. Each of these measures of interpretive skills improved significantly after training in the use of the lexicon, to 87%, 89%, and 78% (P < .0001, P < .01, and P < .0001, respectively). CONCLUSION: As the use of mammography spreads throughout developing countries, it is essential to address training and educational needs, as well as equipment needs. The ACR BI-RADS lexicon provides a systematic and efficient method for training radiologists to interpret screening mammograms. Educating radiologists on the use of this lexicon proved an effective way to improve their interpretive skills in screening mammography.

Treatment process and outcomes for managed care patients receiving new antidepressant prescriptions from psychiatrists and primary care physicians.

BACKGROUND: While many studies describe deficiencies in primary care antidepressant treatment, little research has applied similar standards to psychiatric practice. This study compares baseline characteristics, process of care, and outcomes for managed care patients who received new antidepressant prescriptions from psychiatrists and primary care physicians. METHODS: At a prepaid health plan in Washington State, patients receiving initial antidepressant prescriptions from psychiatrists (n = 165) and primary care physicians (n = 204) completed a baseline assessment, including the Structured Clinical Interview for DSM-IV depression module, a 20-item depression assessment from the Symptom Checklist-90, and the Medical Outcomes Survey 36-Item Short-Form Health Survey functional status questionnaire. All measures were repeated after 2 and 6 months. Computerized data were used to assess antidepressant refills and follow-up visits over 6 months. RESULTS: At baseline, psychiatrists' patients reported slightly higher levels of functional impairment and greater prior use of specialty mental health care. During follow-up, psychiatrists' patients made more frequent follow-up visits, and the proportion making 3 or more visits in 90 days was 57% vs 26% for primary care physicians' patients. The proportion receiving antidepressant medication at an adequate dose for 90 days or more was similar (49% vs 48%). The 2 groups showed similar rates of improvement in all measures of symptom severity and functioning. CONCLUSIONS: In this sample, clinical differences between patients treated by psychiatrists and primary care physicians were modest. Shortcomings in depression treatment frequently noted in primary care (inadequate follow-up care and high rates of inadequate antidepressant treatment) were also common in specialty practice. Possible selection bias limits any conclusions about relative effectiveness or cost-effectiveness.

Changes in breast density associated with initiation, discontinuation, and continuing use of hormone replacement therapy.

CONTEXT: Initiation of hormone replacement therapy (HRT) has been shown to increase breast density. Evidence exists that increased breast density decreases mammographic sensitivity. The effects on breast density of discontinuing and continuing HRT have not been studied systematically. OBJECTIVE: To examine the effects of initiation, discontinuation, and continued use of HRT on breast density in postmenopausal women. DESIGN, SETTING, AND PARTICIPANTS: Observational cohort study of 5212 naturally postmenopausal women aged 40 to 96 years and enrolled in a large health maintenance organization in western Washington State who had 2 screening mammograms between 1996 and 1998. MAIN OUTCOME MEASURES: Breast density, assessed using the clinical radiologists' BI-RADS 4-point scale, compared among women who did not use HRT before either mammogram (nonusers); who used HRT before the first but not before the second mammogram (discontinuers); who used HRT before the second but not before the first mammogram (initiators); and who used HRT prior to both mammograms (continuing users). RESULTS: Relative to nonusers, women who initiated HRT were more likely to show increases in breast density (relative risk [RR], 2.57; 95% confidence interval [CI], 2.12-3.08), while women who discontinued HRT use were more likely to show decreases in density (RR, 1.81; 95% CI, 1.06-2.98) and women who continued to use HRT were more likely to show both increases in density (RR, 1.33; 95% CI, 1.13-1.55) and sustained high density (RR, 1.45; 95% CI, 1.33-1.58). CONCLUSIONS: These results indicate that breast density changes associated with HRT are dynamic, increasing with initiation, and decreasing with discontinuation.

Binge drinking among female Veterans Affairs patients: prevalence and associated risks.

This study evaluated the prevalence and associated risks of binge drinking, defined as having > or = 4 drinks on an occasion in the past year, in a female patient population. Of 1,259 female Veterans Affairs patients surveyed, 780 reported drinking alcohol in the past year, and 305 (24% of respondents, 39% of drinkers) reported binge drinking in the past year; 84 (11% of drinkers) had done so monthly or more often. Age-adjusted logistic regression analyses indicated that women who reported past-year binge drinking monthly or more often reported significantly increased odds of morning drinking (odds ratio [OR] = 40.3), others worrying about their drinking (OR = 38.6), arguments after drinking (OR = 13.5), hepatitis or cirrhosis (OR = 3.1), frequent injuries (OR = 2.6), smoking (OR = 3.7), drug use (OR = 22.2), and multiple sexual partners (OR = 4.6).

Colorectal cancer screening participation by older women.

OBJECTIVE: Although recent screening guidelines recommend annual fecal occult blood testing (FOBT) for adults aged > or = 50, a number of studies report that these tests are underused. Systematic efforts to increase awareness of colorectal cancer (CRC) and to promote screening participation are needed to meet national objectives for CRC control. METHODS: This study examined CRC-screening practices and evaluated factors related to recent participation in screening by FOBT in a sample of women aged 50 to 80 who were surveyed about their use of clinical preventive services at Group Health Cooperative, a managed care organization in western Washington State. RESULTS: Of the 931 women eligible for analysis, 75% reported ever having been screened by FOBT and 48% reported having been screened within 2 years before the survey. Participation in screening did not vary by demographic characteristics or by perceived or actual risk of CRC. Women with a positive attitudes toward CRC screening had sevenfold greater odds of recent screening by FOBT (odds ratio=7.1; 95% confidence interval, 4.4 to 11.6). Only 58% of study women reported that their physicians encouraged CRC screening, but this factor was strongly related to participation (odds ratio=12.7; 95% confidence interval, 6.6 to 24.4). CONCLUSIONS: We identified several areas in which understanding of CRC risk may be low. As a whole, these findings suggest that effective strategies to control CRC may include efforts to improve knowledge of risk and prevention, but must also appeal directly to primary care physicians to identify and address their barriers to screening recommendations.

Are there detectable differences in quality of care or outcome of depression across primary care providers?

OBJECTIVE: The objective of this work was to determine whether there are detectable differences among primary care physicians in measures of quality of care or clinical outcome for depressed patients during the first 2 months of treatment with antidepressant medication. METHODS: We studied 1,599 depressed primary care patients initiating antidepressant treatment from 63 family physicians in 4 primary care clinics of a staff-model health maintenance organization. Patients were interviewed 6 to 8 weeks after initiating antidepressant medication with a telephone structured interview that included the Structural Clinical Interview for DSM-IV Diagnoses (SCID). Automated databases of the HMO were used to examine 3 quality-of-care measures: (1) the percentage of patients who had a refill of their antidepressant by 6 weeks, (2) the percentage of patients who had a return visit by 3 weeks, and (3) the percentage of patients having a return visit by 6 weeks. The percentage of patients in each primary care physician panel who had > or =4 persistent DSM-IV depressive symptoms at 6 to 8 weeks was the main clinical outcome variable. To adjust for case-mix differences between physician panels, patient age, gender, and medical comorbidity were controlled for in the analyses. Two covariates were used to adjust for differences in patients' clinical severity: self-report of > or =2 prior depressive episodes and an SCID diagnosis of major depression during the patient's worst episode in the last 2 years. Physician age, gender, and part-time versus full-time practice were also used as covariates. RESULTS: The wide observed range of variability in quality-of-care and clinical outcome measures by physician practice decreased markedly in the statistical model that controlled for patient-level covariates and differences in the number of patients seen per provider. We did not detect differences in physician practice for the percentage of patients who had a return visit by 3 or 6 weeks, the percentage of patients who had a refill of their antidepressant prescription by 6 weeks, or the percentage of patients with an adverse clinical outcome of depression. CONCLUSIONS: We did not find important differences in measures of quality of care or patient outcomes by physician. These results may have implications for the use of physician profiling and other forms of physician report cards.

Bootstrap estimation of diagnostic accuracy with patient-clustered data.

RATIONALE AND OBJECTIVES: The purpose of this study was to describe a simple bootstrap approach for estimating sensitivity, specificity, and the area under the receiver operating characteristic curve for multisite test outcome data. MATERIALS AND METHODS: The performance of bootstrap estimates was evaluated and compared with that of analytic estimates by using a simulation study. Bootstrapping was demonstrated by using data from a previous study comparing two angiographic methods. RESULTS: Analytic and bootstrap estimates had similar coverage rates for 95% confidence intervals. With many sites per patient, bootstrap estimates had slightly better coverage than analytic estimates. Bootstrap percentile intervals had better coverage than asymptotic normal bootstrap intervals. CONCLUSION: Bootstrapping is a useful method of estimating confidence intervals for the area under the receiver operating characteristic curve, sensitivity, and specificity when data are correlated.

Accuracy of screening mammography using single versus independent double interpretation.

OBJECTIVE: We conducted an analysis among 31 community radiologists to identify the average change in screening mammography interpretive accuracy afforded by independent double interpretation. MATERIALS AND METHODS: We assessed interpretive accuracy using a stratified random sample of test mammograms that included 30 women with cancer and 83 without. Radiologists were unaware of clinical information and of each other's assessments. We describe accuracy for individual radiologists and for double interpretation, including average sensitivity, specificity, diagnostic likelihood ratios positive and negative, and area under the receiver operating characteristic (ROC) curve. We also assessed weighted and nonweighted kappa statistics among all 465 pairs of radiologists and 31,465 pairs of unique pairs. The assessment for double interpretations used the "highest" (i.e., most abnormal) assessment of the two radiologists. We calculated the difference between each radiologist's individual accuracy and the average accuracy across that radiologist's 30 double interpretations. RESULTS: We found the following average accuracy statistics for individual radiologists: sensitivity, 79%; specificity, 81%; diagnostic likelihood ratio positive, 5.53; diagnostic likelihood ratio negative, 0.26; and area under the ROC curve, 0.85. The mean kappa statistic among radiologists for cancer cases increased with double interpretation from 0.59 to 0.70, and for noncancer cases from 0.30 to 0.34. Double interpretation resulted in an average increase in sensitivity of 7%, an average decrease in specificity of 11%, a decrease in diagnostic likelihood ratio positive of 2.35, a decrease in diagnostic likelihood ratio negative of 0.06, and an increase in area under the ROC curve of 0.02. CONCLUSION: Independent double interpretation does not increase accuracy as measured by the area under the ROC curve.

Assessing mammographers' accuracy. A comparison of clinical and test performance.

Direct estimation of mammographers' clinical accuracy requires the ability to capture screening assessments and correctly identify which screened women have breast cancer. This clinical information is often unavailable and when it is available its observational nature can cause analytic problems. Problems with clinical data have led some researchers to evaluate mammographers using a single set of films. Research based on these test film sets implicitly assumes a correspondence between mammographers' accuracy in the test setting and their accuracy in a clinical setting. However, there is no evidence supporting this basic assumption. In this article we use hierarchical models and data from 27 mammographers to directly compare accuracy estimated from clinical practice data to accuracy estimated from a test film set. We found moderate positive correlation [ rho; = 0.206 with 95% credible interval (-0.142-0. 488)] between mammographers' overall preponderance to call a mammogram positive. However, we found no evidence of correlation between clinical and test accuracy [ rho; = -0.025 with 95% credible interval (-0.484-0.447)]. This study is limited by the relatively small number of mammographers evaluated, by the somewhat restricted range of observed sensitivities and specificities, and by differences in the types of films evaluated in test and clinical datasets. Nonetheless, these findings raise important questions about how mammographer accuracy should be measured.

Relapse of depression in primary care. Rate and clinical predictors.

OBJECTIVE: To determine the clinical predictors and rate of relapse for major depression in primary care. DESIGN: A cohort study of subjects in 2 randomized trials of depressed patients diagnosed and prescribed antidepressant medicine by primary care physicians. Baseline, 7-month, and 19-month assessments were conducted. SETTING: A large primary care clinic of a staff-model health maintenance organization. PATIENTS: Two hundred fifty-one primary care patients who did not satisfy Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition (DSM-III-R) criteria for major depression at 7 months. MAIN OUTCOME MEASURES: Relapse was defined as (1) satisfying DSM-III-R criteria for major depression at 19 months, or (2) reporting an interval episode of 2 weeks or more of depressed mood and symptoms between 7 and 19 months. Predictors examined included demographic characteristics, medical comorbidity, disability, and psychological symptoms. Depressive symptoms were measured by Inventory of Depressive Symptoms and Hopkins Symptoms Checklist. RESULTS: Of the patients, 37.1% reported relapse of depression in the 12-month relapse-risk period. The 2 major risk factors associated with relapse were (1) persistence of subthreshold depressive symptoms 7 months after the initiation of antidepressant therapy (odds ratio, 3.3; 95% confidence interval, 2.74-3.93) and (2) history of 2 or more episodes of major depression, or chronic mood symptoms for 2 years (odds ratio, 2.1; 95% confidence interval, 1.41-2.76). Patients with both risk factors were approximately 3 times more likely to relapse than patients with neither. CONCLUSIONS: The relapse rate among primary care patients treated for depression approached that of specialty samples, with more than one third reporting relapse in 1 year. Clinical characteristics can help target high-risk patients for relapse prevention efforts.

Comparison of Doppler US and CT angiography for evaluation of renal artery stenosis.

RATIONALE AND OBJECTIVES: The authors compared Doppler ultrasound (US) with computed tomographic (CT) angiography in the evaluation of stenosis of the main renal artery. MATERIALS AND METHODS: Fifty-six patients who had undergone conventional angiography of the renal arteries participated in a prospective comparison of Doppler US (45 patients) and CT angiography (52 patients). US evaluation included both the main renal artery and segmental renal arteries. RESULTS: There were 27 main renal arteries with at least 50% stenosis in 20 patients. In 36 patients, there was no significant stenosis. All cases of main renal artery stenosis detected with Doppler US of the segmental arteries were also identified with Doppler US of the main renal artery. The by-artery sensitivity (63%) of US of the main renal artery was greater than that (33%) of US of the segmental arteries. CT angiography was more sensitive (96%) than Doppler US (63%) in the detection of stenosis, but the specificity of CT (88%) was similar to that of US (89%). The difference in the area under the receiver operating characteristic curve (AUC) between CT (AUC = 0.94) and US (AUC = 0.82) was statistically significant (P = .038). CONCLUSION: Doppler US of the main renal artery is more sensitive than Doppler US of segmental arteries in the detection of stenosis. CT angiography is more accurate than Doppler US in the evaluation of renal artery stenosis.

Percutaneous varicocele embolization versus surgical ligation for the treatment of infertility: changes in seminal parameters and pregnancy outcomes.

PURPOSE: To compare the success of percutaneous varicocele embolization to surgical ligation with regard to changes in semen characteristics and pregnancy outcome. MATERIALS AND METHODS: Infertility records from 346 men who underwent correction of their varicocele for infertility (surgical ligation 149; embolization 197) were reviewed retrospectively. Preprocedural and postprocedural semen analyses and pregnancy outcomes were obtained with use of chart and telephone follow-up. RESULTS: In men who successfully impregnated their partners, there were significant improvements in sperm density, percent total improvement, motility, and progression. Postprocedural (embolization vs surgery) percentage increases in seminal parameters were density, 156.8% versus 138.5%; total, 168.8% versus 157.91%; and motility, 2.7% versus 3.2%. The percent of individuals who had a change in sperm progression was 31% versus 41%. There was no statistical difference between the techniques based on t tests. The pregnancy rates were similar for the two groups, 39% and 34% for embolization and surgery, respectively. CONCLUSION: There is no significant statistical difference in seminal values or pregnancy outcome between the two techniques.

Achieving guidelines for the treatment of depression in primary care: is physician education enough?

OBJECTIVES: The authors examine whether physician education has enduring effects on treatment of depression. METHODS: Depressed primary care patients initiating antidepressant treatment from primary care clinics of a staff-model health maintenance organization were studied. Quasi-experimental and before-and-after comparisons of physician practices, supplemented with patient surveys, were used to compare the process of care and depression outcomes. Intervention consisted of extensive physician education that spanned a 12-month period. This included case-by-case consultations, didactics, academic detailing (eg, clearly stating the educational and behavioral objectives to individual physicians), and role-play of optimal treatment. Main outcome measures were divided into two groups. Quasi-experimental samples included: (1) antidepressant medication selection and (2) adequacy (dosage and duration) of pharmacotherapy. Survey samples included: (3) intensity of follow-up; (4) physician delivered educational messages regarding depression treatment; (5) patient satisfaction; and (6) depression outcomes. RESULTS: No lasting educational effect was observed consistently in any of the outcomes measured. CONCLUSIONS: There was no enduring improvement in the treatment of depression for primary care patients. Depression treatment guidelines were achieved contemporaneously, however, for intervention patients enrolled in a multifaceted program of collaborative care during the training period. These results suggest that continuing programs of reorganized service delivery to support the role of a primary care physician (eg, on-site mental health personnel, close monitoring of patient progress and adherence), in addition to physician training, are essential for the success of guideline implementation.

Multicenter trial to evaluate vascular magnetic resonance angiography of the lower extremity. American College of Radiology Rapid Technology Assessment Group.

OBJECTIVES: To assess the value of magnetic resonance angiography (MRA) in presurgical evaluation of patients with severe lower limb atherosclerotic occlusive disease and to assess the feasibility of rapidly conducting rigorous technology assessment. DESIGN: Blinded, prospective study of consecutive patients with signs or symptoms of severe infrainguinal peripheral vascular disease who were candidates for percutaneous or surgical intervention. Using both descriptive statistics and multivariate logistic analyses, MRA was compared with contrast arteriography (CA) (the current technique) for imaging 15 arterial segments of the leg and foot. Intraoperative contrast angiography was the "gold" standard. Also studied was the effect of adding MRA to the information used in planning treatment. SETTING: Six US hospitals, one a community hospital. PATIENTS: A total of 155; 84% with either rest pain or tissue loss. RESULTS: Sensitivity in distinguishing patent segments from completely occluded segments was 83% for CA and 85% for MRA; both had 81% specificity. For distinguishing near-normal segments (suitable as bypass graft termini), CA was less sensitive than MRA (77% vs 82%), but more specific (92% vs 84%). After adjusting for same-reader effects, odds of correctly distinguishing patent segments were 1.6 times as great for MRA as for CA (P < .01); for distinguishing near-normal segments, the odds for CA were 1.5 times as great as for MRA (P < .05). The addition of MRA changed the treatment plan in 13% of patients; in 86% of these cases, the surgery actually performed indicated that the MRA-inclusive plan was superior. CONCLUSIONS: Individually, MRA and CA are approximately equivalent in diagnostic accuracy. The addition of MRA to treatment plans based only on CA and other diagnostic information clearly improves the plans. Completed in 15 months (as planned), our study demonstrates the feasibility of conducting rigorous technology assessment rapidly enough to be timely even in fields in which diagnostic and treatment techniques are rapidly changing.

Platelet activation by healing ePTFE grafts.

The purpose of this study was to examine the effect of the in vivo maturing ePTFE graft surface on platelet activation. Ten canines were randomized to receive either a carotid to infrarenal aorta ePTFE graft or sham operation. Animals were sampled at specific time points up to 3 months postoperatively. Whole blood platelet aggregometry (arachidonic acid, ADP, and collagen agonists) and ATP secretion (in response to arachidonic acid, ADP, collagen, and thrombin) were measured. Additionally, complete hematologic analysis and histology were performed. With time, graft animals showed significantly more decrease in platelet aggregation in response to ADP compared to sham animals (P = .023). The total amount of ATP per platelet was not different, as demonstrated by equivalent ATP release per platelet in response to thrombin. Over the first week, grafted dogs developed a decrease in systemic platelet count of 50% (P < .001) that persisted over the 3-month follow-up period. With time, overall regression model slopes of graft and sham platelet count data were not statistically different (P = .29). Histologically, the grafts demonstrated limited cellular ingrowth at both anastomoses, with fibrin matrix along the remainder of the blood-biomaterial interface. These data suggest that, similar to Dacron, exposure to an ePTFE surface results in significant changes in platelet biology, and these platelet-ePTFE interactions persist even after the graft has formed a mature pseudointima. The pseudointima appears to be the primary determinant of the blood-biomaterial interaction.

Analysis of longitudinal data: random coefficient regression modelling.

We review random coefficient regression (RCR) models and methods for fitting these models from an applications perspective. Methods for data with exponential family distributions are presented with the Gaussian distribution as a special case. Attention is given to interpretation of fixed effects and the correlation structures implied by RCR models. Estimation methods are presented wtih computational approaches. Problems associated with testing fixed effects include accurate variance estimation and robustness to misspecification of the covariance structure. Methods for model selection and assessment are presented. An example is used to demonstrate recommended approaches.

Transgenerational patterns of suicide attempt.

The clinical assumption that suicide attempts demonstrate familial aggregation was examined with data from a survey of 2,304 community residents. Approximately 1 in 15 persons (6.6%) in the general population was aware that a parent, sibling, son, or daughter had attempted suicide. Self-reports of suicide attempts were more common among persons with than without a family history of suicide (13.0% vs. 2.8%, p less than .05). Nearly 1 in 4 persons who attempted suicide (24.8%) reported a family history of suicide. In addition to being female and unmarried, respondent mental disorder, parent mental disorder, and parent suicide attempt each exerted independent direct effects on the risk of respondent suicide ideation. Parent attempt was not significantly associated with respondent suicide attempt when a host of risk factors were held constant. Possible mediating factors are discussed, focusing on social isolation as an underlying factor.

Depression in the community: an investigation into age of onset.

Age of onset of any lifetime depressive disorder was investigated to identify periods of the life course associated with increased risk of depression. In this large community-based sample of adults, one fourth of those with a major depressive disorder at some point in their lives reported onset during childhood or adolescence; over one half reported onset by age 25. Women were likely to have an earlier onset of depression than were men; non-Hispanic Whites and Mexican Americans born in the United States reported earlier onset than did Mexican-American immigrants. Age of onset did not appear to be consequential in terms of the number, type, or severity of the symptoms occurring during the worst depressive episode or with the probability of relapse. These findings imply that age of onset may contribute to group differences in prevalence rates.