The log multinomial regression model for nominal outcomes with more than two attributes.

An estimate of the risk or prevalence ratio, adjusted for confounders, can be obtained from a log binomial model (binomial errors, log link) fitted to binary outcome data. We propose a modification of the log binomial model to obtain relative risk estimates for nominal outcomes with more than two attributes (the "log multinomial model"). Extensive data simulations were undertaken to compare the performance of the log multinomial model with that of an expanded data multinomial logistic regression method based on the approach proposed by Schouten et al. (1993) for binary data, and with that of separate fits of a Poisson regression model based on the approach proposed by Zou (2004) and Carter, Lipsitz and Tilley (2005) for binary data. Log multinomial regression resulted in "inadmissable" solutions (out-of-bounds probabilities) exceeding 50% in some data settings. Coefficient estimates by the alternative methods produced out-of-bounds probabilities for the log multinomial model in up to 27% of samples to which a log multinomial model had been successfully fitted. The log multinomial coefficient estimates generally had lesser relative bias and mean squared error than the alternative methods. The practical utility of the log multinomial regression model was demonstrated with a real data example. The log multinomial model offers a practical solution to the problem of obtaining adjusted estimates of the risk ratio in the multinomial setting, but must be used with some care and attention to detail.

Parameter estimation and goodness-of-fit in log binomial regression.

An estimate of the risk, adjusted for confounders, can be obtained from a fitted logistic regression model, but it substantially over-estimates when the outcome is not rare. The log binomial model, binomial errors and log link, is increasingly being used for this purpose. However this model's performance, goodness of fit tests and case-wise diagnostics have not been studied. Extensive simulations are used to compare the performance of the log binomial, a logistic regression based method proposed by Schouten et al. (1993) and a Poisson regression approach proposed by Zou (2004) and Carter, Lipsitz, and Tilley (2005). Log binomial regression resulted in "failure" rates (non-convergence, out-of-bounds predicted probabilities) as high as 59%. Estimates by the method of Schouten et al. (1993) produced fitted log binomial probabilities greater than unity in up to 19% of samples to which a log binomial model had been successfully fit and in up to 78% of samples when the log binomial model fit failed. Similar percentages were observed for the Poisson regression approach. Coefficient and standard error estimates from the three models were similar. Rejection rates for goodness of fit tests for log binomial fit were around 5%. Power of goodness of fit tests was modest when an incorrect logistic regression model was fit. Examples demonstrate the use of the methods. Uncritical use of the log binomial regression model is not recommended.

A simplified method of calculating an overall goodness-of-fit test for the Cox proportional hazards model.

Grønnesby and Borgan (1996) propose an overall goodness-of-fit test for the Cox proportional hazards model. The basis of their test is a grouping of subjects by their estimated risk score. We show that the Grønnesby and Borgan test is algebraically identical to one obtained from adding group indicator variables to the model and testing the hypothesis the coefficients of the group indicator variables are zero via the score test. Thus showing that the test can be calculated using existing software. We demonstrate that the table of observed and estimated expected number of events within each group of the risk score is a useful adjunct to the test to help identify potential problems in fit.

Empirical comparisons of proportional hazards, poisson, and logistic regression modeling of occupational cohort data.

This research was conducted to examine the effect of model choice on the epidemiologic interpretation of occupational cohort data. Three multiplicative models commonly employed in the analysis of occupational cohort studies--proportional hazards. Poisson, and logistic regression--were used to analyze data from an historical cohort study of workers exposed to formaldehyde. Samples were taken from this dataset to create a number of predetermined scenarios for comparing the models, varying study size, outcome frequency, strength of risk factors, and follow-up length. The Poisson and proportional hazards models yielded nearly identical relative risk estimates and confidence intervals in all situations except when confounding by age could not be closely controlled in the Poisson analysis. Logistic regression findings were more variable, with risk estimates differing most from the proportional hazards results when there was a common outcome or strong relative risk. The logistic model also provided less precise estimates than the other two. Thus, although logistic was the easiest model to implement, it should be used only in occupational cohort studies when the outcome is rare (5% or less), and the relative risk is less than approximately 2. Even then, the proportional hazards and Poisson models are better choices. Selecting between these two can be based on convenience in most circumstances.

A comparison of goodness-of-fit tests for the logistic regression model.

Recent work has shown that there may be disadvantages in the use of the chi-square-like goodness-of-fit tests for the logistic regression model proposed by Hosmer and Lemeshow that use fixed groups of the estimated probabilities. A particular concern with these grouping strategies based on estimated probabilities, fitted values, is that groups may contain subjects with widely different values of the covariates. It is possible to demonstrate situations where one set of fixed groups shows the model fits while the test rejects fit using a different set of fixed groups. We compare the performance by simulation of these tests to tests based on smoothed residuals proposed by le Cessie and Van Houwelingen and Royston, a score test for an extended logistic regression model proposed by Stukel, the Pearson chi-square and the unweighted residual sum-of-squares. These simulations demonstrate that all but one of Royston's tests have the correct size. An examination of the performance of the tests when the correct model has a quadratic term but a model containing only the linear term has been fit shows that the Pearson chi-square, the unweighted sum-of-squares, the Hosmer-Lemeshow decile of risk, the smoothed residual sum-of-squares and Stukel's score test, have power exceeding 50 per cent to detect moderate departures from linearity when the sample size is 100 and have power over 90 per cent for these same alternatives for samples of size 500. All tests had no power when the correct model had an interaction between a dichotomous and continuous covariate but only the continuous covariate model was fit. Power to detect an incorrectly specified link was poor for samples of size 100. For samples of size 500 Stukel's score test had the best power but it only exceeded 50 per cent to detect an asymmetric link function. The power of the unweighted sum-of-squares test to detect an incorrectly specified link function was slightly less than Stukel's score test. We illustrate the tests within the context of a model for factors associated with low birth weight.

Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit.

OBJECTIVE: To analyze the effects of patient mix diversity on performance of an intensive care unit (ICU) severity-of-illness model. DESIGN: Multiple patient populations were created using computer simulations. A customized version of the Mortality Probability Model (MPM) II admission model was used to ascertain probabilities of hospital mortality. Performance of the model was assessed using discrimination (area under the receiver operating characteristic curve) and calibration (goodness-of-fit testing). SETTING: Intensive care units. PATIENTS: Data were collected from 4,224 ICU patients from two Massachusetts hospitals (Baystate Medical Center, Springfield, MA; University of Massachusetts Medical Center, Worcester, MA) and two New York hospitals (Albany Medical Center, Albany, NY; Ellis Hospital, Schenectady, NY). INTERVENTIONS: Random samples were taken from a database. The percentage of patients with each model disease characteristic was varied by assigning weights (ranging from 0 to 10) to patients with a disease characteristic. Three simulations were run for each of 15 model variables at each of 16 weights, totaling 720 simulations. MEASUREMENTS AND MAIN RESULTS: The area under the receiver operating characteristic curve and model fit were assessed in each random sample. Removing patients with a given disease characteristic did not affect discrimination or calibration. Increasing frequency of patients with each disease characteristic above the original frequency caused discrimination and calibration to deteriorate. Model fit was more robust to increases in less frequently occurring patient conditions. From the goodness-of-fit test, a critical percentage for each admission model variable was determined for each disease characteristic, defined as the percentage at which the average p value for the test over the three replications decreased to < .10. CONCLUSIONS: The concept of critical percentages is potentially clinically important. It might provide an easy first step in checking applicability of a given severity-of-illness model and in defining a general medical-surgical ICU. If the critical percentages are exceeded, as might occur in a highly specialized ICU, the model would not be accurate. Alternative modeling approaches might be to customize the model coefficients to the population for more accurate probabilities or to develop specialized models. The MPM approach remained robust for a large variation in patient mix factors.

Confidence intervals for measures of interaction.

Interaction, defined as departure of disease rates from an additive model, can be measured by the relative excess risk due to interaction, or the attributable proportion due to interaction. Point estimates can be obtained using multiple logistic regression. Using simulated case-control data, we compare several confidence interval estimation techniques for these measures. These include a symmetrical interval based on the delta method estimate of the variance, and three types of bootstrap confidence intervals. One such bootstrap method has coverage closest to the nominal level and is the most evenly balanced with respect to the direction in which intervals miss the true value. The estimation methods are applied to data from an actual case-control study, and the results are interpreted in light of the simulation study.

The impact of clinical trials on the use of medications for acute myocardial infarction. Results of a community-based study.

BACKGROUND: The impact of clinical trials on medical practice remains controversial, in part because of weak study designs and nonrepresentative study samples. OBJECTIVE: To assess changes in trends in medication use in the setting of acute myocardial infarction (AMI) before and after publication of two large clinical trials: the Second International Study of Infarct Survival (ISIS-2) trial that supported the use of aspirin after AMI and the Multi-center Diltiazem Postinfarction Trial that reported no overall benefit from the use of calcium antagonists after AMI. METHODS: Study patients consisted of 2114 patients hospitalized with AMI in 16 hospitals in metropolitan Worcester, Mass, during 1986, 1988, and 1990. Data were obtained from medical records. We used multivariable logistic regression models to examine the rate of change in the use of selected medications before and after trial publication, controlling for medical history, characteristics and complications of AMI, medications taken, and procedures performed during hospitalization. The dependent variable was receipt of the specific medication under investigation. RESULTS: Before publication of ISIS-2, 26% of patients with AMI received aspirin while hospitalized compared with 66% after its publication. However, in-hospital aspirin use began to rise before ISIS-2 with an immediate increase in the level of use occurring after trial publication but with no significant change in the rate of increase. Before publication of the Multicenter Diltiazem Postinfarction Trial, 57% of patients with AMI were new recipients of calcium antagonists compared with 51% after trial publication. The decrease in calcium antagonist use began after trial publication (odds ratio, 0.79 per 6-month period; 95% confidence interval, 0.71 to 0.88). CONCLUSIONS: The published results of large trials of cardiovascular therapies have had variable impact on medication use. Efforts to assess the effects of publication of new scientific information on medical care need to consider prior trends in treatment patterns and the varying contexts of medical care. They should consider both direct and indirect routes of influence.

Factors affecting the performance of the models in the Mortality Probability Model II system and strategies of customization: a simulation study.

OBJECTIVES: To examine the impact of hospital mortality and intensive care unit (ICU) size on the performance of the Mortality Probability Model II system for use in quality assessment, and to examine the ability of model customization to produce accurate estimates of hospital mortality to characterize patients by severity of illness for clinical trials. DESIGN: Prospective evaluation of model performance, using retrospective data. SETTING: Data for the simulation were assembled from six adult medical and surgical ICUs in Massachusetts and New York. PATIENTS: Consecutive admissions (n = 4,224) to the Massachusetts and New York ICUs were studied. The mortality rate in the database was 18.7%. INTERVENTIONS: A computer simulation of several different hospital mortality rates and ICU sample sizes, using a multicenter database of consecutive ICU admissions, was utilized. We simulated 20 different mortality rates by randomly changing the outcomes at hospital discharge from "survived" to "deceased" and from "deceased" to "survived". Four sample size simulations used 75%, 50%, 25%, and 10% of the database. Ten replications of each mortality rate and samples size were constructed, and model calibration and discrimination were assessed for each replication. Model coefficients were customized, using logistic regression. MEASUREMENTS AND MAIN RESULTS: Vital status at hospital discharge was the outcome measure among the ICU patient population. Model performance was assessed using the Hosmer-Lemeshow C statistic for calibration, and the area under the receiver operating characteristic curve for discrimination. Goodness-of-fit tests and receiver operating characteristic curve areas demonstrated that the models were sensitive to differences in hospital mortality, indicating that they are useful quality assurance tools. Goodness-of-fit tests were more sensitive than the receiver operating characteristic curve areas. The further the hospital mortality rate diverged from the original rate, the worse the performance of the model. Sample size had an impact on these results. The smaller the sample size, the less likely the model was to perform poorly. Model coefficients were successfully customized to demonstrate that improved model performance can be achieved when necessary for clinical trial stratification. CONCLUSION: Mortality Probability Model II models can be used to assess quality of care in ICUs, but the size of the sample should be considered when assessing calibration and discrimination.

Confidence interval estimates of an index of quality performance based on logistic regression models.

This paper considers an index of hospital quality performance defined as the ratio of the observed number deaths to the number predicted by a fitted logistic regression model. We study tests and confidence intervals under two different scenarios depending on the availability of an estimate of the covariance matrix of the coefficients from the fitted logistic regression model. We propose parametric as well as bootstrap-based confidence intervals. We apply the methods to an analysis of the performance of 27 intensive care units.

Survey of methods and statistical models used in the analysis of occupational cohort studies.

OBJECTIVES: This survey was conducted to determine the frequency with which different data analysis techniques are being used in occupational cohort studies. Of particular interest was the relative use of external and internal comparison groups, and the choice of multivariable model. METHODS: Occupational cohort studies published in 1990-91 were located with Medline and Index Medicus, and the contents of several relevant journals were systematically reviewed. Each study was categorised by the methods of external or internal comparisons performed. RESULTS: Of 200 occupational cohort studies identified, 104 (52%) conducted only external comparisons, 46 (23%) conducted only internal, and 50 (25%) presented both. Of those that used an external referent population, about two thirds used a national standard. 40 of the studies that performed internal comparisons fitted multivariable models, with use divided about equally between proportional hazards regression, Poisson regression, and logistic regression. DISCUSSION: The finding that logistic regression is used quite commonly, even though it does not directly model time dependent data of the type frequently encountered in occupational cohort studies, was suprising. The reasons why investigators choose from among the available statistical and modelling techniques are likely to include familiarity, ease of use, restrictions in study population characteristics, especially study size, and others. Authors should be encouraged to be more explicit about the statistical methods used in the analysis of occupational cohort studies, as well as whether important assumptions about their data have been evaluated.

Estimates of blood pressure variability in adolescents using an automatic blood pressure measuring device.

Knowledge of BP variability is important for BP screening and for the design of studies with BP as an outcome. We calculated estimates of the between-visit and within-visit variances from data obtained using an automated BP.device (Dinamap) in 776 students, aged 13-17 years, in the Exeter-Andover Project. Each subject had his or her BP measured three times per visit at up to 24 weekly visits. We compared these estimates of variance with those published for adolescents using a standard mercury sphygmomanometer. For subjects measured using the Dinamap, the between-visit variance was greater for SBP and the within-visit variance for both pressures was almost three times greater than for subjects measured with the standard instrument.

Changing clinical practice. Prospective study of the impact of continuing medical education and quality assurance programs on use of prophylaxis for venous thromboembolism.

OBJECTIVE: To determine the effect of continuing medical education (CME) with and without a quality assurance component (CME+QA) on physician practices in the prevention of venous thromboembolism. METHODS: A communitywide study was performed in 15 short-stay hospitals in central Massachusetts. The study population included 3158 patients in acute-care hospitals with multiple risk factors for venous thromboembolism. Study hospitals were randomly assigned to one of two educational strategies or to a control group that received no intervention. RESULTS: The proportion of patients at high risk for venous thromboembolism who received effective methods of prophylaxis increased significantly from 29% in 1986 to 52% in 1989 (P < .001). This increase was seen in all study groups: control hospitals, 40% to 51% (P < .001); CME hospitals, 21% to 49% (P < .0001); and CME+QA hospitals, 27% to 55% (P < .0001). The increase in prophylaxis use from 1986 to 1989 was significantly greater among patients cared for in hospitals whose physicians participated in a formal CME program (an increase of 28%) than in control hospitals (an increase of 11%) (P < .001). There was no significant difference in the use of prophylaxis in hospitals whose physicians received CME+QA interventions compared with hospitals whose physicians received CME interventions alone (identical increases of 28%). CONCLUSION: A formal CME program significantly increased the frequency with which physicians prescribed prophylaxis for venous thromboembolism. We believe the key factor in our CME interventions that motivated clinicians to change their practices was the provision of hospital-specific data demonstrating a compelling need for improvement. Despite the substantial investment by hospitals in QA, traditional QA intervention appeared to provide no additional benefit. Even after extensive CME/QA interventions, prophylaxis for venous thromboembolism remained underutilized, suggesting the need to develop new approaches to changing clinical practice.

Distance of response to host tree models by female apple maggot flies,Rhagoletis pomonella (Walsh) (Diptera: Tephritidae): Interaction of visual and olfactory stimuli.

Mature female apple maggot flies,Rhagoletis pomonella (Walsh), were released individually onto a single potted, fruitless hawthorne tree in the center of an open field. The tree was surrounded by four 1-m(2) plywood host tree models painted green or white, with or without synthetic host fruit odor (butyl hexanoate), and placed at one of several distances from the release tree. Each fly was permitted to forage freely on the release tree for up to 1 hr, or until it left the tree. Flies left the tree significantly sooner when green models with host fruit were present at 0.5, 1.5, or 2.5 m distance from the release tree than when these models were placed at a greater distance (4.5 m) from the release tree or when no models were present. Flies responded detectably to 1-m(2) models without odor up to a maximum distance of 1.5 m. These results suggest that female apple maggot flies did not detect green 1-m(2) models with odor 4.5 m away or models without odor 2.5 m or more away. Flies responded to white models with and without odor to a much lesser extent, both in terms of response distance and flight to and alightment upon models. Increasing model size to 2 m(2) increased the distance to 2.5 m at which flies responded to green models without odor. Decreasing model size to 0.5 m(2) reduced fly responsiveness to green or white models. The presence of host fruit odor alone, without the visual stimulus of a green model, did not influence residence time on the release tree.

A communitywide perspective of sex differences and temporal trends in the incidence and survival rates after acute myocardial infarction and out-of-hospital deaths caused by coronary heart disease.

BACKGROUND: The purpose of the study was to examine overall differences and temporal trends therein between men and women regarding the incidence rates, in-hospital and long-term survival after initial acute myocardial infarction (AMI), and out-of-hospital deaths caused by coronary disease. METHODS AND RESULTS: This nonconcurrent prospective study was carried out in 16 teaching and community hospitals in Worcester, Mass., in six time periods between 1975 and 1988. A total of 3,148 patients hospitalized with validated initial AMI comprised the study sample. The age-adjusted incidence rates of initial AMI increased between 1975 and 1981 in the two sexes, with a marked decrease thereafter; these rates declined by 26% in men and by 22% in women between 1975 and 1988. The overall unadjusted in-hospital case-fatality rates after initial AMI were significantly higher in women (21.7%) than in men (12.7%). Age- and multivariable-adjusted in-hospital case-fatality rates, however, were not significantly different for men compared with women (multivariate-adjusted OR, 0.90; 95% CI, 0.70, 1.16). No clear trends in in-hospital case-fatality rates were observed in men or women over the periods under study. There were no significant sex differences in the age-adjusted long-term survival rates of discharged hospital survivors of AMI. The multivariate-adjusted risk of total mortality among discharged hospital survivors, however, was significantly increased in men (multivariate-adjusted OR, 1.20; 95% CI, 1.03, 1.39); neither of the sexes experienced an improvement over time in long-term prognosis. The incidence rates of out-of-hospital deaths caused by coronary disease declined by 60% in men and 69% in women between 1975 and 1988. CONCLUSIONS: The results of this multihospital, community-based study suggest declines in the incidence rates of AMI and out-of-hospital deaths caused by coronary disease in men and women over the period under study (1975-1988). No significant sex differences in in-hospital survival were observed, whereas a poorer long-term survival experience after hospital discharge was observed for men compared with women after controlling for potentially confounding prognostic factors.

Factors associated with heavy smoking among men and women: the physician-delivered smoking intervention project.

As part of a randomized trial that compared the effects of three physician-delivered smoking interventions on patients' long-term cessation rates, we examined factors associated with the extent of baseline cigarette smoking separately in 546 men and 715 women who were enrolled in this trial. Several baseline characteristics were significantly related to heavier (> or = 25 cigarettes per day) smoking after controlling for a variety of factors in multivariate analyses, which were performed separately for men and women. Among both men and women, extent of addictiveness to smoking and number of cigarettes smoked during periods of heaviest smoking were significantly related to extent of current cigarette smoking (p < 0.001). In addition, among men shortness of breath, lack of previous attempts to quit and lack of confidence in their ability to stop smoking were significantly related to current heavy cigarette smoking (p < 0.05). The results of this study in ambulatory outpatients suggest a profile of heavy cigarette smokers that may be used for the more effective delivery of targeted smoking intervention efforts.

Comparison of patients' and their resident physicians' responses regarding smoking-cessation interventions.

PURPOSE: To investigate (1) the extent of agreement between what resident physicians and their patients report as having occurred in physician-delivered smoking interventions and (2) the ability of residents to effectively transmit information concerning smoking interventions to their patients. METHODS: A total of 263 patients and 91 residents in internal medicine or family practice completed paper-and-pencil exit interviews after a regularly scheduled clinic appointment between 1986 and 1988 at the University of Massachusetts Medical School; the residents had been trained to deliver counseling interventions. The kappa statistic was used as an index of chance-corrected agreement between the patients' and residents' responses. RESULTS: Agreement was substantial regarding whether a specific plan for the patient to stop or reduce smoking was agreed upon, whether written materials on how to quit smoking were provided, and whether nicotine-containing chewing gum was prescribed. CONCLUSIONS: There was positive agreement between the patients and their resident physicians concerning the residents' delivery of quit-smoking messages and the provision of written materials to assist in stopping. Programs must continue to be designed, for residents and for more senior physicians, so that physicians can be encouraged to incorporate smoking interventions into their practice activities.

Confidence interval estimation of interaction.

Relative excess risk due to interaction, the proportion of disease among those with both exposures that is attributable to their interaction, and the synergy index have been proposed as measures of interaction in epidemiologic studies. This paper presents the methodology for obtaining confidence interval estimates of these indices utilizing routinely available output from multiple logistic regression software.