Selection of predictor variables for pneumonia using neural networks and genetic algorithms.

BACKGROUND: Artificial neural networks (ANN) can be used to select sets of predictor variable that incorporate nonlinear interactions between variables. We used a genetic algorithm, with selection based on maximizing network accuracy and minimizing network input-layer cardinality, to evolve parsimonious sets of variables for predicting community-acquired pneumonia among patients with respiratory complaints. METHODS: ANN were trained on data from 1044 patients in a training cohort, and were applied to 116 patients in a testing cohort. Chromosomes with binary genes representing input-layer variables were operated on by crossover recombination, mutation, and probabilistic selection based on a fitness function incorporating both network accuracy and input-layer cardinality. RESULTS: The genetic algorithm evolved best 10-variable sets that discriminated pneumonia in the training cohort (ROC areas, 0.838 for selection based on average cross entropy (ENT); 0.954 for selection based on ROC area (ROC)), and in the testing cohort (ROC areas, 0.847 for ENT selection; 0.963 for ROC selection), with no significant differences between cohorts. Best variable sets based on the genetic algorithm using ROC selection discriminated pneumonia more accurately than variable sets based on stepwise neural networks (ROC areas, 0.954 versus 0.879, p = 0.030), or stepwise logistic regression (ROC areas, 0.954 versus 0.830, p = 0.000). Variable sets of lower cardinalities were also evolved, which also accurately discriminated pneumonia. CONCLUSION: Variable sets derived using a genetic algorithm for neural networks accurately discriminated pneumonia from other respiratory conditions, and did so with greater accuracy than variables derived using stepwise neural networks or logistic regression in some cases.

Entering the black box of neural networks.

OBJECTIVES: Artificial neural networks have proved to be accurate predictive instruments in several medical domains, but have been criticized for failing to specify the information upon which their predictions are based. We used methods of relevance analysis and sensitivity analysis to determine the most important predictor variables for a validated neural network for community-acquired pneumonia. METHODS: We studied a feed-forward, back-propagation neural network trained to predict pneumonia among patients presenting to an emergency department with fever or respiratory complaints. We used the methods of full retraining, weight elimination, constant substitution, linear substitution, and data permutation to identify a consensus set of important demographic, symptom, sign, and comorbidity predictors that influenced network output for pneumonia. We compared predictors identified by these methods to those identified by a weight propagation analysis based on the matrices of the network, and by logistic regression. RESULTS: Predictors identified by these methods were clinically plausible, and were concordant with those identified by weight analysis, and by logistic regression using the same data. The methods were highly correlated in network error, and led to variable sets with errors below bootstrap 95% confidence intervals for networks with similar numbers of inputs. Scores for variable relevance tended to be higher with methods that precluded network retraining (weight elimination) or that permuted variable values (data permutation), compared with methods that permitted retraining (full retraining) or that approximated its effects (constant and linear substitution). CONCLUSION: Methods of relevance analysis and sensitivity analysis are useful for identifying important predictor variables used by artificial neural networks.

How physicians diagnose urinary tract infections: the potential influence of laboratory regulations on test availability and use.

OBJECTIVE: Physician office laboratory regulations may decrease test availability. We examined the potential effects of regulations on test availability and whether the use of tests in diagnosing uncomplicated urinary tract infections is related to availability. STUDY DESIGN: We performed an analysis of a cross-sectional survey conducted in 1994 and 1995. Test availability and use were determined by physicians' reports. POPULATION: The survey respondents included practicing physicians in 3 specialties (family medicine, general internal medicine, and obstetrics and gynecology) from 4 states: Pennsylvania (which had longstanding office laboratory regulations), and Alabama, Minnesota, and Nebraska (states that were not regulated until the implementation of the Clinical Laboratory Improvement Amendment of 1988). OUTCOMES MEASURED: We determined whether 4 specific tests were available in the office and how the tests were used to diagnose uncomplicated urinary tract infections. RESULTS: Our analysis was based on the responses from the 1898 respondents to the survey. All tests were less commonly available in Pennsylvania; this included the dipstick, microscopic urinalysis, wet prep, and urine culture (odds ratio [OR]=0.20-0.34; all P values < .05). The availability of the microscopic urinalysis and culture increased their use (OR = 4.37 and 2.03, respectively; P=.001). The availability of microscopic urinalysis was associated with a decrease in ordering urine cultures (OR=0.42; P=.001), and the availability of the dipstick was associated with a decrease in the use of both the microscopic urinalysis (OR=0.36; P=.02) and the culture (OR=0.48; P=.05). CONCLUSIONS: We found lower test availability in the state with office laboratory regulations and a decrease in testing when availability is reduced, suggesting that laboratory regulations may influence physicians&rsquo diagnostic approach to urinary tract infections. Further study will be required to determine the level of testing that maximizes patient welfare.

The effect of obesity on medical students' approach to patients with abdominal pain.

Because widely held stereotypes characterize obese people as less intelligent, unhappy, lacking in self control and more prone to psychological problems, we tested whether obese appearance alone would affect medical students' decisions about the diagnosis and management of simulated patients. We videotaped 4 patient simulators presenting each of 4 cases in 2 states: normal and obese (by using padding and bulky clothing). Seventy-two clinical students at 2 medical schools viewed the cases and answered questions about diagnostic tests and management. We found the expected biases toward patients when in their obese form as well as pessimism about patient compliance and success of therapy, but there were no significant differences in tests or treatments ordered except where appropriate for an obese patient (e.g., weight reduction diet). Thus, the appearance of obesity alone biased the students' impressions of the patients, but did not affect diagnostic test ordering.

Procedural experience and comfort level in internal medicine trainees.

BACKGROUND: The American Board of Internal Medicine (ABIM) has recommended a specific number of procedures be done as a minimum standard for ensuring competence in various medical procedures. These minimum standards were determined by consensus of an expert panel and may not reflect actual procedural comfort or competence. OBJECTIVE: To estimate the minimum number of selected procedures at which a majority of internal medicine trainees become comfortable performing that procedure. DESIGN: Cross-sectional, self-administered survey. SETTING: A military-based, a community-based, and 2 university-based programs. PARTICIPANTS: Two hundred thirty-two internal medicine residents. MEASUREMENTS: Survey questions included number of specific procedures performed, comfort level with performing specific procedures, and whether respondents desired further training in specific procedures. The comfort threshold for a given procedure was defined as the number of procedures at which two thirds or more of the respondents reported being comfortable or very comfortable performing that procedure. RESULTS: For three of seven procedures selected, residents were comfortable performing the procedure at or below the number recommended by the ABIM as a minimum requirement. However, residents needed more procedures than recommended by the ABIM to feel comfortable with central venous line placement, knee joint aspiration, lumbar puncture, and thoracentesis. Using multivariate logistic regression analysis, variables independently associated with greater comfort performing selected procedures included increased number performed, more years of training, male gender, career goals, and for skin biopsy, training in the community-based program. Except for skin biopsy, comfort level was independent of training site. A significant number of advanced-year house officers in some programs had little experience in performing selected common ambulatory procedures. CONCLUSION: Minimum standards for certifying internal medicine residents may need to be reexamined in light of house officer comfort level performing selected procedures.

Variation by specialty in the treatment of urinary tract infection in women.

To determine practicing physicians' strategies for diagnosing and managing uncomplicated urinary tract infection, we surveyed physicians in general internal medicine, family practice, obstetrics and gynecology, and emergency medicine in four states. Responses differed significantly by respondents' specialty. For example, nitrofurantoin was the antibiotic of first choice for 46% of obstetricians, while over 80% in the other specialties chose trimethoprim-sulfamethoxazole. Most surveyed said they do not usually order urine culture, but the percentage who do varied by specialty. Most use a colony count of 10(5) colony-forming units or more for diagnosis although evidence favors a lower threshold, and 70% continue antibiotic therapy even if the culture result is negative. This survey found considerable variation by specialty and also among individual physicians regarding diagnosis and treatment of urinary tract infection and also suggests that some of the new information from the literature has not been translated to clinical practice.

Factors explaining variability among caregivers in the intent to restrict life-support interventions in a pediatric intensive care unit.

OBJECTIVE: To explore patient-related factors which influence the decisions of pediatric intensive care unit (ICU) caregivers to restrict life-support interventions. DESIGN: Cross-sectional survey. SETTING: A university-affiliated pediatric ICU. SUBJECTS: All physicians and nurses caring for oncology or cardiology ICU patients. INTERVENTIONS: Seven patient characteristics were systematically presented in 16 theoretical case scenarios. MEASUREMENTS AND MAIN RESULTS: Individual linear regression models were constructed for each participant by calculating the importance caregivers placed on seven patient characteristics when deciding about starting intravenous vasopressors, performing chest compressions, and withdrawing life support. We compared the numerical and descriptive (very low, low, moderate, high) probability of survival. We surveyed 86 caregivers and 56 (65%) responded. The most important factors influencing decisions were family preferences (76% of decisions), followed by probability of survival (50%), and functional status (47%). There was marked variability among respondents in 38 (79%)/48 of the questions; 20% to 50% of caregivers chose opposing directions of patient management when they were asked to indicate the likelihood that they would perform a specific life-support intervention. The same term was never used by all respondents to describe the probability of survival for a scenario. CONCLUSION: Critically ill children and their families could face markedly different attitudes about the restriction of life-support interventions, depending on which nurses and physicians are involved in their care.

Attitudes and expectations of 1995 gastroenterology graduates about gastroenterology.

OBJECTIVE: To learn more about current attitudes and expectations of recent (June 1995) graduates of gastroenterology fellowship programs, why they chose either a private practice or academic career, and what impact managed care or health care reform had in their decision. METHODS: Between April and June 1995, and 8-page, 35-question survey questionnaire was mailed to graduating fellows and returned for evaluation. RESULTS: Graduates believed managed care had an impact on job availability, but it was not a factor in their job choice. Forty percent of the respondents reported that finding a job was either difficult or very difficult. The majority of respondents (67%) are pursuing a career in private practice. Most private practice physicians (PP) trained in 2-yr programs whereas academic physicians (AC) trained for the most part in 3-yr programs. The principal criteria on which decisions regarding job selection were based were similar between the two groups: co-workers, geographic location, access to patient care, and ability to perform endoscopy. Respondents in PP and AC expected to work 50-70 h/wk, care for patients with similar diseases, and have ample time for family. They would choose GI again as a career and believed that there is a future in GI. Salary expectations varied markedly between the two groups, and AC physicians were more concerned about their future financial needs. Twenty percent of PP physicians and 71% of AC physicians plan to participate in clinical research. CONCLUSIONS: Recent graduates of gastroenterology fellowship programs continue to have high expectations of their future careers. Although some had difficulty finding a job and stated that, although managed care had an impact on the job market, it had not yet become a major factor in their job selection.

Advance directives: a policy-capturing approach.

Policy capturing was used to provide insight into those factors that people consider important in the decision of whether to accept life-sustaining medical treatment. First, open-ended interviews with community-dwelling elderly persons (n = 30) were conducted to determine the factors they would consider when drafting an advance directive. College students (n = 53) then made judgments as to whether they would accept life-sustaining treatment for each of 100 hypothetical vignettes comprising a similar set of factors. Results revealed that 1) students made consistent judgments, 2) there was considerable variability in their mean judgments, 3) the most influential factors were mental and physical functioning, 4) mental and physical functioning had an interactive effect on judgments, and 5) subjective estimates of importance were significantly related to policy-capturing weights. This approach for studying the relationship of individuals' values to their acceptance of life-sustaining therapy may be useful in future studies of patient and surrogate decision making.

Procedural skills of practicing pulmonologists. A national survey of 1,000 members of the American College of Physicians.

We surveyed pulmonologists to determine which procedures they do in practice, where they learned the procedures, and how much training they recommend to attain and maintain clinical competence in each. We mailed a survey to a random sample of 1,000 members of the American College of Physicians who were identified as practicing pulmonologists; 755 (75%) responded. Respondents performed a variety of pulmonary procedures, an average of 17 of the 29 listed. Pulmonologists who were more recent graduates, who worked longer hours, and who were involved in critical care did a greater variety of procedures. Only 26% of practicing pulmonologists currently do all the procedures required for board certification in pulmonary medicine. For each of 13 specific procedures, the number reported done in the past year was generally unrelated to practice factors. Many respondents who learned procedures in practice did so without formal training or supervision. Respondents' recommendations regarding numbers of procedures required to attain or maintain competence did not vary greatly. Pulmonologists vary considerably in the types of procedures they do. Their opinions about the training needed for competence help to better define requirements for training programs. More attention should be focused on training and certifying practicing pulmonologists in procedures learned after formal fellowship training.

You can lead a horse to water--improving physicians' knowledge of probabilities may not affect their decisions.

OBJECTIVES: To determine whether improving physicians' judgments of the probability of streptococcal pharyngitis for patients with sore throats would affect their use of antibiotics and affect the variation in such use. DESIGN: Post-hoc retrospective analysis of data previously collected as part of a controlled trial. SETTINGS: University student health services in Pennsylvania and Nebraska. PATIENTS: Sequential patients with pharyngitis seen before and after the time clinicians received either an experimental educational intervention designed to improve probabilistic diagnostic judgments (at the Pennsylvania site) or a control intervention, a standard lecture (at the Nebraska site). The clinician-subjects were the primary case physicians practicing at either site. MEASUREMENTS: Clinical variables prospectively recorded by the clinicians, probability assessments, and treatment decisions. RESULTS: At the experimental site, despite marked decreases in clinicians' overestimations of disease probability after the intervention, the proportion of patients prescribed antibiotics showed a trend toward increasing: 100/290 (34.5%) pre-intervention, 90/225 (40%) post-intervention. The intervention did not decrease practice variation between individual doctors. Univariable and multivariable analyses showed no major change in the relationships between clinical variables and treatment decisions after the intervention. At the control site there was no major change in probability judgments or treatment decisions after the intervention. CONCLUSIONS: Teaching physicians to make better judgments of disease probability may not alter their treatment decisions.

Practice variation in the management of pharyngitis: the importance of variability in patients' clinical characteristics and in physicians' responses to them.

The objective of this study was to assess whether geographic differences in antibiotic-prescribing rates for patients with pharyngitis could be explained by intersite differences in patients' clinical characteristics and in how physicians responded to these clinical cues when making decisions. As part of the initial phase of a prospective controlled trial to improve physicians' diagnostic ability, the authors enrolled cohorts of consecutive patients seen at staff-model--HMO student health services in Pennsylvania and Nebraska. Physicians' decisions whether to prescribe antibiotics for 310 consecutive patients presenting with pharyngitis to the former and 214 such patients presenting to the latter at the time of the initial visit were examined. There was a large discrepancy between the antibiotic-prescribing rates at the student health services in Pennsylvania, 106/310, 32.4%, and Nebraska, 156/214, 72.9%. The clinical variables significantly independently associated with treatment at both sites in a logistic regression model were fever, adjusted odds ratio = 2.1 (95% CI = 1.1, 3.8); exudates, 5.4 (2.8, 10); palatine petechiae, 6.5 (1.5, 28); rhinorrhea, 0.46, (0.25, 0.85); and high risk of complications, 3.8 (1.04, 14). There was a significant interaction between site and anterior cervical adenopathy, 5.5 (1.6, 19); and a borderline interaction between site and rhinorrhea, 2.4 (0.89, 6.7). Site was not a significant independent predictor of treatment, 1.8 (0.45, 6.6.). Practice variation was related to geographic differences in patients' clinical characteristics and in how physicians responded to these factors when prescribing antibiotics. How physicians weight patients' clinical characteristics when making decisions may be an important element of their "practice styles."

Discrimination and reproducibility of an information maximizing multivariable model.

Predictor variables for multivariate rules are frequently selected by methods that maximize likelihood rather than information. We compared the discrimination and reproducibility of a prediction rule for pneumonia derived using extended dependency analysis (EDA), an information maximizing variable selection program, with that of a validated rule derived using logistic regression. Discrimination was measured by receiver-operating characteristic (ROC) analysis, and reproducibility by rederivation of the rule on 200 replicate samples of size 250 and 500, generated from a training cohort of 905 patients using Monte Carlo techniques. Four of the five predictor variables selected by EDA were identical to those selected by logistic regression. With each variable weighted by its conditional contribution to total information transmission, EDA discriminated pneumonia and nonpneumonia in the training cohort with an ROC area of 0.800 (vs 0.816 for logistic regression, p = 0.60), and in the validation cohort with an area of 0.822 (vs 0.821 for logistic regression, p = 0.98). EDA demonstrated reproducibility comparable to that of logistic regression according to most criteria for replicability. Replicate EDA models showed good discrimination in the training and testing cohorts, and met statistical criteria for validation (no significant difference in ROC areas at a one-tailed alpha level of 0.05) in 80.8% to 94.2% of cases. We conclude that extended dependency analysis selected the most important variables for predicting pneumonia, based on a validated logistic regression model. The information-theoretic model showed good discriminatory power, and demonstrated reproducibility according to clinically reasonable criteria. Information-theoretic variable selection by extended dependency analysis appears to be a reasonable basis for developing clinical prediction rules.

The influence of treatment descriptions on advance medical directive decisions.

OBJECTIVE: To determine whether the wording of the descriptions of life-sustaining interventions would affect the choices elderly patients make when completing advance directives. METHODS: Survey. SETTING: General community in Omaha, Nebraska. PATIENTS: Two hundred one community-dwelling elderly were selected from a population-based sample. MAIN OUTCOME MEASURES: Subjects were asked whether they would accept or reject three life-sustaining interventions: cardiopulmonary resuscitation, mechanical ventilation, or tube feeding in three separate hypothetical case scenarios. The three life-sustaining interventions were each described positively, negatively, and exactly as they are worded in a widely used advance directive. Subjects reviewed each scenario three times with three different descriptions of the three interventions. RESULTS: For the three interventions presented in three scenarios, subjects opted for the intervention 12 percent of the time when it was presented negatively, 18 percent of the time when it was phrased as in an advance directive already in use and 30 percent of the time it was phrased positively. One hundred fifty-five of the 201 subjects (77 percent) changed their minds at least once when given the same scenario but a different description of the intervention. Of these 155, 33 percent changed decisions one to three times, 33 percent changed decisions four to seven times, and another 34 percent changed decisions eight to seventeen times based solely on the description of the intervention. CONCLUSION: The decisions patients make about whether to accept or reject life-sustaining treatments are affected by the descriptions of the treatments. These findings emphasize the critical importance of doctor-patient consultation when patients execute advance directives.