Comparative performance of intensive care mortality prediction models based on manually curated versus automatically extracted electronic health record data.

INTRODUCTION: Benchmarking intensive care units for audit and feedback is frequently based on comparing actual mortality versus predicted mortality. Traditionally, mortality prediction models rely on a limited number of input variables and significant manual data entry and curation. Using automatically extracted electronic health record data may be a promising alternative. However, adequate data on comparative performance between these approaches is currently lacking. METHODS: The AmsterdamUMCdb intensive care database was used to construct a baseline APACHE IV in-hospital mortality model based on data typically available through manual data curation. Subsequently, new in-hospital mortality models were systematically developed and evaluated. New models differed with respect to the extent of automatic variable extraction, classification method, recalibration usage and the size of collection window. RESULTS: A total of 13 models were developed based on data from 5,077 admissions divided into a train (80%) and test (20%) cohort. Adding variables or extending collection windows only marginally improved discrimination and calibration. An XGBoost model using only automatically extracted variables, and therefore no acute or chronic diagnoses, was the best performing automated model with an AUC of 0.89 and a Brier score of 0.10. DISCUSSION: Performance of intensive care mortality prediction models based on manually curated versus automatically extracted electronic health record data is similar. Importantly, our results suggest that variables typically requiring manual curation, such as diagnosis at admission and comorbidities, may not be necessary for accurate mortality prediction. These proof-of-concept results require replication using multi-centre data.

Development and evaluation of regression tree models for predicting in-hospital mortality of a national registry of COVID-19 patients over six pandemic surges.

BACKGROUND: Objective prognostic information is essential for good clinical decision making. In case of unknown diseases, scarcity of evidence and limited tacit knowledge prevent obtaining this information. Prediction models can be useful, but need to be not only evaluated on how well they predict, but also how stable these models are under fast changing circumstances with respect to development of the disease and the corresponding clinical response. This study aims to provide interpretable and actionable insights, particularly for clinicians. We developed and evaluated two regression tree predictive models for in-hospital mortality of COVID-19 patient at admission and 24 hours (24 h) after admission, using a national registry. We performed a retrospective analysis of observational routinely collected data. METHODS: Two regression tree models were developed for admission and 24 h after admission. The complexity of the trees was managed via cross validation to prevent overfitting. The predictive ability of the model was assessed via bootstrapping using the Area under the Receiver-Operating-Characteristic curve, Brier score and calibration curves. The tree models were assessed on the stability of their probabilities and predictive ability, on the selected variables, and compared to a full-fledged logistic regression model that uses variable selection and variable transformations using splines. Participants included COVID-19 patients from all ICUs participating in the Dutch National Intensive Care Evaluation (NICE) registry, who were admitted at the ICU between February 27, 2020, and November 23, 2021. From the NICE registry, we included concerned demographic data, minimum and maximum values of physiological data in the first 24 h of ICU admission and diagnoses (reason for admission as well as comorbidities) for model development. The main outcome measure was in-hospital mortality. We additionally analysed the Length-of-Stay (LoS) per patient subgroup per survival status. RESULTS: A total of 13,369 confirmed COVID-19 patients from 70 ICUs were included (with mortality rate of 28%). The optimism-corrected AUROC of the admission tree (with seven paths) was 0.72 (95% CI: 0.71-0.74) and of the 24 h tree (with 11 paths) was 0.74 (0.74-0.77). Both regression trees yielded good calibration and variable selection for both trees was stable. Patient subgroups comprising the tree paths had comparable survival probabilities as the full-fledged logistic regression model, survival probabilities were stable over six COVID-19 surges, and subgroups were shown to have added predictive value over the individual patient variables. CONCLUSIONS: We developed and evaluated regression trees, which operate at par with a carefully crafted logistic regression model. The trees consist of homogenous subgroups of patients that are described by simple interpretable constraints on patient characteristics thereby facilitating shared decision-making.

Can we reliably automate clinical prognostic modelling? A retrospective cohort study for ICU triage prediction of in-hospital mortality of COVID-19 patients in the Netherlands.

BACKGROUND: Building Machine Learning (ML) models in healthcare may suffer from time-consuming and potentially biased pre-selection of predictors by hand that can result in limited or trivial selection of suitable models. We aimed to assess the predictive performance of automating the process of building ML models (AutoML) in-hospital mortality prediction modelling of triage COVID-19 patients at ICU admission versus expert-based predictor pre-selection followed by logistic regression. METHODS: We conducted an observational study of all COVID-19 patients admitted to Dutch ICUs between February and July 2020. We included 2,690 COVID-19 patients from 70 ICUs participating in the Dutch National Intensive Care Evaluation (NICE) registry. The main outcome measure was in-hospital mortality. We asessed model performance (at admission and after 24h, respectively) of AutoML compared to the more traditional approach of predictor pre-selection and logistic regression. FINDINGS: Predictive performance of the autoML models with variables available at admission shows fair discrimination (average AUROC = 0·75-0·76 (sdev = 0·03), PPV = 0·70-0·76 (sdev = 0·1) at cut-off = 0·3 (the observed mortality rate), and good calibration. This performance is on par with a logistic regression model with selection of patient variables by three experts (average AUROC = 0·78 (sdev = 0·03) and PPV = 0·79 (sdev = 0·2)). Extending the models with variables that are available at 24h after admission resulted in models with higher predictive performance (average AUROC = 0·77-0·79 (sdev = 0·03) and PPV = 0·79-0·80 (sdev = 0·10-0·17)). CONCLUSIONS: AutoML delivers prediction models with fair discriminatory performance, and good calibration and accuracy, which is as good as regression models with expert-based predictor pre-selection. In the context of the restricted availability of data in an ICU quality registry, extending the models with variables that are available at 24h after admission showed small (but significantly) performance increase.

Comparison of outcome and characteristics between 6343 COVID-19 patients and 2256 other community-acquired viral pneumonia patients admitted to Dutch ICUs.

PURPOSE: Describe the differences in characteristics and outcomes between COVID-19 and other viral pneumonia patients admitted to Dutch ICUs. MATERIALS AND METHODS: Data from the National-Intensive-Care-Evaluation-registry of COVID-19 patients admitted between February 15th and January 1th 2021 and other viral pneumonia patients admitted between January 1st 2017 and January 1st 2020 were used. Patients' characteristics, the unadjusted, and adjusted in-hospital mortality were compared. RESULTS: 6343 COVID-19 and 2256 other viral pneumonia patients from 79 ICUs were included. The COVID-19 patients included more male (71.3 vs 49.8%), had a higher Body-Mass-Index (28.1 vs 25.5), less comorbidities (42.2 vs 72.7%), and a prolonged hospital length of stay (19 vs 9 days). The COVID-19 patients had a significantly higher crude in-hospital mortality rate (Odds ratio (OR) = 1.80), after adjustment for patient characteristics and ICU occupancy rate the OR was respectively 3.62 and 3.58. CONCLUSION: Higher mortality among COVID-19 patients could not be explained by patient characteristics and higher ICU occupancy rates, indicating that COVID-19 is more severe compared to other viral pneumonia. Our findings confirm earlier warnings of a high need of ICU capacity and high mortality rates among relatively healthy COVID-19 patients as this may lead to a higher mental workload for the staff.

The impact of COVID-19 on nursing workload and planning of nursing staff on the Intensive Care: A prospective descriptive multicenter study.

INTRODUCTION: The impact of the care for COVID-19 patients on nursing workload and planning nursing staff on the Intensive Care Unit has been huge. Nurses were confronted with a high workload and an increase in the number of patients per nurse they had to take care of. OBJECTIVE: The primary aim of this study is to describe differences in the planning of nursing staff on the Intensive Care in the COVID period versus a recent non-COVID period. The secondary aim was to describe differences in nursing workload in COVID-19 patients, pneumonia patients and other patients on the Intensive Care. We finally wanted to assess the cause of possible differences in Nursing Activities Scores between the different groups. METHODS: We analyzed data on nursing staff and nursing workload as measured by the Nursing Activities Score of 3,994 patients and 36,827 different shifts in 6 different hospitals in the Netherlands. We compared data from the COVID-19 period, March 1st 2020 till July 1st 2020, with data in a non-COVID period, March 1st 2019 till July 1st 2019. We analyzed the Nursing Activities Score per patient, the number of patients per nurse and the Nursing Activities Score per nurse in the different cohorts and time periods. Differences were tested by a Chi-square, non-parametric Wilcoxon or Student's t-test dependent on the distribution of the data. RESULTS: Our results showed both a significant higher number of patients per nurse (1.1 versus 1.0, p<0.001) and a significant higher Nursing Activities Score per Intensive Care nurse (76.5 versus 50.0, p<0.001) in the COVID-19 period compared to the non-COVID period. The Nursing Activities Score was significantly higher in COVID-19 patients compared to both the pneumonia patients (55.2 versus 50.0, p<0.001) and the non-COVID patients (55.2 versus 42.6, p<0.001), mainly due to more intense hygienic procedures, mobilization and positioning, support and care for relatives and respiratory care. CONCLUSION: With this study we showed the impact of COVID-19 patients on the planning of nursing care on the Intensive Care. The COVID-19 patients caused a high nursing workload, both in number of patients per nurse and in Nursing Activities Score per nurse.

The objective nursing workload and perceived nursing workload in Intensive Care Units: Analysis of association.

BACKGROUND: A range of classification systems are in use for the measurement of nursing workload in Intensive Care Units. However, it is unknown to what extent the measured (objective) nursing workload, usually in terms of the amount of nursing activities, is related to the workload actually experienced (perceived) by nurses. OBJECTIVES: The aim of this study was to assess the association between the objective nursing workload and the perceived nursing workload and to identify other factors associated with the perceived nursing workload. METHODS: We measured the objective nursing workload with the Nursing Activities Score and the perceived nursing workload with the NASA-Task Load Index during 228 shifts in eight different Intensive Care Units. We used linear mixed-effect regression models to analyze the association between the objective and perceived nursing workload. Furthermore, we investigated the association of patient characteristics (severity of illness, comorbidities, age, body mass index, and planned or unplanned admission), education level of the nurse, and contextual factors (numbers of patients per nurse, the type of shift (day, evening, night) and day of admission or discharge) with perceived nursing workload. We adjusted for confounders. RESULTS: We did not find a significant association between the observed workload per nurse and perceived nursing workload (p=0.06). The APACHE-IV Acute Physiology Score of a patient was significantly associated with the perceived nursing workload, also after adjustment for confounders (p=0.02). None of the other patient characteristics was significantly associated with perceived nursing workload. Being a certified nurse or a student nurse was the only nursing or contextual factor significantly associated with the perceived nursing workload, also after adjustment for confounders (p=0.03). CONCLUSION: Workload is perceived differently by nurses compared to the objectively measured workload by the Nursing Activities Score. Both the severity of illness of the patient and being a student nurse are factors that increase the perceived nursing workload. To keep the workload of nurses in balance, planning nursing capacity should be based on the Nursing Activities Score, on the severity of patient illness and the graduation level of the nurse.

Effects of maltreatment and parental schizophrenia spectrum disorders on early childhood social-emotional functioning: a population record linkage study.

AIMS: Childhood maltreatment and a family history of a schizophrenia spectrum disorder (SSD) are each associated with social-emotional dysfunction in childhood. Both are also strong risk factors for adult SSDs, and social-emotional dysfunction in childhood may be an antecedent of these disorders. We used data from a large Australian population cohort to determine the independent and moderating effects of maltreatment and parental SSDs on early childhood social-emotional functioning. METHODS: The New South Wales Child Development Study combines intergenerational multi-agency data using record linkage methods. Multiple measures of social-emotional functioning (social competency, prosocial/helping behaviour, anxious/fearful behaviour; aggressive behaviour, and hyperactivity/inattention) on 69 116 kindergarten children (age ~5 years) were linked with government records of child maltreatment and parental presentations to health services for SSD. Multivariable analyses investigated the association between maltreatment and social-emotional functioning, adjusting for demographic variables and parental SSD history, in the population sample and in sub-cohorts exposed and not exposed to parental SSD history. We also examined the association of parental SSD history and social-emotional functioning, adjusting for demographic variables and maltreatment. RESULTS: Medium-sized associations were identified between maltreatment and poor social competency, aggressive behaviour and hyperactivity/inattention; small associations were revealed between maltreatment and poor prosocial/helping and anxious/fearful behaviours. These associations did not differ greatly when adjusted for parental SSD, and were greater in magnitude among children with no history of parental SSD. Small associations between parental SSD and poor social-emotional functioning remained after adjusting for demographic variables and maltreatment. CONCLUSIONS: Childhood maltreatment and history of parental SSD are associated independently with poor early childhood social-emotional functioning, with the impact of exposure to maltreatment on social-emotional functioning in early childhood of greater magnitude than that observed for parental SSDs. The impact of maltreatment was reduced in the context of parental SSDs. The influence of parental SSDs on later outcomes of maltreated children may become more apparent during adolescence and young adulthood when overt symptoms of SSD are likely to emerge. Early intervention to strengthen childhood social-emotional functioning might mitigate the impact of maltreatment, and potentially also avert future psychopathology.

A whole-of-population study of term and post-term gestational age at birth and children's development.

OBJECTIVE: To examine the risk of poor child development according to week of gestation at birth, among children born ≥ 37 weeks' gestation. DESIGN: Population-based study using linked data (n = 12,601). SETTING: South Australia. POPULATION: All births ≥ 37 weeks' gestation. METHODS: Relative risks of developmental vulnerability for each week of gestation were calculated with adjustment for confounders and addressing missing information. MAIN OUTCOME MEASURES: Child development was documented by teachers during a national census of children attending their first year of school in 2009, using the Australian Early Development Index (AEDI). Children scoring in the lowest 10% of the AEDI were categorised as developmentally vulnerable. RESULTS: The percentage of children vulnerable on one or more AEDI domains for the following gestational ages 37, 38, 39, 40, 41, 42-45 weeks was 24.8, 22.3, 20.6, 20.0, 20.4 and 24.2, respectively. Compared with children born at 40 weeks, the adjusted relative risks [(95% confidence interval (CI)] for vulnerability on ≥ 1 AEDI domain were; 37 weeks 1.13 (0.99-1.28), 38 weeks 1.05 (0.96-1.15), 39 weeks 1.02 (0.94-1.12), 41 weeks 1.00 (0.90-1.11) and 42-45 weeks 1.20 (0.84-1.72). CONCLUSIONS: Children born at 40-41 weeks' gestation may have the lowest risk of developmental vulnerability at school entry, reinforcing the importance of term birth in perinatal care. Early term or post-term gestational age at birth can help clinicians, teachers and parents recognise children with potential developmental vulnerabilities at school entry.

Nonuniform accumulation of cadmium and copper in kidneys of wild brown trout (Salmo trutta) populations.

Cadmium and copper accumulated nonuniformly in the kidney of wild brown trout on a longitudinal gradient as follows: the concentration in head kidney < anterior portion of tail kidney < posterior portion(s) of tail kidney. Zinc accumulated uniformly along the length of the kidney. Cadmium and copper accumulation in brown trout kidneys was lowest in the head kidney, where hematopoietic tissue dominates, and greatest in posterior tail sections, where filtration tissues dominate. Sampling programs designed to monitor cadmium and copper accumulation in trout kidneys should utilize the entire kidney, including the head kidney, to create comparable data sets.

The role of social and health statistics in measuring harm from alcohol.

Since excess use of alcohol contributes to so many varieties of health and social harms, in most countries, there are many potential sources of data indicative of alcohol-related harms. In few instances, compilation and interpretation of these data are straightforward, but, mostly, they are open to various sources of measurement error, which need to be taken into account if they are to be applied for research purposes. Police and health statistics are the major source of such information, but the underlying systems are not usually set up with the purpose of monitoring alcohol-related events. In both of these domains, types of events can be identified, which are wholly attributable to excess alcohol use, i.e. drunk-driving, alcoholic liver cirrhosis. Specific alcohol-related events are particularly prone to variations in, respectively, police enforcement practices, medical diagnostic fashion and sensitivity to prejudices about alcohol-related problems. A case will be made in this paper for the use of multiple surrogate measures of alcohol-related harm drawn from several sources in order to measure and track local, regional and national trends. For health statistics on mortality and morbidity, the aetiologic fraction (AF) method will be recommended for such monitoring purposes. It will also be recommended that these data be categorised by the degree to which cases are attributable to alcohol and also by whether the underlying hazardous drinking pattern is a brief drinking bout or a sustained pattern of heavy intake over a number of years. Nighttime occurrences of road crashes, public violence from both police and emergency room attendance data will also be recommended. It will be argued that routine recording of alcohol relatedness of events is usually unreliable, and the above surrogate measures are preferable. Recommendations will also be made for utilising national surveys of drinking behaviour to improve the calculation of alcohol-related morbidity and mortality, as well as refine estimates of per capita alcohol consumption, another major 'surrogate' measure of alcohol-related harm. The arguments will be illustrated with reference to Australia's National Alcohol Indicators Project and related research projects.

Traumatic brain injury: influence of blood alcohol level on post-acute cognitive function.

Alcohol intoxication frequently contributes to the occurrence of traumatic brain injury. Few studies, however, have examined whether acute pre-injury alcohol intoxication or premorbid history of alcohol abuse exacerbate cognitive impairments that commonly result from traumatic brain injury. This study examined the influence of blood alcohol level at time of hospital admission on cognitive functioning during the post-acute stage of recovery from traumatic brain injury. After controlling for pre-injury history of alcohol abuse, hospital admission blood alcohol level was predictive of poorer delayed verbal memory, greater decrement in verbal memory over time, and poorer visuospatial functioning. Moreover, there were non-significant trends for higher blood alcohol levels to predict poorer performance on measures of immediate verbal memory and perseveration.

Neuropsychological improvement following endarterectomy as a function of outcome measure and reconstructed vessel.

30 patient receiving right or left carotid reconstruction and 15 medically matched controls were compared pre- and post-surgically on measures of motor speed, sustained vigilance, verbal memory and verbal and nonverbal intellectual function. The group receiving right sided vessel reconstruction demonstrated the largest post-operative improvement in intellectual function in any of the groups. The findings suggest that increased blood perfusion following right sided endarterectomy facilitates the right hemisphere's exclusive control of bilateral attention/arousal responses. In addition, findings suggest that detection of post-endarterectomy improvement may be dependent on the specific task dimension sampled, e.g., speed vs. cognitive ability and verbal-graphic vs. nonverbal symbol manipulation.

Event-related potential in Alzheimer disease.

Auditory event - related potentials were studied in 20 patients with SDAT and 20 age and sex matched normal controls. Patients with SDAT showed prolonged latencies of N200 and P300 components. The mean amplitudes of N200 and P300 were lower in the SDAT group. This reflects the impairment of the speed of neural processing in patients with SDAT. There were no significant correlations of the progression of P300 latencies from mild to severe dementia according to global dementia scales.

Classification of dementia patients by a WAIS profile related to central cholinergic deficiencies.

Sixty-two patients were classified as having Alzheimer-type dementia (AD) or multi-infarct dementia (MID) on the basis of clinical criteria. Protocols from the Wechsler Adult Intelligence Scale (WAIS) were scored according to a formula reported by Fuld to reflect the effects of scopolamine administration in young adults. The formula correctly classified 13 of 23 AD patients and 37 of 39 MID patients. The formula was similar to Wechsler's deterioration quotient, which produced a greater number of false positives. The formula did not appear to be biased by age, sex, or severity of impairments. Results suggest that the neuropsychological investigation of AD which emphasizes the deficiencies in cholinergic neurotransmission may be a fruitful avenue of further investigation.

Proline in the cerebrospinal fluid of normal subjects and Alzheimer's-disease patients, as determined with a new double-labeling assay technique.

Past studies have implicated proline involvement in the function of memory and learning. A new micromethod has been developed that is suitable for measuring proline accurately in as little as 0.1 ml of CSF. In normal human CSF, the average proline level was found to be consistently about 1.3 microM. In the CSF of patients with Alzheimer's disease and mixed dementias, the levels of proline showed no statistically significant difference from proline levels in the CSF of normal controls. Furthermore, the proline levels in the CSF of the Alzheimer's disease patients did not reflect, consistently, the cognitive deficits or the symptomatic severity of the disease. Proline levels in CSF showed no statistically significant change with the age of individuals tested.

Neuropsychological impairment after carotid endarterectomy correlates with intraoperative ischemia.

Detailed neuropsychological assessments were performed before and shortly after carotid endarterectomy in thirty-four patients. The degree of intraoperative ischemia was assessed by monitoring the somatosensory evoked cortical potential change upon carotid clamping. Changed neuropsychological performance was found to be related to intraoperative ischemia most clearly in patients with a history of previous stroke and in those with more severe vascular disease. In such patients greater SSEP change was correlated with greater neuropsychological change postoperatively.

Neuropsychological performance one week after carotid endarterectomy reflects intra-operative ischemia.

Fourteen patients with severe bilateral carotid artery stenosis underwent carotid endarterectomy. Intra-operative ischemia was monitored by somatosensory evoked potentials (SSEP) bilaterally. Neuropsychological evaluations were completed within two days before operation and 4-9 days after operation. Complete loss of N1-P1 or P1-N2 components of the SSEP (seen in 4 patients) was associated with a worsening of neuropsychological abilities (p less than .01). Two of these patients subsequently had strokes (7 and 35 days after operation). No other patients in the series have had strokes. Patients whose N1-P1 or P1-N2 amplitudes decreased by 50% or more performed worse after operation than patients with less severe reductions in these amplitudes (p less than .02). Time since first ischemic symptoms, age, education, clamp time, pre-operative stroke, and interval from surgery to assessment were not statistically related to changes in neuropsychological abilities. Patients with ischemic events in the week prior to surgery tended to improve in neuropsychological abilities 4-9 days after operation (p less than .05). Recentness of ischemic episode, however, was not related to intra-operative SSEP change. Results suggest the potential utility of intra-operative SSEP monitoring and early post-operative neuropsychological assessments both for clinical and research purposes.

Lithium-induced increases in red blood cell choline and memory performance in Alzheimer-type dementia.

To investigate the relationship between RBC choline and memory in Alzheimer-type senile dementia (SDAT), lithium carbonate was administered to 14 SDAT patients in doses of 400-600 mg/day for 5 weeks. A battery of memory tests was administered at baseline and at weekly intervals. Five patients with serum concentrations below 0.6 meq/liter developed neurotoxicity and were dropped from further analysis. For the remaining patients, Li+ with mean serum concentrations up to 0.6 meq/liter did not alter memory scores significantly. The dramatic increases in RBC choline during the study, however, suggest that RBC choline is not correlated with memory functioning in SDAT.

Changes in brain ventricular size with repeated CAT scans in suspected Alzheimer's disease.

The interpretation of brain ventricular size in CAT scans of patients with dementia is made difficult by the marked variability in ventricular size that occurs with normal aging. Five patients with probable Alzheimer's disease had serial CAT scans. Published normative data were interpolated to estimate the normal rate of change in ventricular size. Over a period of 15 to 35 months, four of the five patients had increases in ventricular size that were remarkably greater than the norms. The data suggest that the rate of change in ventricular size may be of considerable interest in longitudinal analyses and as a neurodiagnostic parameter.