Predicting hospitalization following psychiatric crisis care using machine learning.

BACKGROUND: Accurate prediction models for whether patients on the verge of a psychiatric criseis need hospitalization are lacking and machine learning methods may help improve the accuracy of psychiatric hospitalization prediction models. In this paper we evaluate the accuracy of ten machine learning algorithms, including the generalized linear model (GLM/logistic regression) to predict psychiatric hospitalization in the first 12 months after a psychiatric crisis care contact. We also evaluate an ensemble model to optimize the accuracy and we explore individual predictors of hospitalization. METHODS: Data from 2084 patients included in the longitudinal Amsterdam Study of Acute Psychiatry with at least one reported psychiatric crisis care contact were included. Target variable for the prediction models was whether the patient was hospitalized in the 12 months following inclusion. The predictive power of 39 variables related to patients' socio-demographics, clinical characteristics and previous mental health care contacts was evaluated. The accuracy and area under the receiver operating characteristic curve (AUC) of the machine learning algorithms were compared and we also estimated the relative importance of each predictor variable. The best and least performing algorithms were compared with GLM/logistic regression using net reclassification improvement analysis and the five best performing algorithms were combined in an ensemble model using stacking. RESULTS: All models performed above chance level. We found Gradient Boosting to be the best performing algorithm (AUC = 0.774) and K-Nearest Neighbors to be the least performing (AUC = 0.702). The performance of GLM/logistic regression (AUC = 0.76) was slightly above average among the tested algorithms. In a Net Reclassification Improvement analysis Gradient Boosting outperformed GLM/logistic regression by 2.9% and K-Nearest Neighbors by 11.3%. GLM/logistic regression outperformed K-Nearest Neighbors by 8.7%. Nine of the top-10 most important predictor variables were related to previous mental health care use. CONCLUSIONS: Gradient Boosting led to the highest predictive accuracy and AUC while GLM/logistic regression performed average among the tested algorithms. Although statistically significant, the magnitude of the differences between the machine learning algorithms was in most cases modest. The results show that a predictive accuracy similar to the best performing model can be achieved when combining multiple algorithms in an ensemble model.

A prediction model for the incidence of civil detention for crisis patients with psychiatric illnesses; the Amsterdam study of acute psychiatry VII.

OBJECTIVE: Given raised numbers of civil detentions in the Netherlands and other European countries, it is important to assess the patient risk profile with respect to the incidence of those far-reaching treatment decisions. The aim of the ASAP study is to develop a comprehensive prediction model that considers all possible patient-related predictors known from earlier research. METHODS: We took a random sample of 252 from the 2,682 patients coming into contact with two psychiatric emergency teams in Amsterdam between September 2004 and September 2006. We recorded socio-demographic and clinical characteristics, aspects of social support and psychiatric history. We interviewed the patients using the Verona Service Satisfaction Scale (Verona-EU) and the Birchwood Insight Scale. During a two-year follow-up period we noted their use of mental health care facilities. RESULTS: Stepwise logistic regression analyses with resulted in a final prediction model (P ≤ 0.001) including preceding involuntary admission (OR 9.4, 95% CI 3.6-24.7, P ≤ 0.001), domestic situation alone (OR 4.5, 95% CI 1.9-11.0, P = 0.001) and VSSS score satisfactory (OR 0.2, 95% CI 0.0-0.8, P = 0.030) as predictors of civil detention during 2 years of follow-up. CONCLUSION: With the presented prediction model it will be possible to identify patients at a high risk of civil detention: patients with a history of previous involuntary admissions who live alone and are not satisfied with the mental health care they got before. This suggests the possibility that timely preventive measures can be taken comprising the adjustment or intensification of the treatment plan for this specific group of patients.

Social support and risk of compulsory admission: part IV of the Amsterdam Study of Acute Psychiatry.

OBJECTIVE: Social support for patients with a mental illness has been associated with a lower rate of hospitalization. It is important to clarify the role played by a lack of social support as a possible predictor of emergency compulsory admission. METHODS: A random sample of 252 patients who were evaluated by two psychiatric emergency teams in Amsterdam from September 2004 to September 2006 were interviewed approximately one month later about their social networks and social interactions. The number of emergency compulsory admissions was recorded for 244 patients during a two-year follow-up period after the interviews. RESULTS: Patients who lived alone had a higher risk of compulsory admission (p≤.05) and had fewer people in their social network (4.6 versus 6.1, p≤.001) compared with patients who lived with others. Among patients who lived alone, the percentage of patients with a compulsory admission was significantly higher among the patients with a high score for negative interactions than among patients with a low score (34% versus 13%, p≤.05). CONCLUSIONS: Of the social support variables, living alone proved to be the only predictor of emergency compulsory admission and readmission, and patients who lived alone had a smaller social network. A high level of negative social interactions increased the risk of compulsory admission among patients who lived alone.