A multimodal dialog approach to mental state characterization in clinically depressed, anxious, and suicidal populations.

Background: The rise of depression, anxiety, and suicide rates has led to increased demand for telemedicine-based mental health screening and remote patient monitoring (RPM) solutions to alleviate the burden on, and enhance the efficiency of, mental health practitioners. Multimodal dialog systems (MDS) that conduct on-demand, structured interviews offer a scalable and cost-effective solution to address this need. Objective: This study evaluates the feasibility of a cloud based MDS agent, Tina, for mental state characterization in participants with depression, anxiety, and suicide risk. Method: Sixty-eight participants were recruited through an online health registry and completed 73 sessions, with 15 (20.6%), 21 (28.8%), and 26 (35.6%) sessions screening positive for depression, anxiety, and suicide risk, respectively using conventional screening instruments. Participants then interacted with Tina as they completed a structured interview designed to elicit calibrated, open-ended responses regarding the participants' feelings and emotional state. Simultaneously, the platform streamed their speech and video recordings in real-time to a HIPAA-compliant cloud server, to compute speech, language, and facial movement-based biomarkers. After their sessions, participants completed user experience surveys. Machine learning models were developed using extracted features and evaluated with the area under the receiver operating characteristic curve (AUC). Results: For both depression and suicide risk, affected individuals tended to have a higher percent pause time, while those positive for anxiety showed reduced lip movement relative to healthy controls. In terms of single-modality classification models, speech features performed best for depression (AUC = 0.64; 95% CI = 0.51-0.78), facial features for anxiety (AUC = 0.57; 95% CI = 0.43-0.71), and text features for suicide risk (AUC = 0.65; 95% CI = 0.52-0.78). Best overall performance was achieved by decision fusion of all models in identifying suicide risk (AUC = 0.76; 95% CI = 0.65-0.87). Participants reported the experience comfortable and shared their feelings. Conclusion: MDS is a feasible, useful, effective, and interpretable solution for RPM in real-world clinical depression, anxiety, and suicidal populations. Facial information is more informative for anxiety classification, while speech and language are more discriminative of depression and suicidality markers. In general, combining speech, language, and facial information improved model performance on all classification tasks.

Virtually screening adults for depression, anxiety, and suicide risk using machine learning and language from an open-ended interview.

Background: Current depression, anxiety, and suicide screening techniques rely on retrospective patient reported symptoms to standardized scales. A qualitative approach to screening combined with the innovation of natural language processing (NLP) and machine learning (ML) methods have shown promise to enhance person-centeredness while detecting depression, anxiety, and suicide risk from in-the-moment patient language derived from an open-ended brief interview. Objective: To evaluate the performance of NLP/ML models to identify depression, anxiety, and suicide risk from a single 5-10-min semi-structured interview with a large, national sample. Method: Two thousand four hundred sixteen interviews were conducted with 1,433 participants over a teleconference platform, with 861 (35.6%), 863 (35.7%), and 838 (34.7%) sessions screening positive for depression, anxiety, and suicide risk, respectively. Participants completed an interview over a teleconference platform to collect language about the participants' feelings and emotional state. Logistic regression (LR), support vector machine (SVM), and extreme gradient boosting (XGB) models were trained for each condition using term frequency-inverse document frequency features from the participants' language. Models were primarily evaluated with the area under the receiver operating characteristic curve (AUC). Results: The best discriminative ability was found when identifying depression with an SVM model (AUC = 0.77; 95% CI = 0.75-0.79), followed by anxiety with an LR model (AUC = 0.74; 95% CI = 0.72-0.76), and an SVM for suicide risk (AUC = 0.70; 95% CI = 0.68-0.72). Model performance was generally best with more severe depression, anxiety, or suicide risk. Performance improved when individuals with lifetime but no suicide risk in the past 3 months were considered controls. Conclusion: It is feasible to use a virtual platform to simultaneously screen for depression, anxiety, and suicide risk using a 5-to-10-min interview. The NLP/ML models performed with good discrimination in the identification of depression, anxiety, and suicide risk. Although the utility of suicide risk classification in clinical settings is still undetermined and suicide risk classification had the lowest performance, the result taken together with the qualitative responses from the interview can better inform clinical decision-making by providing additional drivers associated with suicide risk.

User Feedback on the Use of a Natural Language Processing Application to Screen for Suicide Risk in the Emergency Department.

Suicide is the 10th leading cause of death in the USA and globally. Despite decades of research, the ability to predict who will die by suicide is still no better than 50%. Traditional screening instruments have helped identify risk factors for suicide, but they have not provided accurate predictive power for reducing death rates. Over the past decade, natural language processing (NLP), a form of machine learning (ML), has been used to identify suicide risk by analyzing language data. Recent work has demonstrated the successful integration of a suicide risk screening interview to collect language data for NLP analysis from patients in two emergency departments (ED) of a large healthcare system. Results indicated that ML/NLP models performed well identifying patients that came to the ED for suicide risk. However, little is known about the clinician's perspective of how a qualitative brief interview suicide risk screening tool to collect language data for NLP integrates into an ED workflow. This report highlights the feedback and observations of patient experiences obtained from clinicians using brief suicide screening interviews. The investigator used an open-ended, narrative interview approach to inquire about the qualitative interview process. Three overarching themes were identified: behavioral health workflow, clinical implications of interview probes, and integration of an application into provider patient experience. Results suggest a brief, qualitative interview method was feasible, person-centered, and useful as a suicide risk detection approach.

Are funeral homes in the United States safe spaces for sexual and gender minorities? A website content analysis.

LGBTQIA+ people, particularly those aging into end-of-life care decisions, need safety cues to identify safe spaces to access equitable death care. We conducted a website content analysis of 90 randomly selected funeral homes across the United States to evaluate the presence of LGBTQIA+ safety cues, such as inclusive language, symbols, imagery, and LGBTQIA+-friendly collaborations. Results showed that none of the selected funeral homes displayed any kind of safety cues. A significant change in funeral home marketing strategies is warranted so sexual and gender minorities can easily locate inclusive and affirming death care services.

Integration and Validation of a Natural Language Processing Machine Learning Suicide Risk Prediction Model Based on Open-Ended Interview Language in the Emergency Department.

BACKGROUND: Emergency departments (ED) are an important intercept point for identifying suicide risk and connecting patients to care, however, more innovative, person-centered screening tools are needed. Natural language processing (NLP) -based machine learning (ML) techniques have shown promise to assess suicide risk, although whether NLP models perform well in differing geographic regions, at different time periods, or after large-scale events such as the COVID-19 pandemic is unknown. OBJECTIVE: To evaluate the performance of an NLP/ML suicide risk prediction model on newly collected language from the Southeastern United States using models previously tested on language collected in the Midwestern US. METHOD: 37 Suicidal and 33 non-suicidal patients from two EDs were interviewed to test a previously developed suicide risk prediction NLP/ML model. Model performance was evaluated with the area under the receiver operating characteristic curve (AUC) and Brier scores. RESULTS: NLP/ML models performed with an AUC of 0.81 (95% CI: 0.71-0.91) and Brier score of 0.23. CONCLUSION: The language-based suicide risk model performed with good discrimination when identifying the language of suicidal patients from a different part of the US and at a later time period than when the model was originally developed and trained.

Interprofessional Suicide Prevention Education.

The COVID-19 pandemic has raised concerns regarding possible spikes in suicidal behavior in light of heightened risk factors such as social isolation and financial strain; thus, comprehensive suicide prevention training for emerging health service providers is increasingly vital. This article summarizes an interprofessional education (IPE) suicide prevention course delivered in-person in Spring 2020. Pilot data demonstrate that despite the impact of COVID-19 on higher education, this course had long-term impacts on trainee suicide prevention efficacy, IPE attitudes, and use of course content in practice. Discussion serves to address enhancements for interprofessional and suicide prevention education during and after the pandemic. Emphasis is placed on adaptable training strategies, considerations in the delivery format, guidelines for intensive virtual meetings with trainee teams, and future directions in IPE suicide prevention training research.

A Feasibility Study Using a Machine Learning Suicide Risk Prediction Model Based on Open-Ended Interview Language in Adolescent Therapy Sessions.

BACKGROUND: As adolescent suicide rates continue to rise, innovation in risk identification is warranted. Machine learning can identify suicidal individuals based on their language samples. This feasibility pilot was conducted to explore this technology's use in adolescent therapy sessions and assess machine learning model performance. METHOD: Natural language processing machine learning models to identify level of suicide risk using a smartphone app were tested in outpatient therapy sessions. Data collection included language samples, depression and suicidality standardized scale scores, and therapist impression of the client's mental state. Previously developed models were used to predict suicidal risk. RESULTS: 267 interviews were collected from 60 students in eight schools by ten therapists, with 29 students indicating suicide or self-harm risk. During external validation, models were trained on suicidal speech samples collected from two separate studies. We found that support vector machines (AUC: 0.75; 95% CI: 0.69-0.81) and logistic regression (AUC: 0.76; 95% CI: 0.70-0.82) lead to good discriminative ability, with an extreme gradient boosting model performing the best (AUC: 0.78; 95% CI: 0.72-0.84). CONCLUSION: Voice collection technology and associated procedures can be integrated into mental health therapists' workflow. Collected language samples could be classified with good discrimination using machine learning methods.

Integrating Interprofessional Education into Suicide Prevention Training: Results from A Pilot Evaluation.

Suicide prevention training for health professions students is lacking, often occurring in disciplinary silos. The present study reports outcomes from an interprofessional education (IPE)-based suicide prevention course for health professions students across a variety of disciplines (e.g., social work, counseling, public health). Using a quasi-experimental design, students either took part in a fully online or blended version of the course. Primary outcomes included: (1) significant moderate-to-large positive gains in suicide prevention knowledge, perceived clinical care skills, and perceived ability to help self-harming patients; (2) moderate positive shifts in sensitivity to risk factors of those who died by suicide; (3) non-significant impacts on IPE-related outcomes; (4) overall high course satisfaction; and (5) students in the blended course preferred several interactive methods more than students in the online course version (large effects). Recommendations are provided for course revision and future implementation in educational and community-based settings.

Managing physical and mental health conditions: Consumer perspectives on integrated care.

BACKGROUND: Despite the growing trend of integrating primary care and mental health services, little research has documented how consumers with severe mental illnesses manage comorbid conditions or view integrated services. OBJECTIVES: We sought to better understand how consumers perceive and manage both mental and physical health conditions and their views of integrated services. METHODS: We conducted semi-structured interviews with consumers receiving primary care services integrated in a community mental health setting. RESULTS: Consumers described a range of strategies to deal with physical health conditions and generally viewed mental and physical health conditions as impacting one another. Consumers viewed integration of primary care and mental health services favorably, specifically its convenience, friendliness and knowledge of providers, and collaboration between providers. CONCLUSIONS: Although integration was viewed positively, consumers with SMI may need a myriad of strategies and supports to both initiate and sustain lifestyle changes that address common physical health problems.

Comparison of Assertive Community Treatment Fidelity Assessment Methods: Reliability and Validity.

Assertive community treatment is known for improving consumer outcomes, but is difficult to implement. On-site fidelity measurement can help ensure model adherence, but is costly in large systems. This study compared reliability and validity of three methods of fidelity assessment (on-site, phone-administered, and expert-scored self-report) using a stratified random sample of 32 mental health intensive case management teams from the Department of Veterans Affairs. Overall, phone, and to a lesser extent, expert-scored self-report fidelity assessments compared favorably to on-site methods in inter-rater reliability and concurrent validity. If used appropriately, these alternative protocols hold promise in monitoring large-scale program fidelity with limited resources.

Factor structure of the autonomy preference index in people with severe mental illness.

People vary in the amount of control they want to exercise over decisions about their healthcare. Given the importance of patient-centered care, accurate measurement of these autonomy preferences is critical. This study aimed to assess the factor structure of the Autonomy Preference Index (API), used widely in general healthcare, in individuals with severe mental illness. Data came from two studies of people with severe mental illness (N=293) who were receiving mental health and/or primary care/integrated care services. Autonomy preferences were assessed with the API regarding both psychiatric and primary care services. Confirmatory factor analysis was used to evaluate fit of the hypothesized two-factor structure of the API (decision-making autonomy and information-seeking autonomy). Results indicated the hypothesized structure for the API did not adequately fit the data for either psychiatric or primary care services. Three problematic items were dropped, resulting in adequate fit for both types of treatment. These results suggest that with relatively minor modifications the API has an acceptable factor structure when asking people with severe mental illness about their preferences to be involved in decision-making. The modified API has clinical and research utility for this population in the burgeoning field of autonomy in patient-centered healthcare.

Consumer and provider responses to a computerized version of the Illness Management and Recovery Program.

OBJECTIVE: To explore mental health consumer and provider responses to a computerized version of the Illness Management and Recovery (IMR) program. METHOD: Semistructured interviews were conducted to gather data from 6 providers and 12 consumers who participated in a computerized prototype of the IMR program. An inductive-consensus-based approach was used to analyze the interview responses. RESULTS: Qualitative analysis revealed consumers perceived various personal benefits and ease of use afforded by the new technology platform. Consumers also highly valued provider assistance and offered several suggestions to improve the program. The largest perceived barriers to future implementation were lack of computer skills and access to computers. Similarly, IMR providers commented on its ease and convenience, and the reduction of time intensive material preparation. Providers also expressed that the use of technology creates more options for the consumer to access treatment. CONCLUSIONS AND IMPLICATIONS FOR PRACTICE: The technology was acceptable, easy to use, and well-liked by consumers and providers. Clinician assistance with technology was viewed as helpful to get clients started with the program, as lack of computer skills and access to computers was a concern. Access to materials between sessions appears to be desired; however, given perceived barriers of computer skills and computer access, additional supports may be needed for consumers to achieve full benefits of a computerized version of IMR.

A comparison of self-reported and phone-administered methods of ACT fidelity assessment: a pilot study in Indiana.

OBJECTIVE: Monitoring fidelity of assertive community treatment (ACT) teams is costly. This study investigated the reliability and validity of a less burdensome approach: self-reported assessment. METHODS: Phone-administered and self-reported assessments were compared for 16 ACT teams. Team leaders completed a self-report protocol providing information sufficient to score the Dartmouth Assertive Community Treatment Scale (DACTS). Two raters scored the DACTS using only self-reported information. Two additional raters conducted phone interviews with team leaders, verifying the self-reported data, and independently scored the DACTS. RESULTS: DACTS total scores obtained via self-reported assessments were reliable and valid compared with phone-administered assessment on the basis of interrater consistency (intraclass correlation) and consensus (mean rating differences). Phone-administered assessments agreed with self-reported assessments within .25 scale points (out of 5 points) for 15 of 16 teams. CONCLUSIONS: A self-report approach could address concerns regarding costs of monitoring as part of a stepped approach to quality assurance.

A review of consumer-provided services on assertive community treatment and intensive case management teams: implications for future research and practice.

BACKGROUND: Assertive community treatment (ACT) is an evidence-based practice that provides intensive, in vivo services for adults with severe mental illness. Some ACT and intensive case management teams have integrated consumers as team members with varying results. METHODS: The authors reviewed the literature examining the outcomes of having consumer providers on case management teams, with attention devoted to randomized controlled trials (RCTs). RESULTS: Sixteen published studies were identified, including eight RCTs. Findings were mixed, with evidence supporting consumer-provided services for improving engagement and limited support for reduced hospitalizations. However, evidence was lacking for other outcomes areas such as symptom reduction or improved quality of life. CONCLUSION: Including a consumer provider on an ACT team could enhance the outreach mechanisms of ACT, using a more recovery-focused approach to bring consumers into services and help engage them over time. More rigorous research is needed to further evaluate integrating consumer providers on teams.

Red blood cell storage is associated with length of stay and renal complications after cardiac surgery.

BACKGROUND: The association of red blood cell (RBC) storage on morbidity outcome after cardiac surgery is debated. We sought to clarify the association of the age of transfused blood on outcome in patients undergoing cardiac surgery. STUDY DESIGN AND METHODS: Data were drawn from a prospective, observational cohort study of morbidity outcome in patients undergoing cardiac surgery. Blood transfusion data were obtained retrospectively via the Trust blood bank electronic records. Old blood was defined as more than 14 days old. The primary outcome measure was postoperative length of stay (PLOS). Secondary outcome measures included renal failure and morbidity as defined within the postoperative morbidity survey. RESULTS: A total of 176 (39.6%) of 444 participants received a blood transfusion. Patients transfused with new blood had a reduced PLOS compared with patients receiving exclusively old or any old blood (old blood ± new blood; 7 days vs. 8 days, p = 0.04 and vs. 10 days, p = 0.002, respectively). In patients who only had 1 unit transfused, PLOS was longer in those receiving only old blood compared with those receiving only new blood (8 days vs. 6 days, p = 0.02) with a 3.8-fold risk of longer stay. Compared with patients receiving exclusively new blood, patients receiving any old blood had a higher incidence of new renal complications (65.7% vs. 43.9%, p = 0.008). Each 1-day increase in storage was associated with a 7% increase in risk of new renal complications. CONCLUSION: Our data support previous suggestions of an association between transfusion of older RBCs and poorer outcome in cardiac surgery patients. Randomized controlled trials are required to determine the true causal nature of any such association.