Clinical Needs Assessment of a Machine Learning-Based Asthma Management Tool: User-Centered Design Approach.

BACKGROUND: Personalized asthma management depends on a clinician's ability to efficiently review patient's data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. OBJECTIVE: We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians' experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. METHODS: At the "discovery" phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the "define" phase, a synthesis analysis was conducted to converge key results from interviewees' insights into themes, eventually forming critical "how might we" research questions to guide model development and implementation. RESULTS: We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients' high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. CONCLUSIONS: As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design.

The Role of Artificial Intelligence Model Documentation in Translational Science: Scoping Review.

BACKGROUND: Despite the touted potential of artificial intelligence (AI) and machine learning (ML) to revolutionize health care, clinical decision support tools, herein referred to as medical modeling software (MMS), have yet to realize the anticipated benefits. One proposed obstacle is the acknowledged gaps in AI translation. These gaps stem partly from the fragmentation of processes and resources to support MMS transparent documentation. Consequently, the absence of transparent reporting hinders the provision of evidence to support the implementation of MMS in clinical practice, thereby serving as a substantial barrier to the successful translation of software from research settings to clinical practice. OBJECTIVE: This study aimed to scope the current landscape of AI- and ML-based MMS documentation practices and elucidate the function of documentation in facilitating the translation of ethical and explainable MMS into clinical workflows. METHODS: A scoping review was conducted in accordance with PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. PubMed was searched using Medical Subject Headings key concepts of AI, ML, ethical considerations, and explainability to identify publications detailing AI- and ML-based MMS documentation, in addition to snowball sampling of selected reference lists. To include the possibility of implicit documentation practices not explicitly labeled as such, we did not use documentation as a key concept but as an inclusion criterion. A 2-stage screening process (title and abstract screening and full-text review) was conducted by 1 author. A data extraction template was used to record publication-related information; barriers to developing ethical and explainable MMS; available standards, regulations, frameworks, or governance strategies related to documentation; and recommendations for documentation for papers that met the inclusion criteria. RESULTS: Of the 115 papers retrieved, 21 (18.3%) papers met the requirements for inclusion. Ethics and explainability were investigated in the context of AI- and ML-based MMS documentation and translation. Data detailing the current state and challenges and recommendations for future studies were synthesized. Notable themes defining the current state and challenges that required thorough review included bias, accountability, governance, and explainability. Recommendations identified in the literature to address present barriers call for a proactive evaluation of MMS, multidisciplinary collaboration, adherence to investigation and validation protocols, transparency and traceability requirements, and guiding standards and frameworks that enhance documentation efforts and support the translation of AI- and ML-based MMS. CONCLUSIONS: Resolving barriers to translation is critical for MMS to deliver on expectations, including those barriers identified in this scoping review related to bias, accountability, governance, and explainability. Our findings suggest that transparent strategic documentation, aligning translational science and regulatory science, will support the translation of MMS by coordinating communication and reporting and reducing translational barriers, thereby furthering the adoption of MMS.

A Technical Performance Study and Proposed Systematic and Comprehensive Evaluation of an ML-based CDS Solution for Pediatric Asthma.

Achieving optimal care for pediatric asthma patients depends on giving clinicians efficient access to pertinent patient information. Unfortunately, adherence to guidelines or best practices has shown to be challenging, as relevant information is often scattered throughout the patient record in both structured data and unstructured clinical notes. Furthermore, in the absence of supporting tools, the onus of consolidating this information generally falls upon the clinician. In this study, we propose a machine learning-based clinical decision support (CDS) system focused on pediatric asthma care to alleviate some of this burden. This framework aims to incorporate a machine learning model capable of predicting asthma exacerbation risk into the clinical workflow, emphasizing contextual data, supporting information, and model transparency and explainability. We show that this asthma exacerbation model is capable of predicting exacerbation with an 0.8 AUC-ROC. This model, paired with a comprehensive informatics-based process centered on clinical usability, emphasizes our focus on meeting the needs of the clinical practice with machine learning technology.

Assessment of Proteomic Measures Across Serious Psychiatric Illness.

The diagnoses of serious psychiatric illnesses, such as schizophrenia, schizoaffective disorder, and bipolar disorder, rely on the subjective recall and interpretation of often overlapping symptoms, and are not based on the objective pathophysiology of the illnesses. The subjectivity of symptom reporting and interpretation contributes to the delay of accurate diagnoses and limits effective treatment of these illnesses. Proteomics, the study of the types and quantities of proteins an organism produces, may offer an objective biological approach to psychiatric diagnosis. For this pilot study, we used the Myriad RBM Discovery Map 250+ platform to quantify 205 serum proteins in subjects with schizophrenia (n=26), schizoaffective disorder (n=20), bipolar disorder (n=16), and healthy controls with no psychiatric illness (n=23). Fifty-seven analytes that differed significantly between groups were used for multivariate modeling with linear discriminant analysis (LDA). Diagnoses generated from these models were compared to SCID-generated clinical diagnoses to determine whether the proteomic markers: 1) distinguished the three disorders from controls, and 2) distinguished between the three disorders. We found that a series of binary classification models including 8-12 analytes produced separation between all subjects and controls, and between each diagnostic group and controls. There was a high degree of accuracy in the separations, with training areas-under-the-curve (AUC) of 0.94-1.0, and cross-validation AUC of 0.94-0.95. Models with 7-14 analytes produced separation between the diagnostic groups, though less robustly, with training AUC of 0.72-1.0 and validation AUC of 0.69-0.89. While based on a small sample size, not adjusted for medication state, these preliminary results support the potential of proteomics as a diagnostic aid in psychiatry. The separation of schizophrenia, schizoaffective disorder, and bipolar disorder suggests that further work in this area is warranted.

Attenuation of high sweet solution preference by mood stabilizers: a possible mouse model for the increased reward-seeking domain of mania.

The lack of appropriate animal models for bipolar disorder (BPD) is a major factor hindering the research of its pathophysiology and the development of new drug treatments. In line with the notion that BPD might represent a heterogeneous group of disorders, it was suggested that models for specific domains of BPD should be developed and then integrated. The present study tested sweet solution preference as a rodent model for increased reward seeking, a central component of manic behavior and a possible endophenotype of the disorder. The study identified that Black Swiss mice show high baseline saccharin preference compared with C57bl/6, CBA/J and A/J strains. Sweet solution preference in Black Swiss mice was therefore evaluated across a number of saccharin concentrations, with or without treatment with the mood stabilizers lithium and valproate and the antidepressant imipramine. Results indicated that the structurally dissimilar mood stabilizers lithium and valproate, but not the antidepressant imipramine, reduce sweet solution preference. However, different dosing schedules were needed for the two drugs to induce this effect. These findings support the face and the predictive validity of the sweet solution preference test as an animal model for the elevated reward-seeking domain of mania. As such, this test might be well integrated into a battery of models for different domains of BPD. Such a battery can be effectively utilized to screen new treatments, to distinguish between specific effects of different drugs, and to explore the mechanisms underlying BPD.