Provision and evaluation of explanations within an automated planning-based approach to solving the multimorbidity problem.

The multimorbidity problem involves the identification and mitigation of adverse interactions that occur when multiple computer interpretable guidelines are applied concurrently to develop a treatment plan for a patient diagnosed with multiple diseases. Solving this problem requires decision support approaches which are difficult to comprehend for physicians. As such, the rationale for treatment plans generated by these approaches needs to be provided. OBJECTIVE: To develop an explainability component for an automated planning-based approach to the multimorbidity problem, and to assess the fidelity and interpretability of generated explanations using a clinical case study. METHODS: The explainability component leverages the task-network model for representing computer interpretable guidelines. It generates post-hoc explanations composed of three aspects that answer why specific clinical actions are in a treatment plan, why specific revisions were applied, and how factors like medication cost, patient's adherence, etc. influence the selection of specific actions. The explainability component is implemented as part of MitPlan, where we revised our planning-based approach to support explainability. We developed an evaluation instrument based on the system causability scale and other vetted surveys to evaluate the fidelity and interpretability of its explanations using a two dimensional comparison study design. RESULTS: The explainability component was implemented for MitPlan and tested in the context of a clinical case study. The fidelity and interpretability of the generated explanations were assessed using a physician-focused evaluation study involving 21 participants from two different specialties and two levels of experience. Results show that explanations provided by the explainability component in MitPlan are of acceptable fidelity and interpretability, and that the clinical justification of the actions in a treatment plan is important to physicians. CONCLUSION: We created an explainability component that enriches an automated planning-based approach to solving the multimorbidity problem with meaningful explanations for actions in a treatment plan. This component relies on the task-network model to represent computer interpretable guidelines and as such can be ported to other approaches that also use the task-network model representation. Our evaluation study demonstrated that explanations that support a physician's understanding of the clinical reasons for the actions in a treatment plan are useful and important.

How can we reward you? A compliance and reward ontology (CaRO) for eliciting quantitative reward rules for engagement in mHealth app and healthy behaviors.

OBJECTIVE: When developing mHealth apps with point reward systems, knowledge engineers and domain experts should define app requirements capturing quantitative reward patterns that reflect patient compliance with health behaviors. This is a difficult task, and they could be aided by an ontology that defines systematically quantitative behavior goals that address more than merely the recommended behavior but also rewards for partial compliance or practicing the behavior more than recommended. No ontology and algorithm exist for defining point rewards systematically. METHODS: We developed an OWL ontology for point rewards that leverages the Basic Formal Ontology, the Behaviour Change Intervention Ontology and the Gamification Domain Ontology. This Compliance and Reward Ontology (CaRO) allows defining temporal elementary reward patterns for single and multiple sessions of practicing a behavior. These could be assembled to define more complex temporal patterns for persistence behavior over longer time intervals as well as logical combinations of simpler reward patterns. We also developed an algorithm for calculating the points that should be rewarded to users, given data regarding their actual performance. A natural language generation algorithm generates from ontology instances app requirements in the form of user stories. To assess the usefulness of the ontology and algorithms, information system students who are trained to be system analysts/knowledge engineers evaluated whether the ontology and algorithms can improve the requirement elicitation of point rewards for compliance patterns more completely and correctly. RESULTS: For single-session rewards, the ontology improved formulation of two of the six requirements as well as the total time for specifying them. For multi-session rewards, the ontology improved formulation of five of the 11 requirements. CONCLUSION: CaRO is a first attempt of its kind, and it covers all of the cases of compliance and reward pattern definitions that were needed for a full-scale system that was developed as part of a large European project. The ontology and algorithm are available at https://github.com/capable-project/rewards.

An eHealth App (CAPABLE) Providing Symptom Monitoring, Well-Being Interventions, and Educational Material for Patients With Melanoma Treated With Immune Checkpoint Inhibitors: Protocol for an Exploratory Intervention Trial.

BACKGROUND: Since treatment with immune checkpoint inhibitors (ICIs) is becoming standard therapy for patients with high-risk and advanced melanoma, an increasing number of patients experience treatment-related adverse events such as fatigue. Until now, studies have demonstrated the benefits of using eHealth tools to provide either symptom monitoring or interventions to reduce treatment-related symptoms such as fatigue. However, an eHealth tool that facilitates the combination of both symptom monitoring and symptom management in patients with melanoma treated with ICIs is still needed. OBJECTIVE: In this pilot study, we will explore the use of the CAPABLE (Cancer Patients Better Life Experience) app in providing symptom monitoring, education, and well-being interventions on health-related quality of life (HRQoL) outcomes such as fatigue and physical functioning, as well as patients' acceptance and usability of using CAPABLE. METHODS: This prospective, exploratory pilot study will examine changes in fatigue over time in 36 patients with stage III or IV melanoma during treatment with ICI using CAPABLE (a smartphone app and multisensory smartwatch). This cohort will be compared to a prospectively collected cohort of patients with melanoma treated with standard ICI therapy. CAPABLE will be used for a minimum of 3 and a maximum of 6 months. The primary endpoint in this study is the change in fatigue between baseline and 3 and 6 months after the start of treatment. Secondary end points include HRQoL outcomes, usability, and feasibility parameters. RESULTS: Study inclusion started in April 2023 and is currently ongoing. CONCLUSIONS: This pilot study will explore the effect, usability, and feasibility of CAPABLE in patients with melanoma during treatment with ICI. Adding the CAPABLE system to active treatment is hypothesized to decrease fatigue in patients with high-risk and advanced melanoma during treatment with ICIs compared to a control group receiving standard care. The Medical Ethics Committee NedMec (Amsterdam, The Netherlands) granted ethical approval for this study (reference number 22-981/NL81970.000.22). TRIAL REGISTRATION: ClinicalTrials.gov NCT05827289; https://clinicaltrials.gov/study/NCT05827289. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/49252.

The Case Manager: An Agent Controlling the Activation of Knowledge Sources in a FHIR-Based Distributed Reasoning Environment.

BACKGROUND: Within the CAPABLE project the authors developed a multi-agent system that relies on a distributed architecture. The system provides cancer patients with coaching advice and supports their clinicians with suitable decisions based on clinical guidelines. OBJECTIVES: As in many multi-agent systems we needed to coordinate the activities of all agents involved. Moreover, since the agents share a common blackboard where all patients' data are stored, we also needed to implement a mechanism for the prompt notification of each agent upon addition of new information potentially triggering its activation. METHODS: The communication needs have been investigated and modeled using the HL7-FHIR (Health Level 7-Fast Healthcare Interoperability Resources) standard to ensure proper semantic interoperability among agents. Then a syntax rooted in the FHIR search framework has been defined for representing the conditions to be monitored on the system blackboard for activating each agent. RESULTS: The Case Manager (CM) has been implemented as a dedicated component playing the role of an orchestrator directing the behavior of all agents involved. Agents dynamically inform the CM about the conditions to be monitored on the blackboard, using the syntax we developed. The CM then notifies each agent whenever any condition of interest occurs. The functionalities of the CM and other actors have been validated using simulated scenarios mimicking the ones that will be faced during pilot studies and in production. CONCLUSION: The CM proved to be a key facilitator for properly achieving the required behavior of our multi-agent system. The proposed architecture may also be leveraged in many clinical contexts for integrating separate legacy services, turning them into a consistent telemedicine framework and enabling application reusability.

A community-of-practice-based evaluation methodology for knowledge intensive computational methods and its application to multimorbidity decision support.

OBJECTIVE: The study has dual objectives. Our first objective (1) is to develop a community-of-practice-based evaluation methodology for knowledge-intensive computational methods. We target a whitebox analysis of the computational methods to gain insight on their functional features and inner workings. In more detail, we aim to answer evaluation questions on (i) support offered by computational methods for functional features within the application domain; and (ii) in-depth characterizations of the underlying computational processes, models, data and knowledge of the computational methods. Our second objective (2) involves applying the evaluation methodology to answer questions (i) and (ii) for knowledge-intensive clinical decision support (CDS) methods, which operationalize clinical knowledge as computer interpretable guidelines (CIG); we focus on multimorbidity CIG-based clinical decision support (MGCDS) methods that target multimorbidity treatment plans. MATERIALS AND METHODS: Our methodology directly involves the research community of practice in (a) identifying functional features within the application domain; (b) defining exemplar case studies covering these features; and (c) solving the case studies using their developed computational methods-research groups detail their solutions and functional feature support in solution reports. Next, the study authors (d) perform a qualitative analysis of the solution reports, identifying and characterizing common themes (or dimensions) among the computational methods. This methodology is well suited to perform whitebox analysis, as it directly involves the respective developers in studying inner workings and feature support of computational methods. Moreover, the established evaluation parameters (e.g., features, case studies, themes) constitute a re-usable benchmark framework, which can be used to evaluate new computational methods as they are developed. We applied our community-of-practice-based evaluation methodology on MGCDS methods. RESULTS: Six research groups submitted comprehensive solution reports for the exemplar case studies. Solutions for two of these case studies were reported by all groups. We identified four evaluation dimensions: detection of adverse interactions, management strategy representation, implementation paradigms, and human-in-the-loop support. Based on our whitebox analysis, we present answers to the evaluation questions (i) and (ii) for MGCDS methods. DISCUSSION: The proposed evaluation methodology includes features of illuminative and comparison-based approaches; focusing on understanding rather than judging/scoring or identifying gaps in current methods. It involves answering evaluation questions with direct involvement of the research community of practice, who participate in setting up evaluation parameters and solving exemplar case studies. Our methodology was successfully applied to evaluate six MGCDS knowledge-intensive computational methods. We established that, while the evaluated methods provide a multifaceted set of solutions with different benefits and drawbacks, no single MGCDS method currently provides a comprehensive solution for MGCDS. CONCLUSION: We posit that our evaluation methodology, applied here to gain new insights into MGCDS, can be used to assess other types of knowledge-intensive computational methods and answer other types of evaluation questions. Our case studies can be accessed at our GitHub repository (https://github.com/william-vw/MGCDS).

Using graph rewriting to operationalize medical knowledge for the revision of concurrently applied clinical practice guidelines.

Clinical practice guidelines (CPGs) are patient management tools that synthesize medical knowledge into an actionable format. CPGs are disease specific with limited applicability to the management of complex patients suffering from multimorbidity. For the management of these patients, CPGs need to be augmented with secondary medical knowledge coming from a variety of knowledge repositories. The operationalization of this knowledge is key to increasing CPGs' uptake in clinical practice. In this work, we propose an approach to operationalizing secondary medical knowledge inspired by graph rewriting. We assume that the CPGs can be represented as task network models, and provide an approach for representing and applying codified medical knowledge to a specific patient encounter. We formally define revisions that model and mitigate adverse interactions between CPGs and we use a vocabulary of terms to instantiate these revisions. We demonstrate the application of our approach using synthetic and clinical examples. We conclude by identifying areas for future work with the vision of developing a theory of mitigation that will facilitate the development of comprehensive decision support for the management of multimorbid patients.

Why did AI get this one wrong? - Tree-based explanations of machine learning model predictions.

Increasingly complex learning methods such as boosting, bagging and deep learning have made ML models more accurate, but harder to interpret and explain, culminating in black-box machine learning models. Model developers and users alike are often presented with a trade-off between performance and intelligibility, especially in high-stakes applications like medicine. In the present article we propose a novel methodological approach for generating explanations for the predictions of a generic machine learning model, given a specific instance for which the prediction has been made. The method, named AraucanaXAI, is based on surrogate, locally-fitted classification and regression trees that are used to provide post-hoc explanations of the prediction of a generic machine learning model. Advantages of the proposed XAI approach include superior fidelity to the original model, ability to deal with non-linear decision boundaries, and native support to both classification and regression problems. We provide a packaged, open-source implementation of the AraucanaXAI method and evaluate its behaviour in a number of different settings that are commonly encountered in medical applications of AI. These include potential disagreement between the model prediction and physician's expert opinion and low reliability of the prediction due to data scarcity.

SATO (IDEAS expAnded wiTh BCIO): Workflow for designers of patient-centered mobile health behaviour change intervention applications.

Designing effective theory-driven digital behaviour change interventions (DBCI) is a challenging task. To ease the design process, and assist with knowledge sharing and evaluation of the DBCI, we propose the SATO (IDEAS expAnded wiTh BCIO) design workflow based on the IDEAS (Integrate, Design, Assess, and Share) framework and aligned with the Behaviour Change Intervention Ontology (BCIO). BCIO is a structural representation of the knowledge in behaviour change domain supporting evaluation of behaviour change interventions (BCIs) but it is not straightforward to utilise it during DBCI design. IDEAS (Integrate, Design, Assess, and Share) framework guides multi-disciplinary teams through the mobile health (mHealth) application development life-cycle but it is not aligned with BCIO entities. SATO couples BCIO entities with workflow steps and extends IDEAS Integrate stage with consideration of customisation and personalisation. We provide a checklist of the activities that should be performed during intervention planning with concrete examples and a tutorial accompanied with case studies from the Cancer Better Life Experience (CAPABLE) European project. In the process of creating this workflow, we found the necessity to extend the BCIO to support the scenarios of multiple clinical goals in the same application. To ensure the SATO steps are easy to follow for the incomers to the field, we performed a preliminary evaluation of the workflow with two knowledge engineers, working on novel mHealth app design tasks.

A Health eLearning Ontology and Procedural Reasoning Approach for Developing Personalized Courses to Teach Patients about Their Medical Condition and Treatment.

We propose a methodological framework to support the development of personalized courses that improve patients' understanding of their condition and prescribed treatment. Inspired by Intelligent Tutoring Systems (ITSs), the framework uses an eLearning ontology to express domain and learner models and to create a course. We combine the ontology with a procedural reasoning approach and precompiled plans to operationalize a design across disease conditions. The resulting courses generated by the framework are personalized across four patient axes-condition and treatment, comprehension level, learning style based on the VARK (Visual, Aural, Read/write, Kinesthetic) presentation model, and the level of understanding of specific course content according to Bloom's taxonomy. Customizing educational materials along these learning axes stimulates and sustains patients' attention when learning about their conditions or treatment options. Our proposed framework creates a personalized course that prepares patients for their meetings with specialists and educates them about their prescribed treatment. We posit that the improvement in patients' understanding of prescribed care will result in better outcomes and we validate that the constructs of our framework are appropriate for representing content and deriving personalized courses for two use cases: anticoagulation treatment of an atrial fibrillation patient and lower back pain management to treat a lumbar degenerative disc condition. We conduct a mostly qualitative study supported by a quantitative questionnaire to investigate the acceptability of the framework among the target patient population and medical practitioners.

The Case Manager: Driving Medical Reasoning in a Distributed Environment for Home Patient Monitoring.

The CAPABLE project has been funded by the EU Horizon 2020 Programme over the years 2020-24 to support home care. A system is being designed and implemented supporting remote monitoring and virtual coaching for cancer patients. The system is based on a distributed modular architecture involving many components encapsulating various knowledge. The Case Manager has been designed as a separate component with the aim of coordinating the problem solving strategies. A first version of the Case Manager has been released and used by the components in a prototypical scenario shown at the first project review.

MitPlan: A planning approach to mitigating concurrently applied clinical practice guidelines.

As the population ages, patients' complexity and the scope of their care is increasing. Over 60% of the population is 65 years of age or older and suffers from multi-morbidity, which is associated with two times as many patient-physician encounters. Yet clinical practice guidelines (CPGs) are developed to treat a single disease. To reconcile these two competing issues, previously we developed a framework for mitigation, i.e., identifying and addressing adverse interactions in multi-morbid patients managed according to multiple CPGs. That framework relies on first-order logic (FOL) to represent CPGs and secondary medical knowledge and FOL theorem proving to establish valid patient management plans. In the work presented here, we leverage our earlier research and simplify the mitigation process by representing it as a planning problem using the Planning Domain Definition Language (PDDL). This new framework, called MitPlan, identifies and addresses adverse interactions using durative planning actions that embody clinical actions (including medication administration and patient testing), supports a physician-defined length of planning horizons, and optimizes plans based on patient preferences and action costs. It supports a variety of criteria when developing management plans, including the total cost of prescribed treatment and the cost of the revisions to be introduced. The solution to MitPlan's planning problem is a sequence of timed actions that are easy to interpret when creating a management plan. We demonstrate MitPlan's capabilities using illustrative and clinical case studies.

Towards a framework for comparing functionalities of multimorbidity clinical decision support: A literature-based feature set and benchmark cases.

Multimorbidity, the coexistence of two or more health conditions, has become more prevalent as mortality rates in many countries have declined and their populations have aged. Multimorbidity presents significant difficulties for Clinical Decision Support Systems (CDSS), particularly in cases where recommendations from relevant clinical guidelines offer conflicting advice. A number of research groups are developing computer-interpretable guideline (CIG) modeling formalisms that integrate recommendations from multiple Clinical Practice Guidelines (CPGs) for knowledge-based multimorbidity decision support. In this paper we describe work towards the development of a framework for comparing the different approaches to multimorbidity CIG-based clinical decision support (MGCDS). We present (1) a set of features for MGCDS, which were derived using a literature review and evaluated by physicians using a survey, and (2) a set of benchmarking case studies, which illustrate the clinical application of these features. This work represents the first necessary step in a broader research program aimed at the development of a benchmark framework that allows for standardized and comparable MGCDS evaluations, which will facilitate the assessment of functionalities of MGCDS, as well as highlight important gaps in the state-of-the-art. We also outline our future work on developing the framework, specifically, (3) a standard for reporting MGCDS solutions for the benchmark case studies, and (4) criteria for evaluating these MGCDS solutions. We plan to conduct a large-scale comparison study of existing MGCDS based on the comparative framework.

An ontology-driven framework to support the dynamic formation of an interdisciplinary healthcare team.

BACKGROUND AND PURPOSE: Teamwork has become a modus operandi in healthcare and delivery of patient care by an interdisciplinary healthcare team (IHT) is now a prevailing modality of care. We argue that a formal and automated support framework is needed for an IHT to properly leverage information technology resources. Such a framework should allow for patient preferences and expand a representation of a clinical workflow with a formal model of dynamic formation of a team, especially with regards to team leader- and membership, and the assignment of tasks to team members. Our goal was to develop such a support framework, present its prototype software implementation and verify the implementation using a proof-of-concept use case. Specifically, we focused on clinical workflows for in-patient tertiary care and on patient preferences with regards to selecting team members and team leaders. MATERIALS AND METHODS: Drawing on the research on clinical teamwork we defined the conceptual foundations for the proposed framework. Then, we designed its architecture and used ontology-driven design and first-order logic with associated reasoning methods to create and operationalize architectural elements. Finally, we incorporated existing solutions for business workflow modeling and execution as a backend for implementing the proposed framework. RESULTS: We developed a Team and Workflow Management Framework (TWMF) with semantic components that allow for formalizing and operationalizing team formation in in-patient tertiary care setting and support provider-related patient preferences. We also created a prototype software implementation of TWMF using the IBM Business Process Manager platform. This implementation was evaluated in several simulated patient scenarios. CONCLUSIONS: TWMF integrates existing workflow technologies and extends them with the capabilities to support dynamic formation of an IHT. Results of this research can be used to support real-time execution of clinical workflows, or to simulate their execution in order to assess the impact of various conditions (e.g., patterns of work shifts, staffing) on IHT operations.

Incorporating Laboratory Values Into a Machine Learning Model Improves In-Hospital Mortality Predictions After Rapid Response Team Call.

OBJECTIVES: Machine learning models have been used to predict mortality among patients requiring rapid response team activation. The goal of our study was to assess the impact of adding laboratory values into the model. DESIGN: A gradient boosted decision tree model was derived and internally validated to predict a primary outcome of in-hospital mortality. The base model was then augmented with laboratory values. SETTING: Two tertiary care hospitals within The Ottawa Hospital network. PATIENTS: Inpatients over the age of 18 years who experienced a rapid response team activation between January 1, 2015, and May 31, 2016. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: A total of 2,061 rapid response team activations occurred during the study period. The in-hospital mortality rate was 29.4%. Patients who died were older (median age, 72 vs 68 yr; p < 0.001), had a longer length of stay (length of stay) prior to rapid response team activation (4 vs 2 d; p < 0.001), and more often had respiratory distress (31% vs 22%; p < 0.001). Our base model without laboratory values performed with an area under the receiver operating curve of 0.71 (95% CI, 0.71-0.72). When the base model was augmented with laboratory values, the area under the receiver operating curve improved to 0.77 (95% CI, 0.77-0.78). Important mortality predictors in the base model were age, estimated ratio of Pao2 to Fio2 (calculated using oxygen saturation and estimated Fio2), length of stay prior to rapid response team activation, and systolic blood pressure. CONCLUSIONS: Machine learning models can identify rapid response team patients at a high risk of mortality and potentially supplement clinical decision making. Incorporating laboratory values into model development significantly improved predictive performance in this study.

Ideating Mobile Health Behavioral Support for Compliance to Therapy for Patients with Chronic Disease: A Case Study of Atrial Fibrillation Management.

Poor patient compliance to therapy results in a worsening condition that often increases healthcare costs. In the MobiGuide project, we developed an evidence-based clinical decision-support system that delivered personalized reminders and recommendations to patients, helping to achieve higher therapy compliance. Yet compliance could still be improved and therefore building on the MobiGuide project experience, we designed a new component called the Motivational Patient Assistant (MPA) that is integrated within the MobiGuide architecture to further improve compliance. This component draws from psychological theories to provide behavioral support to improve patient engagement and thereby increasing patients' compliance. Behavior modification interventions are delivered via mobile technology at patients' home environments. Our approach was inspired by the IDEAS (Integrate, Design, Assess, and Share) framework for developing effective digital interventions to change health behavior; it goes beyond this approach by extending the Ideation phase' concepts into concrete backend architectural components and graphical user-interface designs that implement behavioral interventions. We describe in detail our ideation approach and how it was applied to design the user interface of MPA for anticoagulation therapy for the atrial fibrillation patients. We report results of a preliminary evaluation involving patients and care providers that shows the potential usefulness of the MPA for improving compliance to anticoagulation therapy.

Shared Decision-Making Ontology for a Healthcare Team Executing a Workflow, an Instantiation for Metastatic Spinal Cord Compression Management.

Regardless of potential benefits and better outcomes, adoption of shared decision-making between a patient and providers involved in his/her care is still in its infancy. This paper intends to fill this gap by formalizing shared decision-making, situating it as part of team-based care delivery, and incorporating workflow concepts allowing for identification of shared decision-making tasks. We accomplish that by creating novel shared decision-making ontology which constitutes the first step required in the development of a decision support system for shared decision-making. The proposed ontology formally defines and describes the key concepts and relations in the shared decision-making domain and lays the foundation for the formalization and support of the patient management process. We illustrate the applicability of the proposed ontology by creating its instantiation for the complex patient management scenario involving shared decision-making about the treatment of metastatic spinal cord compression.

Comprehensive mitigation framework for concurrent application of multiple clinical practice guidelines.

In this work we propose a comprehensive framework based on first-order logic (FOL) for mitigating (identifying and addressing) interactions between multiple clinical practice guidelines (CPGs) applied to a multi-morbid patient while also considering patient preferences related to the prescribed treatment. With this framework we respond to two fundamental challenges associated with clinical decision support: (1) concurrent application of multiple CPGs and (2) incorporation of patient preferences into the decision making process. We significantly expand our earlier research by (1) proposing a revised and improved mitigation-oriented representation of CPGs and secondary medical knowledge for addressing adverse interactions and incorporating patient preferences and (2) introducing a new mitigation algorithm. Specifically, actionable graphs representing CPGs allow for parallel and temporal activities (decisions and actions). Revision operators representing secondary medical knowledge support temporal interactions and complex revisions across multiple actionable graphs. The mitigation algorithm uses the actionable graphs, revision operators and available (and possibly incomplete) patient information represented in FOL. It relies on a depth-first search strategy to find a valid sequence of revisions and uses theorem proving and model finding techniques to identify applicable revision operators and to establish a management scenario for a given patient if one exists. The management scenario defines a safe (interaction-free) and preferred set of activities together with possible patient states. We illustrate the use of our framework with a clinical case study describing two patients who suffer from chronic kidney disease, hypertension, and atrial fibrillation, and who are managed according to CPGs for these diseases. While in this paper we are primarily concerned with the methodological aspects of mitigation, we also briefly discuss a high-level proof of concept implementation of the proposed framework in the form of a clinical decision support system (CDSS). The proposed mitigation CDSS "insulates" clinicians from the complexities of the FOL representations and provides semantically meaningful summaries of mitigation results. Ultimately we plan to implement the mitigation CDSS within our MET (Mobile Emergency Triage) decision support environment.