Consolidated Health Economic Evaluation Reporting Standards for Interventions that use Artificial Intelligence (CHEERS-AI).

OBJECTIVES: Economic evaluations (EEs) are commonly used by decision makers to understand the value of health interventions. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS 2022) provide reporting guidelines for EEs. Healthcare systems will increasingly see new interventions that use artificial intelligence (AI) to perform their function. We developed CHEERS-AI to ensure EEs of AI-based health interventions are reported in a transparent and reproducible manner. METHODS: Potential CHEERS-AI reporting items were informed by 2 published systematic literature reviews of EEs and a contemporary update. A Delphi study was conducted using 3 survey rounds to elicit multidisciplinary expert views on 26 potential items, through a 9-point Likert rating scale and qualitative comments. An online consensus meeting was held to finalise outstanding reporting items. A digital health patient group reviewed the final checklist from a patient perspective. RESULTS: A total of 58 participants responded to survey round 1, 42 and 31 of whom responded to rounds 2 and 3, respectively. Nine participants joined the consensus meeting. Ultimately, 38 reporting items were included in CHEERS-AI. They comprised the 28 original CHEERS 2022 items, plus 10 new AI-specific reporting items. Additionally, 8 of the original CHEERS 2022 items were elaborated on to ensure AI-specific nuance is reported. CONCLUSIONS: CHEERS-AI should be used when reporting an EE of an intervention that uses AI to perform its function. CHEERS-AI will help decision makers and reviewers to understand important AI-specific details of an intervention, and any implications for the EE methods used and cost-effectiveness conclusions.

Leveraging machine learning for predicting acute graft-versus-host disease grades in allogeneic hematopoietic cell transplantation for T-cell prolymphocytic leukaemia.

Orphan diseases, exemplified by T-cell prolymphocytic leukemia, present inherent challenges due to limited data availability and complexities in effective care. This study delves into harnessing the potential of machine learning to enhance care strategies for orphan diseases, specifically focusing on allogeneic hematopoietic cell transplantation (allo-HCT) in T-cell prolymphocytic leukemia. The investigation evaluates how varying numbers of variables impact model performance, considering the rarity of the disease. Utilizing data from the Center for International Blood and Marrow Transplant Research, the study scrutinizes outcomes following allo-HCT for T-cell prolymphocytic leukemia. Diverse machine learning models were developed to forecast acute graft-versus-host disease (aGvHD) occurrence and its distinct grades post-allo-HCT. Assessment of model performance relied on balanced accuracy, F1 score, and ROC AUC metrics. The findings highlight the Linear Discriminant Analysis (LDA) classifier achieving the highest testing balanced accuracy of 0.58 in predicting aGvHD. However, challenges arose in its performance during multi-class classification tasks. While affirming the potential of machine learning in enhancing care for orphan diseases, the study underscores the impact of limited data and disease rarity on model performance.

Predicting Emotion with Biosignals: A Comparison of Classification and Regression Models for Estimating Valence and Arousal Level Using Wearable Sensors.

This study aims to predict emotions using biosignals collected via wrist-worn sensor and evaluate the performance of different prediction models. Two dimensions of emotions were considered: valence and arousal. The data collected by the sensor were used in conjunction with target values obtained from questionnaires. A variety of classification and regression models were compared, including Long Short-Term Memory (LSTM) models. Additionally, the effects of different normalization methods and the impact of using different sensors were studied, and the way in which the results differed between the study subjects was analyzed. The results revealed that regression models generally performed better than classification models, with LSTM regression models achieving the best results. The normalization method called baseline reduction was found to be the most effective, and when used with an LSTM-based regression model it achieved high accuracy in detecting valence (mean square error = 0.43 and R2-score = 0.71) and arousal (mean square error = 0.59 and R2-score = 0.81). Moreover, it was found that even if all biosignals were not used in the training phase, reliable models could be obtained; in fact, for certain study subjects the best results were obtained using only a few of the sensors.

Data-Driven Identification of Long-Term Glycemia Clusters and Their Individualized Predictors in Finnish Patients with Type 2 Diabetes.

Purpose: To gain an understanding of the heterogeneous group of type 2 diabetes (T2D) patients, we aimed to identify patients with the homogenous long-term HbA1c trajectories and to predict the trajectory membership for each patient using explainable machine learning methods and different clinical-, treatment-, and socio-economic-related predictors. Patients and Methods: Electronic health records data covering primary and specialized healthcare on 9631 patients having T2D diagnosis were extracted from the North Karelia region, Finland. Six-year HbA1c trajectories were examined with growth mixture models. Linear discriminant analysis and neural networks were applied to predict the trajectory membership individually. Results: Three HbA1c trajectories were distinguished over six years: "stable, adequate" (86.5%), "improving, but inadequate" (7.3%), and "fluctuating, inadequate" (6.2%) glycemic control. Prior glucose levels, duration of T2D, use of insulin only, use of insulin together with some oral antidiabetic medications, and use of only metformin were the most important predictors for the long-term treatment balance. The prediction model had a balanced accuracy of 85% and a receiving operating characteristic area under the curve of 91%, indicating high performance. Moreover, the results based on SHAP (Shapley additive explanations) values show that it is possible to explain the outcomes of machine learning methods at the population and individual levels. Conclusion: Heterogeneity in long-term glycemic control can be predicted with confidence by utilizing information from previous HbA1c levels, fasting plasma glucose, duration of T2D, and use of antidiabetic medications. In future, the expected development of HbA1c could be predicted based on the patient's unique risk factors offering a practical tool for clinicians to support treatment planning.

Barriers to Use Artificial Intelligence Methodologies in Health Technology Assessment in Central and East European Countries.

The aim of this paper is to identify the barriers that are specifically relevant to the use of Artificial Intelligence (AI)-based evidence in Central and Eastern European (CEE) Health Technology Assessment (HTA) systems. The study relied on two main parallel sources to identify barriers to use AI methodologies in HTA in CEE, including a scoping literature review and iterative focus group meetings with HTx team members. Most of the other selected articles discussed AI from a clinical perspective (n = 25), and the rest are from regulatory perspective (n = 13), and transfer of knowledge point of view (n = 3). Clinical areas studied are quite diverse-from pediatric, diabetes, diagnostic radiology, gynecology, oncology, surgery, psychiatry, cardiology, infection diseases, and oncology. Out of all 38 articles, 25 (66%) describe the AI method and the rest are more focused on the utilization barriers of different health care services and programs. The potential barriers could be classified as data related, methodological, technological, regulatory and policy related, and human factor related. Some of the barriers are quite similar, especially concerning the technologies. Studies focusing on the AI usage for HTA decision making are scarce. AI and augmented decision making tools are a novel science, and we are in the process of adapting it to existing needs. HTA as a process requires multiple steps, multiple evaluations which rely on heterogenous data. Therefore, the observed range of barriers come as a no surprise, and experts in the field need to give their opinion on the most important barriers in order to develop recommendations to overcome them and to disseminate the practical application of these tools.

ClinFlow - An Interactive Application for Clinical Data Mining.

Analyzing clinical data comes with many challenges. Medical expertise combined with statistical and programming knowledge must go hand-in-hand when applying data mining methods on clinical datasets. This work aims at bridging the gap between clinical expertise and computer science knowledge by providing an application for clinical data analysis with no requirement for statistical programming knowledge. Our tool allows clinical researchers to conduct data processing and visualization in an interactive environment, thus providing an assisting tool for clinical studies. The application was experimentally evaluated with an analysis of Type 1 Diabetes clinical data. The results obtained with the tool are in line with the domain literature, demonstrating the value of our application in data exploration and hypothesis testing.

Comparison of Regression and Classification Models for User-Independent and Personal Stress Detection.

In this article, regression and classification models are compared for stress detection. Both personal and user-independent models are experimented. The article is based on publicly open dataset called AffectiveROAD, which contains data gathered using Empatica E4 sensor and unlike most of the other stress detection datasets, it contains continuous target variables. The used classification model is Random Forest and the regression model is Bagged tree based ensemble. Based on experiments, regression models outperform classification models, when classifying observations as stressed or not-stressed. The best user-independent results are obtained using a combination of blood volume pulse and skin temperature features, and using these the average balanced accuracy was 74.1% with classification model and 82.3% using regression model. In addition, regression models can be used to estimate the level of the stress. Moreover, the results based on models trained using personal data are not encouraging showing that biosignals have a lot of variation not only between the study subjects but also between the session gathered from the same person. On the other hand, it is shown that with subject-wise feature selection for user-independent model, it is possible to improve recognition models more than by using personal training data to build personal models. In fact, it is shown that with subject-wise feature selection, the average detection rate can be improved as much as 4%-units, and it is especially useful to reduce the variance in the recognition rates between the study subjects.

Incremental Learning to Personalize Human Activity Recognition Models: The Importance of Human AI Collaboration.

This study presents incremental learning based methods to personalize human activity recognition models. Initially, a user-independent model is used in the recognition process. When a new user starts to use the human activity recognition application, personal streaming data can be gathered. Of course, this data does not have labels. However, there are three different ways to obtain this data: non-supervised, semi-supervised, and supervised. The non-supervised approach relies purely on predicted labels, the supervised approach uses only human intelligence to label the data, and the proposed method for semi-supervised learning is a combination of these two: It uses artificial intelligence (AI) in most cases to label the data but in uncertain cases it relies on human intelligence. After labels are obtained, the personalization process continues by using the streaming data and these labels to update the incremental learning based model, which in this case is Learn++. Learn++ is an ensemble method that can use any classifier as a base classifier, and this study compares three base classifiers: linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and classification and regression tree (CART). Moreover, three datasets are used in the experiment to show how well the presented method generalizes on different datasets. The results show that personalized models are much more accurate than user-independent models. On average, the recognition rates are: 87.0% using the user-independent model, 89.1% using the non-supervised personalization approach, 94.0% using the semi-supervised personalization approach, and 96.5% using the supervised personalization approach. This means that by relying on predicted labels with high confidence, and asking the user to label only uncertain observations (6.6% of the observations when using LDA, 7.7% when using QDA, and 18.3% using CART), almost as low error rates can be achieved as by using the supervised approach, in which labeling is fully based on human intelligence.

Using Sleep Time Data from Wearable Sensors for Early Detection of Migraine Attacks.

The migraine is a chronic, incapacitating neurovascular disorder, characterized by attacks of severe headache and autonomic nervous system dysfunction. Among the working age population, the costs of migraine are 111 billion euros in Europe alone. The early detection of migraine attacks would reduce these costs, as it would shorten the migraine attack by enabling correct timing when taking preventive medication. In this article, whether it is possible to detect migraine attacks beforehand using wearable sensors is studied, and t preliminary results about how accurate the recognition can be are provided. The data for the study were collected from seven study subjects using a wrist-worn Empatica E4 sensor, which measures acceleration, galvanic skin response, blood volume pulse, heart rate and heart rate variability, and temperature. Only sleep time data were used in this study. A novel method to increase the number of training samples is introduced, and the results show that, using personal recognition models and quadratic discriminant analysis as a classifier, balanced accuracy for detecting attacks one night prior is over 84%. While this detection rate is high, the results also show that balance accuracy varies greatly between study subjects, which shows how complicated the problem actually is. However, at this point, the results are preliminary as the data set contains only seven study subjects, so these do not cover all migraine types. If the findings of this article can be confirmed in a larger population, it may potentially contribute to early diagnosis of migraine attacks.

Feasibility of Gamified Mobile Service Aimed at Physical Activation in Young Men: Population-Based Randomized Controlled Study (MOPO).

BACKGROUND: The majority of young people do not meet the recommendations on physical activity for health. New innovative ways to motivate young people to adopt a physically active lifestyle are needed. OBJECTIVE: The study aimed to study the feasibility of an automated, gamified, tailored Web-based mobile service aimed at physical and social activation among young men. METHODS: A population-based sample of 496 young men (mean age 17.8 years [standard deviation 0.6]) participated in a 6-month randomized controlled trial (MOPO study). Participants were randomized to an intervention (n=250) and a control group (n=246). The intervention group was given a wrist-worn physical activity monitor (Polar Active) with physical activity feedback and access to a gamified Web-based mobile service, providing fitness guidelines, tailored health information, advice of youth services, social networking, and feedback on physical activity. Through the trial, the physical activity of the men in the control group was measured continuously with an otherwise similar monitor but providing only the time of day and no feedback. The primary outcome was the feasibility of the service based on log data and questionnaires. Among completers, we also analyzed the change in anthropometry and fitness between baseline and 6 months and the change over time in weekly time spent in moderate to vigorous physical activity. RESULTS: Mobile service users considered the various functionalities related to physical activity important. However, compliance of the service was limited, with 161 (64.4%, 161/250) participants visiting the service, 118 (47.2%, 118/250) logging in more than once, and 41 (16.4%, 41/250) more than 5 times. Baseline sedentary time was higher in those who uploaded physical activity data until the end of the trial (P=.02). A total of 187 (74.8%, 187/250) participants in the intervention and 167 (67.9%, 167/246) in the control group participated in the final measurements. There were no differences in the change in anthropometry and fitness from baseline between the groups, whereas waist circumference was reduced in the most inactive men within the intervention group (P=.01). Among completers with valid physical activity data (n=167), there was a borderline difference in the change in mean daily time spent in moderate to vigorous physical activity between the groups (11.9 min vs -9.1 min, P=.055, linear mixed model). Within the intervention group (n=87), baseline vigorous physical activity was inversely associated with change in moderate to vigorous physical activity during the trial (R=-.382, P=.01). CONCLUSIONS: The various functionalities related to physical activity of the gamified tailored mobile service were considered important. However, the compliance was limited. Within the current setup, the mobile service had no effect on anthropometry or fitness, except reduced waist circumference in the most inactive men. Among completers with valid physical activity data, the trial had a borderline positive effect on moderate to vigorous physical activity. Further development is needed to improve the feasibility and adherence of an integrated multifunctional service. TRIAL REGISTRATION: Clinicaltrials.gov NCT01376986; http://clinicaltrials.gov/ct2/show/NCT01376986 (Archived by WebCite at http://www.webcitation.org/6tjdmIroA).