Use of the FallAkte Plus System as an IT Infrastructure for the North Rhine-Westphalian General Practice Research Network: Mixed Methods Usability Study.

BACKGROUND: Primary care research networks can generate important information in the setting where most patients are seen and treated. However, this requires a suitable IT infrastructure (ITI), which the North Rhine-Westphalian general practice research network is looking to implement. OBJECTIVE: This mixed methods research study aims to evaluate (study 1) requirements for an ITI and (study 2) the usability of an IT solution already available on the market, the FallAkte Plus (FA+) system for the North Rhine-Westphalian general practice research network, which comprises 8 primary care university institutes in Germany's largest state. METHODS: In study 1, a survey was conducted among researchers from the institutes to identify the requirements for a suitable ITI. The questionnaire consisted of standardized questions with open-ended responses. In study 2, a mixed method approach combining a think-aloud approach and a quantitative survey was used to evaluate the usability and acceptance of the FA+ system among 3 user groups: researchers, general practitioners, and practice assistants. Respondents were asked to assess the usability with the validated system usability scale and to test a short questionnaire on vaccination management through FA+. RESULTS: In study 1, five of 8 institutes participated in the requirements survey. A total of 32 user requirements related primarily to study management were identified, including data entry, data storage, and user access management. In study 2, a total of 36 participants (24 researchers and 12 general practitioners or practice assistants) were surveyed in the mixed methods study of an already existing IT solution. The tutorial video and handouts explaining how to use the FA+ system were well received. Researchers, unlike practice personnel, were concerned about data security and data protection regarding the system's emergency feature, which enables access to all patient data. The median overall system usability scale rating was 60 (IQR 33.0-85.0), whereby practice personnel (median 82, IQR 58.0-94.0) assigned higher ratings than researchers (median 44, IQR 14.0-61.5). Users appreciated the option to integrate data from practices and other health care facilities. However, they voted against the use of the FA+ system due to a lack of support for various study formats. CONCLUSIONS: Usability assessments vary markedly by professional group and role. In its current stage of development, the FA+ system does not fully meet the requirements for a suitable ITI. Improvements in the user interface, performance, interoperability, security, and advanced features are necessary to make it more effective and user-friendly. Collaborating with end users and incorporating their feedback are crucial for the successful development of any practice network research ITI.

The Predominance of the Health-Promoting Patterns of Work Behavior and Experience in General Practice Teams-Results of the IMPROVEjob Study.

This study aims to identify the distribution of the "Work-related behavior and experience patterns" (Arbeitsbezogenes Verhaltens-und Erlebnismuster, AVEM) in general practitioners and their teams by using baseline data of the IMPROVEjob study. Members of 60 general practices with 84 physicians in a leadership position, 28 employed physicians, and 254 practice assistants participated in a survey in 2019 and 2020. In this analysis, we focused on AVEM variables. Age, practice years, work experience, and working time were used as control variables in the Spearman Rho correlations and analysis of variance. The majority of the participants (72.1%) revealed a health-promoting pattern (G or S). Three of eleven AVEM dimensions were above the norm for the professional group "employed physicians". The AVEM dimensions "striving for perfection" (p < 0.001), "experience of success at work" (p < 0.001), "satisfaction with life" (p = 0.003), and "experience of social support" (p = 0.019) differed significantly between the groups' practice owners and practice assistants, with the practice owners achieving the higher values, except for experience of social support. Practice affiliation had no effect on almost all AVEM dimensions. We found a high prevalence of AVEM health-promoting patterns in our sample. Nearly half of the participants in all professional groups showed an unambitious pattern (S). Adapted interventions for the represented AVEM patterns are possible and should be utilized for maintaining mental health among general practice teams.

Prediction of Chronic Stress and Protective Factors in Adults: Development of an Interpretable Prediction Model Based on XGBoost and SHAP Using National Cross-sectional DEGS1 Data.

BACKGROUND: Chronic stress is highly prevalent in the German population. It has known adverse effects on mental health, such as burnout and depression. Known long-term effects of chronic stress are cardiovascular disease, diabetes, and cancer. OBJECTIVE: This study aims to derive an interpretable multiclass machine learning model for predicting chronic stress levels and factors protecting against chronic stress based on representative nationwide data from the German Health Interview and Examination Survey for Adults, which is part of the national health monitoring program. METHODS: A data set from the German Health Interview and Examination Survey for Adults study including demographic, clinical, and laboratory data from 5801 participants was analyzed. A multiclass eXtreme Gradient Boosting (XGBoost) model was constructed to classify participants into 3 categories including low, middle, and high chronic stress levels. The model's performance was evaluated using the area under the receiver operating characteristic curve, precision, recall, specificity, and the F1-score. Additionally, SHapley Additive exPlanations was used to interpret the prediction XGBoost model and to identify factors protecting against chronic stress. RESULTS: The multiclass XGBoost model exhibited the macroaverage scores, with an area under the receiver operating characteristic curve of 81%, precision of 63%, recall of 52%, specificity of 78%, and F1-score of 54%. The most important features for low-level chronic stress were male gender, very good general health, high satisfaction with living space, and strong social support. CONCLUSIONS: This study presents a multiclass interpretable prediction model for chronic stress in adults in Germany. The explainable artificial intelligence technique SHapley Additive exPlanations identified relevant protective factors for chronic stress, which need to be considered when developing interventions to reduce chronic stress.

Chronic stress in practice assistants: An analytic approach comparing four machine learning classifiers with a standard logistic regression model.

BACKGROUND: Occupational stress is associated with adverse outcomes for medical professionals and patients. In our cross-sectional study with 136 general practices, 26.4% of 550 practice assistants showed high chronic stress. As machine learning strategies offer the opportunity to improve understanding of chronic stress by exploiting complex interactions between variables, we used data from our previous study to derive the best analytic model for chronic stress: four common machine learning (ML) approaches are compared to a classical statistical procedure. METHODS: We applied four machine learning classifiers (random forest, support vector machine, K-nearest neighbors', and artificial neural network) and logistic regression as standard approach to analyze factors contributing to chronic stress in practice assistants. Chronic stress had been measured by the standardized, self-administered TICS-SSCS questionnaire. The performance of these models was compared in terms of predictive accuracy based on the 'operating area under the curve' (AUC), sensitivity, and positive predictive value. FINDINGS: Compared to the standard logistic regression model (AUC 0.636, 95% CI 0.490-0.674), all machine learning models improved prediction: random forest +20.8% (AUC 0.844, 95% CI 0.684-0.843), artificial neural network +12.4% (AUC 0.760, 95% CI 0.605-0.777), support vector machine +15.1% (AUC 0.787, 95% CI 0.634-0.802), and K-nearest neighbours +7.1% (AUC 0.707, 95% CI 0.556-0.735). As best prediction model, random forest showed a sensitivity of 99% and a positive predictive value of 79%. Using the variable frequencies at the decision nodes of the random forest model, the following five work characteristics influence chronic stress: too much work, high demand to concentrate, time pressure, complicated tasks, and insufficient support by practice leaders. CONCLUSIONS: Regarding chronic stress prediction, machine learning classifiers, especially random forest, provided more accurate prediction compared to classical logistic regression. Interventions to reduce chronic stress in practice personnel should primarily address the identified workplace characteristics.