A scoping review of rule-based clinical decision support malfunctions.

OBJECTIVE: Conduct a scoping review of research studies that describe rule-based clinical decision support (CDS) malfunctions. MATERIALS AND METHODS: In April 2022, we searched three bibliographic databases (MEDLINE, CINAHL, and Embase) for literature referencing CDS malfunctions. We coded the identified malfunctions according to an existing CDS malfunction taxonomy and added new categories for factors not already captured. We also extracted and summarized information related to the CDS system, such as architecture, data source, and data format. RESULTS: Twenty-eight articles met inclusion criteria, capturing 130 malfunctions. Architectures used included stand-alone systems (eg, web-based calculator), integrated systems (eg, best practices alerts), and service-oriented architectures (eg, distributed systems like SMART or CDS Hooks). No standards-based CDS malfunctions were identified. The "Cause" category of the original taxonomy includes three new types (organizational policy, hardware error, and data source) and two existing causes were expanded to include additional layers. Only 29 malfunctions (22%) described the potential impact of the malfunction on patient care. DISCUSSION: While a substantial amount of research on CDS exists, our review indicates there is a limited focus on CDS malfunctions, with even less attention on malfunctions associated with modern delivery architectures such as SMART and CDS Hooks. CONCLUSION: CDS malfunctions can and do occur across several different care delivery architectures. To account for advances in health information technology, existing taxonomies of CDS malfunctions must be continually updated. This will be especially important for service-oriented architectures, which connect several disparate systems, and are increasing in use.

Digital health technologies for high-risk pregnancy management: three case studies using Digilego framework.

Objective: High-risk pregnancy (HRP) conditions such as gestational diabetes mellitus (GDM), hypertension (HTN), and peripartum depression (PPD) affect maternal and neonatal health. Patient engagement is critical for effective HRP management (HRPM). While digital technologies and analytics hold promise, emerging research indicates limited and suboptimal support offered by the highly prevalent pregnancy digital solutions within the commercial marketplace. In this article, we describe our efforts to develop a portfolio of digital products leveraging advances in social computing, data science, and digital health. Methods: We describe three studies that leverage core methods from Digilego digital health development framework to (1) conduct large-scale social media analysis (n = 55 301 posts) to understand population-level patterns in women's needs, (2) architect a digital repository to enable women curate HRP related information, and (3) develop a digital platform to support PPD prevention. We applied a combination of qualitative coding, machine learning, theory-mapping, and programmatic implementation of theory-linked digital features. Further, we conducted preliminary testing of the resulting products for acceptance with sample of pregnant women for GDM/HTN information management (n = 10) and PPD prevention (n = 30). Results: Scalable social computing models using deep learning classifiers with reasonable accuracy have allowed us to capture and examine psychosociobehavioral drivers associated with HRPM. Our work resulted in two digital health solutions, MyPregnancyChart and MomMind are developed. Initial evaluation of both tools indicates positive acceptance from potential end users. Further evaluation with MomMind revealed statistically significant improvements (P < .05) in PPD recognition and knowledge on how to seek PPD information. Discussion: Digilego framework provides an integrative methodological lens to gain micro-macro perspective on women's needs, theory integration, engagement optimization, as well as subsequent feature and content engineering, which can be organized into core and specialized digital pathways for women engagement in disease management. Conclusion: Future works should focus on implementation and testing of digital solutions that facilitate women to capture, aggregate, preserve, and utilize, otherwise siloed, prenatal information artifacts for enhanced self-management of their high-risk conditions, ultimately leading to improved health outcomes.

Mobile Health Applications for Postpartum Depression Management: A Theory-Informed Analysis of Change-Use-Engagement (CUE) Criteria in the Digital Environment.

Postpartum Depression (PPD) is the most common childbirth complication, with approximately 15% of postpartum women experiencing depression symptoms. Mobile applications have potential to expand delivery of mental health interventions. However, our understanding of how these tools engage women with PPD and facilitate positive behavioral changes is limited. In our paper, we analyze 15 commercial PPD applications to understand their role as facilitators of change, engagement, and sustained use. Applications reviewed contained an average of four theory-based behavioral change techniques, and highest patient engagement level reached was to empower patients through patient-generated data. Heuristic violations were identified in areas including user control and freedom, aesthetic and minimalist design, and help and documentation. An inverse correlation was found between the number of theory-based behavior change features and patient engagement. Findings suggest underserved populations may suffer further limitations accessing relevant health resources in the current application market.

Digital Technology Needs in Maternal Mental Health: A Qualitative Inquiry.

Digital technologies offer many opportunities to improve mental healthcare management for women seeking pre- and-postnatal care. They provide a discrete, practical medium that is well-suited for the sensitive nature of mental health. Women who are more prone to experiencing peripartum depression (PPD), such as those of low-socioeconomic background or in high-risk pregnancies, can benefit the most from such technologies. However, current digital interventions directed towards this population provide suboptimal support, and their responsiveness to end user needs is quite limited. Our objective is to understand the digital terrain of information needs for low-socioeconomic status women with high-risk pregnancies, specifically within the management of their mental health. This qualitative study consists of semi-structured focus groups and interviews with a sample of nineteen patients. A total of eleven core themes emerged from participant comments. Resulting themes highlighted the need for digital technologies that promote personalized care, a sense of community, and improved provider communication.

Decreased Plasma Levels of Angiotensin-Converting Enzyme Among Patients With Bipolar Disorder.

BACKGROUND: Dysfunctions in the renin-angiotensin system (RAS) seem to be involved in the pathophysiology of several mental illness, including schizophrenia and mood disorders. We carried out a cross-sectional study assessing the levels of RAS-related molecules among bipolar disorder (BD) patients compared to healthy controls. METHODS: our sample consisted of 30 outpatients with BD type 1 (10 males, 20 females, age = 35.53 ± 10.59 years, 14 euthymic, 16 experiencing mood episodes) and 30 healthy controls (10 males, 20 females, age = 34.83 ± 11.49 years). Plasma levels of angiotensin-converting enzyme (ACE), angiotensin-converting enzyme 2 (ACE2), angiotensin-II (Ang II), and angiotensin (1-7) [Ang-(1-7)] were determined by ELISA. RESULTS: BD patients experiencing ongoing mood episodes had significantly lower ACE levels compared to controls (median: 459.00 vs. 514.10, p < 0.05). There was no association between the levels of these biomarkers and clinical parameters. CONCLUSION: Our findings support the involvement of RAS dysfunction in the pathophysiology of BD. Considering the potential therapeutic implications linked to a better understanding of the role of RAS dysfunction in BD, studies allowing a better characterization of RAS-related molecules level and activity across different mood states are of high interest.

Application of Human Factors Methods to Understand Missed Follow-up of Abnormal Test Results.

OBJECTIVE: This study demonstrates application of human factors methods for understanding causes for lack of timely follow-up of abnormal test results ("missed results") in outpatient settings. METHODS: We identified 30 cases of missed test results by querying electronic health record data, developed a critical decision method (CDM)-based interview guide to understand decision-making processes, and interviewed physicians who ordered these tests. We analyzed transcribed responses using a contextual inquiry (CI)-based methodology to identify contextual factors contributing to missed results. We then developed a CI-based flow model and conducted a fault tree analysis (FTA) to identify hierarchical relationships between factors that delayed action. RESULTS: The flow model highlighted barriers in information flow and decision making, and the hierarchical model identified relationships between contributing factors for delayed action. Key findings including underdeveloped methods to track follow-up, as well as mismatches, in communication channels, timeframes, and expectations between patients and physicians. CONCLUSION: This case report illustrates how human factors-based approaches can enable analysis of contributing factors that lead to missed results, thus informing development of preventive strategies to address them.

Generating sequential electronic health records using dual adversarial autoencoder.

OBJECTIVE: Recent studies on electronic health records (EHRs) started to learn deep generative models and synthesize a huge amount of realistic records, in order to address significant privacy issues surrounding the EHR. However, most of them only focus on structured records about patients' independent visits, rather than on chronological clinical records. In this article, we aim to learn and synthesize realistic sequences of EHRs based on the generative autoencoder. MATERIALS AND METHODS: We propose a dual adversarial autoencoder (DAAE), which learns set-valued sequences of medical entities, by combining a recurrent autoencoder with 2 generative adversarial networks (GANs). DAAE improves the mode coverage and quality of generated sequences by adversarially learning both the continuous latent distribution and the discrete data distribution. Using the MIMIC-III (Medical Information Mart for Intensive Care-III) and UT Physicians clinical databases, we evaluated the performances of DAAE in terms of predictive modeling, plausibility, and privacy preservation. RESULTS: Our generated sequences of EHRs showed the comparable performances to real data for a predictive modeling task, and achieved the best score in plausibility evaluation conducted by medical experts among all baseline models. In addition, differentially private optimization of our model enables to generate synthetic sequences without increasing the privacy leakage of patients' data. CONCLUSIONS: DAAE can effectively synthesize sequential EHRs by addressing its main challenges: the synthetic records should be realistic enough not to be distinguished from the real records, and they should cover all the training patients to reproduce the performance of specific downstream tasks.

Digilego for Peripartum Depression: A Novel Patient-Facing Digital Health Instantiation.

Digital health technologies offer unique opportunities to improve health outcomes for mental health conditions such as peripartum depression (PPD), a disorder that affects approximately 10-15% of women in the U.S. every year. In this paper, we present the adaption of a digital technology development framework, Digilego, in the context of PPD. Methods include mapping of the Behavior Intervention Technology (BIT) model and the Patient Engagement Framework (PEF) to translate patient needs captured through focus groups. This informs formative development and implementation of digital health features for optimal patient engagement in PPD screening and management. Results show an array ofPPD-specific Digilego blocks ("My Diary", "Mom Talk", "My Care", "Library", "How am I doing today?"). Initial evaluation results from comparative market analysis indicate that our proposed platform offers advantageous technology aspects. Limitations and future work in areas of interdisciplinary care coordination and patient engagement optimization are discussed.

NewCope: A Theory-Linked Mobile Application for Stress Education and Management.

The negative effects of long-term stress on health outcomes are well-documented. Emerging technologies that harness mobile technologies have been linked to positive effects on stress management. However, the ways in which existing inter- and intrapersonal theories of behavior change are integrated into development processes of these mHealth technologies for stress coping are limited. In this paper, we present a novel theory-driven approach to develop and implement a sustainable mobile application for stress education and management. Specifically, we integrate the taxonomy of Behavior Change Techniques and user engagement framework to model and adapt theory-driven techniques in the context of mobile technologies. A total of 12 behavior change techniques were incorporated into our mobile application. Initial user evaluation and usability testing was conducted. Results indicate heuristic modifications could improve overall delivery of content, and potential user satisfaction is likely. We conclude that this novel approach may have implications well beyond stress management.

Barriers and facilitators impacting reliability of the electronic health record-facilitated total testing process.

OBJECTIVE: Despite ongoing efforts to improve reliability of the total testing process (TTP), breakdowns continue to occur resulting in diagnostic delays and suboptimal patient outcomes. We performed an exploratory study to identify factors that impact TTP reliability in electronic health record (EHR)-enabled care. MATERIALS AND METHODS: We interviewed experts at three large EHR-enabled health care organizations and identified all TTP steps performed from clinician test ordering to result communication to patients. Findings from all sites were combined to develop a detailed process map of known TTP activities. We additionally asked experts about factors that positively or negatively impacted TTP reliability at each step. We describe the specific TTP steps identified and associated barriers and facilitators to TTP reliability. RESULTS: We interviewed 39 experts involved in or overseeing the TTP. Most TTP activities identified were similar across sites, but we found significant differences with test order transmission to diagnostic services and relay of results back to clinicians and patients. Twenty-five unique barriers were identified related to technology and EHR usability issues, time and resource constraints, suboptimal clinic workflows, patient-related factors, information access limitations, and insufficient clinician training. Twenty-four unique facilitators were identified related to personnel training, workflow optimization and standardization, helpful EHR features, and improved electronic communication between clinics and diagnostic services. DISCUSSION: Barriers related to EHR usability and with communication between clinicians and diagnostic services increase TTP vulnerability and should be targeted by future efforts to improve process reliability. Several facilitators identified in the study could inform future strategies and solutions to improve TTP reliability.

DataMed - an open source discovery index for finding biomedical datasets.

OBJECTIVE: Finding relevant datasets is important for promoting data reuse in the biomedical domain, but it is challenging given the volume and complexity of biomedical data. Here we describe the development of an open source biomedical data discovery system called DataMed, with the goal of promoting the building of additional data indexes in the biomedical domain. MATERIALS AND METHODS: DataMed, which can efficiently index and search diverse types of biomedical datasets across repositories, is developed through the National Institutes of Health-funded biomedical and healthCAre Data Discovery Index Ecosystem (bioCADDIE) consortium. It consists of 2 main components: (1) a data ingestion pipeline that collects and transforms original metadata information to a unified metadata model, called DatA Tag Suite (DATS), and (2) a search engine that finds relevant datasets based on user-entered queries. In addition to describing its architecture and techniques, we evaluated individual components within DataMed, including the accuracy of the ingestion pipeline, the prevalence of the DATS model across repositories, and the overall performance of the dataset retrieval engine. RESULTS AND CONCLUSION: Our manual review shows that the ingestion pipeline could achieve an accuracy of 90% and core elements of DATS had varied frequency across repositories. On a manually curated benchmark dataset, the DataMed search engine achieved an inferred average precision of 0.2033 and a precision at 10 (P@10, the number of relevant results in the top 10 search results) of 0.6022, by implementing advanced natural language processing and terminology services. Currently, we have made the DataMed system publically available as an open source package for the biomedical community.

User needs analysis and usability assessment of DataMed - a biomedical data discovery index.

OBJECTIVE: To present user needs and usability evaluations of DataMed, a Data Discovery Index (DDI) that allows searching for biomedical data from multiple sources. MATERIALS AND METHODS: We conducted 2 phases of user studies. Phase 1 was a user needs analysis conducted before the development of DataMed, consisting of interviews with researchers. Phase 2 involved iterative usability evaluations of DataMed prototypes. We analyzed data qualitatively to document researchers' information and user interface needs. RESULTS: Biomedical researchers' information needs in data discovery are complex, multidimensional, and shaped by their context, domain knowledge, and technical experience. User needs analyses validate the need for a DDI, while usability evaluations of DataMed show that even though aggregating metadata into a common search engine and applying traditional information retrieval tools are promising first steps, there remain challenges for DataMed due to incomplete metadata and the complexity of data discovery. DISCUSSION: Biomedical data poses distinct problems for search when compared to websites or publications. Making data available is not enough to facilitate biomedical data discovery: new retrieval techniques and user interfaces are necessary for dataset exploration. Consistent, complete, and high-quality metadata are vital to enable this process. CONCLUSION: While available data and researchers' information needs are complex and heterogeneous, a successful DDI must meet those needs and fit into the processes of biomedical researchers. Research directions include formalizing researchers' information needs, standardizing overviews of data to facilitate relevance judgments, implementing user interfaces for concept-based searching, and developing evaluation methods for open-ended discovery systems such as DDIs.

Using Fault Trees to Advance Understanding of Diagnostic Errors.

PROBLEM DEFINITION: Diagnostic errors annually affect at least 5% of adults in the outpatient setting in the United States. Formal analytic techniques are only infrequently used to understand them, in part because of the complexity of diagnostic processes and clinical work flows involved. In this article, diagnostic errors were modeled using fault tree analysis (FTA), a form of root cause analysis that has been successfully used in other high-complexity, high-risk contexts. How factors contributing to diagnostic errors can be systematically modeled by FTA to inform error understanding and error prevention is demonstrated. INITIAL APPROACH: A team of three experts reviewed 10 published cases of diagnostic error and constructed fault trees. The fault trees were modeled according to currently available conceptual frameworks characterizing diagnostic error. The 10 trees were then synthesized into a single fault tree to identify common contributing factors and pathways leading to diagnostic error. KEY INSIGHTS: FTA is a visual, structured, deductive approach that depicts the temporal sequence of events and their interactions in a formal logical hierarchy. The visual FTA enables easier understanding of causative processes and cognitive and system factors, as well as rapid identification of common pathways and interactions in a unified fashion. In addition, it enables calculation of empirical estimates for causative pathways. Thus, fault trees might provide a useful framework for both quantitative and qualitative analysis of diagnostic errors. NEXT STEPS: Future directions include establishing validity and reliability by modeling a wider range of error cases, conducting quantitative evaluations, and undertaking deeper exploration of other FTA capabilities.

Measuring Workload Demand of Informatics Systems with the Clinical Case Demand Index.

Introduction: The increasing use of Health Information Technology (HIT) can add substantially to workload on clinical providers. Current methods for assessing workload do not take into account the nature of clinical cases and the use of HIT tools while solving them. Methods: The Clinical Case Demand Index (CCDI), consisting of a summary score and visual representation, was developed to meet this need. Consistency with current perceived workload measures was evaluated in a Randomized Control Trial of a mobile health system. Results: CCDI is significantly correlated with existing workload measures and inversely related to provider performance. Discussion: CCDI combines subjective and objective characteristics of clinical cases along with cognitive and clinical dimensions. Applications include evaluation of HIT tools, clinician scheduling, medical education. Conclusion: CCDI supports comparative effectiveness research of HIT tools. In addition, CCDI could have numerous applications including training, clinical trials, design of clinical workflows, and others.

Patient knowledge and information-seeking about personalized cancer therapy.

BACKGROUND: Understanding patients' knowledge and prior information-seeking regarding personalized cancer therapy (PCT) may inform future patient information systems, consent for molecular testing and PCT protocols. We evaluated breast cancer patients' knowledge and information-seeking behaviors regarding PCT. METHODS: Newly registered female breast cancer patients (n=100) at a comprehensive cancer center completed a self-administered questionnaire prior to their first clinic visit. RESULTS: Knowledge regarding cancer genetics and PCT was moderate (mean 8.7±3.8 questions correct out of 16). A minority of patients (27%) indicated that they had sought information regarding PCT. Higher education (p=0.009) and income levels (p=0.04) were associated with higher knowledge scores and with seeking PCT information (p=0.04). Knowledge was not associated with willingness to participate in PCT research. CONCLUSION: Educational background and financial status impact patient knowledge as well as information-seeking behavior. For most patients, clinicians are likely to be patients' initial source of information about PCT. Understanding patients' knowledge deficits at presentation may help inform patient education efforts.

Attitudes toward molecular testing for personalized cancer therapy.

BACKGROUND: This study assessed attitudes of breast cancer patients toward molecular testing for personalized therapy and research. METHODS: A questionnaire was given to female breast cancer patients presenting to a cancer center. Associations between demographic and clinical variables and attitudes toward molecular testing were evaluated. RESULTS: Three hundred eight patients were approached, and 100 completed the questionnaire (a 32% response rate). Most participants were willing to undergo molecular testing to assist in the selection of approved drugs (81%) and experimental therapy (59%) if testing was covered by insurance. Most participants were white (71%). Even if testing was financially covered, nonwhite participants were less willing to undergo molecular testing for the selection of approved drugs (54% of nonwhites vs 90% of whites, odds ratio [OR] = 0.13, P = .0004) or experimental drugs (35% vs 68%, OR = 0.26, P = .0072). Most participants (75%) were willing to undergo a biopsy to guide therapy, and 46% were willing to undergo research biopsies. Nonwhite participants were less willing to undergo research biopsies (17% vs 55%, OR = 0.17, P = .0033). Most participants wanted to be informed when research results had implications for treatment (91%), new cancer risk (90%), and other preventable/treatable diseases (87%). CONCLUSIONS: Most patients were willing to undergo molecular testing and minimally invasive procedures to guide approved or experimental therapy. There were significant differences in attitudes toward molecular testing between racial groups; nonwhites were less willing to undergo testing even if the results would guide their own therapy. Novel approaches are needed to prevent disparities in the delivery of genomically informed care and to increase minority participation in biomarker-driven trials. Cancer 2015;121:243-50. © 2014 American Cancer Society.

Attitudes regarding privacy of genomic information in personalized cancer therapy.

OBJECTIVE: To evaluate attitudes regarding privacy of genomic data in a sample of patients with breast cancer. METHODS: Female patients with breast cancer (n=100) completed a questionnaire assessing attitudes regarding concerns about privacy of genomic data. RESULTS: Most patients (83%) indicated that genomic data should be protected. However, only 13% had significant concerns regarding privacy of such data. Patients expressed more concern about insurance discrimination than employment discrimination (43% vs 28%, p<0.001). They expressed less concern about research institutions protecting the security of their molecular data than government agencies or drug companies (20% vs 38% vs 44%; p<0.001). Most did not express concern regarding the association of their genomic data with their name and personal identity (49% concerned), billing and insurance information (44% concerned), or clinical data (27% concerned). Significantly fewer patients were concerned about the association with clinical data than other data types (p<0.001). In the absence of direct benefit, patients were more willing to consent to sharing of deidentified than identified data with researchers not involved in their care (76% vs 60%; p<0.001). Most (85%) patients were willing to consent to DNA banking. DISCUSSION: While patients are opposed to indiscriminate release of genomic data, privacy does not appear to be their primary concern. Furthermore, we did not find any specific predictors of privacy concerns. CONCLUSIONS: Patients generally expressed low levels of concern regarding privacy of genomic data, and many expressed willingness to consent to sharing their genomic data with researchers.

Development of a clinician reputation metric to identify appropriate problem-medication pairs in a crowdsourced knowledge base.

BACKGROUND: Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on problems and medications being encoded in particular terminologies. Crowdsourcing represents one approach to generating robust knowledge bases across a variety of terminologies, but more sophisticated approaches are necessary to improve accuracy and reduce manual data review requirements. OBJECTIVE: We sought to develop and evaluate a clinician reputation metric to facilitate the identification of appropriate problem-medication pairs through crowdsourcing without requiring extensive manual review. APPROACH: We retrieved medications from our clinical data warehouse that had been prescribed and manually linked to one or more problems by clinicians during e-prescribing between June 1, 2010 and May 31, 2011. We identified measures likely to be associated with the percentage of accurate problem-medication links made by clinicians. Using logistic regression, we created a metric for identifying clinicians who had made greater than or equal to 95% appropriate links. We evaluated the accuracy of the approach by comparing links made by those physicians identified as having appropriate links to a previously manually validated subset of problem-medication pairs. RESULTS: Of 867 clinicians who asserted a total of 237,748 problem-medication links during the study period, 125 had a reputation metric that predicted the percentage of appropriate links greater than or equal to 95%. These clinicians asserted a total of 2464 linked problem-medication pairs (983 distinct pairs). Compared to a previously validated set of problem-medication pairs, the reputation metric achieved a specificity of 99.5% and marginally improved the sensitivity of previously described knowledge bases. CONCLUSION: A reputation metric may be a valuable measure for identifying high quality clinician-entered, crowdsourced data.

Data analysis: evaluation of nanoscale contrast agent enhanced CT scan to differentiate between benign and malignant lung cancer in mouse model.

Proposed is a method for statistical analysis for a small sample size, repeated measure experiment with nesting factors. In the original experiment the Student t-test was used for analysis. Using the same data, we modeled the experiment into two groups of mice with benign and malignant primary lung tumors. 4 tumor nodules were selected from each mouse (N= 36). The dependent variables are the volume, diameter, and signal attenuation measured using computed tomography (CT). The measurements are made before injecting the contrast and at 0, 72, and 168 hours after injection. The contrast agent enhances tumor nodule volume and volume differences between benign and malignant tumor nodules measured across time (p < 0.05). The signal attenuation measured across time differentiates between benign and malignant groups (p < 0.05). There is significant correlation between rate of change of volume and diameter of tumor. The advantages of this statistical method are discussed.