Development of a quantitative index system for evaluating the quality of electronic medical records in disease risk intelligent prediction.

OBJECTIVE: This study aimed to develop and validate a quantitative index system for evaluating the data quality of Electronic Medical Records (EMR) in disease risk prediction using Machine Learning (ML). MATERIALS AND METHODS: The index system was developed in four steps: (1) a preliminary index system was outlined based on literature review; (2) we utilized the Delphi method to structure the indicators at all levels; (3) the weights of these indicators were determined using the Analytic Hierarchy Process (AHP) method; and (4) the developed index system was empirically validated using real-world EMR data in a ML-based disease risk prediction task. RESULTS: The synthesis of review findings and the expert consultations led to the formulation of a three-level index system with four first-level, 11 second-level, and 33 third-level indicators. The weights of these indicators were obtained through the AHP method. Results from the empirical analysis illustrated a positive relationship between the scores assigned by the proposed index system and the predictive performances of the datasets. DISCUSSION: The proposed index system for evaluating EMR data quality is grounded in extensive literature analysis and expert consultation. Moreover, the system's high reliability and suitability has been affirmed through empirical validation. CONCLUSION: The novel index system offers a robust framework for assessing the quality and suitability of EMR data in ML-based disease risk predictions. It can serve as a guide in building EMR databases, improving EMR data quality control, and generating reliable real-world evidence.

Sample-Wise and Gene-Wise Comparisons Confirm a Greater Similarity of RNA and Protein Expression Data at the Level of Molecular Pathways and Suggest an Approach for the Data Quality Check in High-Throughput Expression Databases.

Identification of genes and molecular pathways with congruent profiles in the proteomic and transcriptomic datasets may result in the discovery of promising transcriptomic biomarkers that would be more relevant to phenotypic changes. In this study, we conducted comparative analysis of 943 paired RNA and proteomic profiles obtained for the same samples of seven human cancer types from The Cancer Genome Atlas (TCGA) and NCI Clinical Proteomic Tumor Analysis Consortium (CPTAC) [two major open human cancer proteomic and transcriptomic databases] that included 15,112 protein-coding genes and 1611 molecular pathways. Overall, our findings demonstrated statistically significant improvement of the congruence between RNA and proteomic profiles when performing analysis at the level of molecular pathways rather than at the level of individual gene products. Transition to the molecular pathway level of data analysis increased the correlation to 0.19-0.57 (Pearson) and 0.14-057 (Spearman), or 2-3-fold for some cancer types. Evaluating the gain of the correlation upon transition to the data analysis the pathway level can be used to refine the omics data by identifying outliers that can be excluded from the comparison of RNA and proteomic profiles. We suggest using sample- and gene-wise correlations for individual genes and molecular pathways as a measure of quality of RNA/protein paired molecular data. We also provide a database of human genes, molecular pathways, and samples related to the correlation between RNA and protein products to facilitate an exploration of new cancer transcriptomic biomarkers and molecular mechanisms at different levels of human gene expression.

Engaging the Next Generation in Communication to Address Information Quality.

Information disorder can have adverse consequences on health. While there has been growing attention to health information quality among the general population, there has been less focus on the young adult age group and how their insights and ideas can help to explore the effects and potential interventions to address information quality. Since certain information consumption habits and effects vary among young people, their perspective can provide valuable insights for tackling the increasing issue of misinformation. This Perspective examines past youth involvement efforts to suggest ways to incorporate the youth perspective into improving the quality of health information, particularly through engagement strategies aimed at combating misinformation traits. We then propose a set of five recommendations to advance research to address information disorder, researchers can consider the following steps to engage youth.

Smoking data quality of primary care practices in comparison with smoking data from the New Zealand Maori and Pacific abdominal aortic aneurysm screening programme: an observational study.

BACKGROUND: Quality smoking data is crucial for assessing smoking-related health risk and eligibility for interventions related to that risk. Smoking information collected in primary care practices (PCPs) is a major data source; however, little is known about the PCP smoking data quality. This project compared PCP smoking data to that collected in the Maori and Pacific Abdominal Aortic Aneurysm (AAA) screening programme. METHODS: A two stage review was conducted. In Stage 1, data quality was assessed by comparing the PCP smoking data recorded close to AAA screening episodes with the data collected from participants at the AAA screening session. Inter-rater reliability was analysed using Cohen's kappa scores. In Stage 2, an audit of longitudinal smoking status was conducted, of a subset of participants potentially misclassified in Stage 1. Data were compared in three groups: current smoker (smoke at least monthly), ex-smoker (stopped > 1 month ago) and never smoker (smoked < 100 cigarettes in lifetime). RESULTS: Of the 1841 people who underwent AAA screening, 1716 (93%) had PCP smoking information. Stage 1 PCP smoking data showed 82% concordance with the AAA data (adjusted kappa 0.76). Fewer current or ex-smokers were recorded in PCP data. In the Stage 2 analysis of discordant and missing data (N = 313), 212 were enrolled in the 29 participating PCPs, and of these 13% were deceased and 41% had changed PCP. Of the 93 participants still enrolled in the participating PCPs, smoking status had been updated for 43%. Data on quantity, duration, or quit date of smoking were largely missing in PCP records. The AAA data of ex-smokers who were classified as never smokers in the Stage 2 PCP data (N = 27) showed a median smoking cessation duration of 32 years (range 0-50 years), with 85% (N = 23) having quit more than 15 years ago. CONCLUSIONS: PCP smoking data quality compared with the AAA data is consistent with international findings. PCP data captured fewer current and ex-smokers, suggesting ongoing improvement is important. Intervention programmes based on smoking status should consider complementary mechanisms to ensure eligible individuals are not missed from programme invitation.

Undercounting diagnoses in Australian general practice: a data quality study with implications for population health reporting.

BACKGROUND: Diagnosis can often be recorded in electronic medical records (EMRs) as free-text or using a term with a diagnosis code. Researchers, governments, and agencies, including organisations that deliver incentivised primary care quality improvement programs, frequently utilise coded data only and often ignore free-text entries. Diagnosis data are reported for population healthcare planning including resource allocation for patient care. This study sought to determine if diagnosis counts based on coded diagnosis data only, led to under-reporting of disease prevalence and if so, to what extent for six common or important chronic diseases. METHODS: This cross-sectional data quality study used de-identified EMR data from 84 general practices in Victoria, Australia. Data represented 456,125 patients who attended one of the general practices three or more times in two years between January 2021 and December 2022. We reviewed the percentage and proportional difference between patient counts of coded diagnosis entries alone and patient counts of clinically validated free-text entries for asthma, chronic kidney disease, chronic obstructive pulmonary disease, dementia, type 1 diabetes and type 2 diabetes. RESULTS: Undercounts were evident in all six diagnoses when using coded diagnoses alone (2.57-36.72% undercount), of these, five were statistically significant. Overall, 26.4% of all patient diagnoses had not been coded. There was high variation between practices in recording of coded diagnoses, but coding for type 2 diabetes was well captured by most practices. CONCLUSION: In Australia clinical decision support and the reporting of aggregated patient diagnosis data to government that relies on coded diagnoses can lead to significant underreporting of diagnoses compared to counts that also incorporate clinically validated free-text diagnoses. Diagnosis underreporting can impact on population health, healthcare planning, resource allocation, and patient care. We propose the use of phenotypes derived from clinically validated text entries to enhance the accuracy of diagnosis and disease reporting. There are existing technologies and collaborations from which to build trusted mechanisms to provide greater reliability of general practice EMR data used for secondary purposes.

Validation of Multi-State EHR-Based Network for Disease Surveillance (MENDS) Data and Implications for Improving Data Quality and Representativeness.

Introduction: Surveillance modernization efforts emphasize the potential use of electronic health record (EHR) data to inform public health surveillance and prevention. However, EHR data streams vary widely in their completeness, accuracy, and representativeness. Methods: We developed a validation process for the Multi-State EHR-Based Network for Disease Surveillance (MENDS) pilot project to identify and resolve data quality issues that could affect chronic disease prevalence estimates. We examined MENDS validation processes from December 2020 through August 2023 across 5 data-contributing organizations and outlined steps to resolve data quality issues. Results: We identified gaps in the EHR databases of data contributors and in the processes to extract, map, integrate, and analyze their EHR data. Examples of source-data problems included missing data on race and ethnicity and zip codes. Examples of data processing problems included duplicate or missing patient records, lower-than-expected volumes of data, use of multiple fields for a single data type, and implausible values. Conclusion: Validation protocols identified critical errors in both EHR source data and in the processes used to transform these data for analysis. Our experience highlights the value and importance of data validation to improve data quality and the accuracy of surveillance estimates that use EHR data. The validation process and lessons learned can be applied broadly to other EHR-based surveillance efforts.

Accuracy, potential, and limitations of probabilistic record linkage in identifying deaths by gender identity and sexual orientation in the state of Rio De Janeiro, Brazil.

BACKGROUND: Globally, the counting of deaths based on gender identity and sexual orientation has been a challenge for health systems. In most cases, non-governmental organizations have dedicated themselves to this work. Despite these efforts in generating information, the scarcity of official data presents significant limitations in policy formulation and actions guided by population needs. Therefore, this manuscript aims to evaluate the accuracy, potential, and limits of probabilistic data relationships to yield information on deaths according to gender identity and sexual orientation in the State of Rio de Janeiro. METHODS: This study evaluated the accuracy of the probabilistic record linkage to obtain information on deaths according to gender and sexual orientation. Data from two information systems were used from June 15, 2015 to December 31, 2020. We constructed nine probabilistic data relationship strategies and identified the performance and cutoff points of the best strategy. RESULTS: The best data blocking strategy was established through logical blocks with the first and last names, birthdate, and mother's name in the pairing strategy. With a population base of 80,178 records, 1556 deaths were retrieved. With an area under the curve of 0.979, this strategy presented 93.26% accuracy, 98.46% sensitivity, and 90.04% specificity for the cutoff point ≥ 17.9 of the data relationship score. The adoption of the cutoff point optimized the manual review phase, identifying 2259 (90.04%) of the 2509 false pairs and identifying 1532 (98.46%) of the 1556 true pairs. CONCLUSION: With the identification of possible strategies for determining probabilistic data relationships, the retrieval of information on mortality according to sexual and gender markers has become feasible. Based on information from the daily routine of health services, the formulation of public policies that consider the LGBTQ + population more closely reflects the reality experienced by these population groups.

Precious Information: Getting Interpretable, Actionable Health Communications Data.

Doing high-quality health communications requires high-quality data, both to describe information environments and diets and as outcomes for program evaluations that seek to change them and the behaviors they foster. When we understand where people gather information, what they learn there, and how it encourages them to act, we can better diagnose where to focus energy and resources-for example, whether to amplify high-quality sources, address low-quality information, or fill information voids with actionable ideas. Too often, however, researchers collect data in ways that conflate the sources of information, the channels through which information is communicated, and the content of that information. This thwarts opportunities to identify specific problems and opportunities. We call for conceptual precision to make good use of respondents' time, to be good stewards of funders' money, and to effectively address the large challenges we face in encouraging protective, pro-social health behaviors.

Improving Clinical Registry Data Quality via Linkage With Survival Data From State-Based Population Registries.

PURPOSE: Real-world data (RWD) collected on patients treated as part of routine clinical care form the basis of cancer clinical registries. Capturing accurate death data can be challenging, with inaccurate survival data potentially compromising the integrity of registry-based research. Here, we explore the utility of data linkage (DL) to state-based registries to enhance the capture of survival outcomes. METHODS: We identified consecutive adult patients with brain tumors treated in the state of Victoria from the Brain Tumour Registry Australia: Innovation and Translation (BRAIN) database, who had no recorded date of death and no follow-up within the last 6 months. Full name and date of birth were used to match patients in the BRAIN registry with those in the Victorian Births, Deaths and Marriages (BDM) registry. Overall survival (OS) outcomes were compared pre- and post-DL. RESULTS: Of the 7,346 clinical registry patients, 5,462 (74%) had no date of death and no follow-up recorded within the last 6 months. Of the 5,462 patients, 1,588 (29%) were matched with a date of death in BDM. Factors associated with an increased number of matches were poor prognosis tumors, older age, and social disadvantage. OS was significantly overestimated pre-DL compared with post-DL for the entire cohort (pre- v post-DL: hazard ratio, 1.43; P < .001; median, 29.9 months v 16.7 months) and for most individual tumor types. This finding was present independent of the tumor prognosis. CONCLUSION: As revealed by linkage with BDM, a high proportion of patients in a brain cancer clinical registry had missing death data, contributed to by informative censoring, inflating OS calculations. DL to pertinent registries on an ongoing basis should be considered to ensure accurate reporting of survival data and interpretation of RWD outcomes.

Multi-source heterogeneous blockchain data quality assessment model for enterprise business activities.

Blockchain-based applications are becoming more and more widespread in business operations. In view of the shortcomings of existing enterprise blockchain evaluation methods, this paper proposes a multi-source heterogeneous blockchain data quality evaluation model for enterprise business activities, so as to achieve efficient evaluation of business activity information consistency, credibility and value. This paper proposes a multi-source heterogeneous blockchain data quality assessment method for enterprise business activities, aiming at the problems that most of the data in enterprise business activities come from different data sources, information representation is inconsistent, information ambiguity between the same block chain is serious, and it is difficult to evaluate the consistency, credibility and value of information. The method firstly proposes an entity information representation method based on the Representation learning for fusing entity category information (CEKGRL) model, which introduces the triad structure of related entities in blockchain, then associates them with enterprise business activity categories, and carries out similarity calculation through contextual information to achieve blockchain information consistency assessment. After that, a trustworthiness characterization method is proposed based on information sources, information comments, and information contents, to obtain the trustworthiness assessment of the business. Finally, based on the information trustworthiness characterization, a value assessment method is introduced to assess the total value of business activity information in the blockchain, and a blockchain quality assessment model is constructed. The experimental results show that the proposed model has great advantages over existing methods in assessing inter-block consistency, intra-block activity information trustworthiness and the value of blockchain.

Understanding the value of curation: A survey of US data repository curation practices and perceptions.

Data curators play an important role in assessing data quality and take actions that may ultimately lead to better, more valuable data products. This study explores the curation practices of data curators working within US-based data repositories. We performed a survey in January 2021 to benchmark the levels of curation performed by repositories and assess the perceived value and impact of curation on the data sharing process. Our analysis included 95 responses from 59 unique data repositories. Respondents primarily were professionals working within repositories and examined curation performed within a repository setting. A majority 72.6% of respondents reported that "data-level" curation was performed by their repository and around half reported their repository took steps to ensure interoperability and reproducibility of their repository's datasets. Curation actions most frequently reported include checking for duplicate files, reviewing documentation, reviewing metadata, minting persistent identifiers, and checking for corrupt/broken files. The most "value-add" curation action across generalist, institutional, and disciplinary repository respondents was related to reviewing and enhancing documentation. Respondents reported high perceived impact of curation by their repositories on specific data sharing outcomes including usability, findability, understandability, and accessibility of deposited datasets; respondents associated with disciplinary repositories tended to perceive higher impact on most outcomes. Most survey participants strongly agreed that data curation by the repository adds value to the data sharing process and that it outweighs the effort and cost. We found some differences between institutional and disciplinary repositories, both in the reported frequency of specific curation actions as well as the perceived impact of data curation. Interestingly, we also found variation in the perceptions of those working within the same repository regarding the level and frequency of curation actions performed, which exemplifies the complexity of a repository curation work. Our results suggest data curation may be better understood in terms of specific curation actions and outcomes than broadly defined curation levels and that more research is needed to understand the resource implications of performing these activities. We share these results to provide a more nuanced view of curation, and how curation impacts the broader data lifecycle and data sharing behaviors.

Effectiveness of remote risk-based monitoring and potential benefits for combination with direct data capture.

BACKGROUND: In recent years, alternative monitoring approaches, such as risk-based and remote monitoring techniques, have been recommended instead of traditional on-site monitoring to achieve more efficient monitoring. Remote risk-based monitoring (R2BM) is a monitoring technique that combines risk-based and remote monitoring and focuses on the detection of critical data and process errors. Direct data capture (DDC), which directly collects electronic source data, can facilitate R2BM by minimizing the extent of source documents that must be reviewed and reducing the additional workload on R2BM. In this study, we evaluated the effectiveness of R2BM and the synergistic effect of combining R2BM with DDC. METHODS: R2BM was prospectively conducted with eight participants in a randomized clinical trial using a remote monitoring system that uploaded photographs of source documents to a cloud location. Critical data and processes were verified by R2BM, and later, all were confirmed by on-site monitoring to evaluate the ability of R2BM to detect critical data and process errors and the workload of uploading photographs for clinical trial staff. In addition, the reduction of the number of uploaded photographs was evaluated by assuming that the DDC was introduced for data collection. RESULTS: Of the 4645 data points, 20.9% (n = 973, 95% confidence interval = 19.8-22.2) were identified as critical. All critical data errors corresponding to 5.4% (n = 53/973, 95% confidence interval = 4.1-7.1) of the critical data and critical process errors were detectable by R2BM. The mean number of uploaded photographs and the mean time to upload them per visit per participant were 34.4 ± 11.9 and 26.5 ± 11.8 min (mean ± standard deviation), respectively. When assuming that DDC was introduced for data collection, 45.0% (95% confidence interval = 42.2-47.9) of uploaded photographs for R2BM were reduced. CONCLUSIONS: R2BM can detect 100% of the critical data and process errors without on-site monitoring. Combining R2BM with DDC reduces the workload of R2BM and further improves its efficiency.

Data integrity of 14 randomised controlled trials.

BACKGROUND: A review of the literature on iron treatments for iron-deficient anaemia in pregnancy indicated duplication of baseline and outcome tables in two separate randomised controlled trials (RCTs) that share only a single author. AIM: To assess the integrity of randomised clinical trials from Dr A.M. Darwish, Assiut University, Egypt. DESIGN: Assessment of Research Integrity. METHODS: We tabulated the characteristics of studies, compared baseline and outcome tables between articles and looked for implausible findings. We used the distribution of baseline p-values to assess whether the summary statistics of baseline characteristics were consistent with properly conducted randomisation. RESULTS: We identified 14 RCTs (1,405 participants) published between October 2004 and September 2019. Two pairs of studies showed considerable similarities in baseline characteristics, while another pair of studies was plagiarized. The analysis of baseline p-values indicated a low probability that all the studies featured randomised treatment allocation. CONCLUSION: Our analysis of the RCTs of Dr Darwish suggests possible integrity problems. We recommend a critical investigation of the studies that have not been retracted. Until that has been completed, these studies should not be used to inform clinical practice.

Developing evaluation measures for health departments sharing and linking HIV and STI surveillance data.

Enhancing data sharing, quality, and use across siloed HIV and STI programs is critical for national and global initiatives to reduce new HIV infections and improve the health of people with HIV. As part of the Enhancing Linkage of STI and HIV Surveillance Data in the Ryan White HIV/AIDS Program initiative, four health departments (HDs) in the U.S. received technical assistance to better share and link their HIV and STI surveillance data. The process used to develop evaluation measures assessing implementation and outcomes of linking HIV and STI data systems involved six steps: 1) measure selection and development, 2) review and refinement, 3) testing, 4) implementation and data collection, 5) data quality review and feedback, and 6) dissemination. Findings from pilot testing warranted slight adaptations, including starting with a core set of measures and progressively scaling up. Early findings showed improvements in data quality over time. Lessons learned included identifying and engaging key stakeholders early; developing resources to assist HDs; and considering measure development as iterative processes requiring periodic review and reassessment to ensure continued utility. These findings can guide programs and evaluations, especially those linking data across multiple systems, in developing measures to track implementation and outcomes over time.

Improving Accuracy of Administrative Data for Perforated Appendicitis Classification.

INTRODUCTION: Reliance on International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes may misclassify perforated appendicitis with resultant research, fiscal, and public health implications. We aimed to improve the accuracy of administrative data for perforated appendicitis classification relying on ICD-10-CM codes from 2015 to 2018. METHODS: We conducted a retrospective study of randomly sampled patients aged ≤18 years diagnosed with acute appendicitis from eight children's hospitals. Patients were identified using the Pediatric Health Information System, and true perforation status was determined by medical record review. We developed two algorithms by leveraging Pediatric Health Information System data elements and data mining (DM) approaches. The two developed algorithm performance was compared against algorithms that exclusively relied on ICD-10-CM codes using area under the curve and other measures. RESULTS: Of 1051 clinically validated encounters that were included, 383 (36.4%) patients were identified to have perforated appendicitis. The two algorithms developed using DM approaches primarily leveraged ICD-10-CM codes and length of stay. DM-developed algorithms had a significantly higher accuracy than algorithms relying exclusively on ICD-10-CM (P value < 0.01): sensitivity and specificity for DM-developed algorithms were 0.86-0.88 and 0.95-0.97, respectively, which were overall higher than algorithms that relied on only ICD-10-CM. CONCLUSIONS: This study provides an algorithm that can improve the accuracy of perforated appendicitis classification using commonly available elements in administrative data. We recommend that this algorithm is used in future appendicitis classification to ensure valid reporting, hospital-level benchmarking, and fiscal or public health assessments.

Optimizing Spectronaut Search Parameters to Improve Data Quality with Minimal Proteome Coverage Reductions in DIA Analyses of Heterogeneous Samples.

Data-independent acquisition has seen breakthroughs that enable comprehensive proteome profiling using short gradients. As the proteome coverage continues to increase, the quality of the data generated becomes much more relevant. Using Spectronaut, we show that the default search parameters can be easily optimized to minimize the occurrence of false positives across different samples. Using an immunological infection model system to demonstrate the impact of adjusting search settings, we analyzed Mus musculus macrophages and compared their proteome to macrophages spiked withCandida albicans. This experimental system enabled the identification of "false positives" as Candida albicans peptides and proteins should not be present in the Mus musculus-only samples. We show that adjusting the search parameters reduced "false positive" identifications by 89% at the peptide and protein level, thereby considerably increasing the quality of the data. We also show that these optimized parameters incurred a moderate cost, only reducing the overall number of "true positive" identifications across each biological replicate by <6.7% at both the peptide and protein level. We believe the value of our updated search parameters extends beyond a two-organism analysis and would be of great value to any DIA experiment analyzing heterogeneous populations of cell types or tissues.

Improving Eye-Tracking Data Quality: A Framework for Reproducible Evaluation of Detection Algorithms.

High-quality eye-tracking data are crucial in behavioral sciences and medicine. Even with a solid understanding of the literature, selecting the most suitable algorithm for a specific research project poses a challenge. Empowering applied researchers to choose the best-fitting detector for their research needs is the primary contribution of this paper. We developed a framework to systematically assess and compare the effectiveness of 13 state-of-the-art algorithms through a unified application interface. Hence, we more than double the number of algorithms that are currently usable within a single software package and allow researchers to identify the best-suited algorithm for a given scientific setup. Our framework validation on retrospective data underscores its suitability for algorithm selection. Through a detailed and reproducible step-by-step workflow, we hope to contribute towards significantly improved data quality in scientific experiments.