Assessing the Effect of Electronic Health Record Data Quality on Identifying Patients with Type 2 Diabetes.

BACKGROUND: Increasing and substantial reliance on Electronic health records (EHR) and data types (i.e., diagnosis (Dx), medication (Rx), laboratory (Lx)) demands assessment of its data quality (DQ) as a fundamental approach; especially since there is need to identify appropriate denominator population with chronic conditions, such as Type-2 Diabetes (T2D), using commonly available computable phenotype definitions (phenotype). OBJECTIVE: To bridge this gap, our study aims to assess how issues of EHR DQ, and variations and robustness (or lack thereof) in phenotypes may have potential impact in identifying denominator population. METHODS: Approximately 208k patients with T2D were included in our study using retrospective EHR data of Johns Hopkins Medical Institution (JHMI) during 2017-2019. Our assessment included 4 published phenotypes, and 1 definition from a panel of experts at Hopkins. We conducted descriptive analyses of demographics (i.e., age, sex, race, ethnicity), healthcare utilization (inpatient and emergency room visits), and average Charlson Comorbidity score of each phenotype. We then used different methods to induce/simulate DQ issues of completeness, accuracy and timeliness separately across each phenotype. For induced data incompleteness, our model randomly dropped Dx, Rx, and Lx codes independently at increments of 10%; for induced data inaccuracy, our model randomly replaced a Dx or Rx code with another code of the same data type and induced 2% incremental change from -100% to +10% in Lx result values; and lastly, for timeliness, data was modeled for induced incremental shift of date records by 30 days up to a year. RESULTS: Less than a quarter (23%) of population overlapped across all phenotypes using EHR. The population identified by each phenotype varied across all combination of data types. Induced incompleteness identified fewer patients with each increment, for e.g., at 100% diagnostic incompleteness, Chronic Conditions Data Warehouse (CCW) phenotype identified zero patients as its phenotypic characteristics included only Dx codes. Induced inaccuracy and timeliness similarly demonstrated variations in performance of each phenotype and therefore, resulting in fewer patients being identified with each incremental change. CONCLUSIONS: We utilized EHR data with Dx, Rx, and Lx data types from a large tertiary hospital system to understand the T2D phenotypic differences and performance. We learned how issues of DQ, using induced DQ methods, may impact identification of the denominator populations upon which clinical (e.g., clinical research and trials, population health evaluations) and financial/operational decisions are made. The novel results from our study may inform in shaping a common T2D computable phenotype definition that can be applicable to clinical informatics, managing chronic conditions, and additional healthcare industry-wide efforts.

Computable phenotype for diagnostic error: developing the data schema for application of symptom-disease pair analysis of diagnostic error (SPADE).

OBJECTIVES: Diagnostic errors are the leading cause of preventable harm in clinical practice. Implementable tools to quantify and target this problem are needed. To address this gap, we aimed to generalize the Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) framework by developing its computable phenotype and then demonstrated how that schema could be applied in multiple clinical contexts. METHODS: We created an information model for the SPADE processes, then mapped data fields from electronic health records (EHR) and claims data in use to that model to create the SPADE information model (intention) and the SPADE computable phenotype (extension). Later we validated the computable phenotype and tested it in four case studies in three different health systems to demonstrate its utility. RESULTS: We mapped and tested the SPADE computable phenotype in three different sites using four different case studies. We showed that data fields to compute an SPADE base measure are fully available in the EHR Data Warehouse for extraction and can operationalize the SPADE framework from provider and/or insurer perspective, and they could be implemented on numerous health systems for future work in monitor misdiagnosis-related harms. CONCLUSIONS: Data for the SPADE base measure is readily available in EHR and administrative claims. The method of data extraction is potentially universally applicable, and the data extracted is conveniently available within a network system. Further study is needed to validate the computable phenotype across different settings with different data infrastructures.

Racial Disparities in Diabetes Care and Outcomes for Patients with Visual Impairment: A Descriptive Analysis of the TriNetX Research Network.

Background: This research delves into the confluence of racial disparities and health inequities among individuals with disabilities, with a focus on those contending with both diabetes and visual impairment. Methods: Utilizing data from the TriNetX Research Network, which includes electronic medical records of roughly 115 million patients from 83 anonymous healthcare organizations, this study employs a directed acyclic graph (DAG) to pinpoint confounders and augment interpretation. We identified patients with visual impairments using ICD-10 codes, deliberately excluding diabetes-related ophthalmology complications. Our approach involved multiple race-stratified analyses, comparing co-morbidities like chronic pulmonary disease in visually impaired patients against their counterparts. We assessed healthcare access disparities by examining the frequency of annual visits, instances of two or more A1c measurements, and glomerular filtration rate (GFR) measurements. Additionally, we evaluated diabetes outcomes by comparing the risk ratio of uncontrolled diabetes (A1c > 9.0) and chronic kidney disease in patients with and without visual impairments. Results: The incidence of diabetes was substantially higher (nearly double) in individuals with visual impairments across White, Asian, and African American populations. Higher rates of chronic kidney disease were observed in visually impaired individuals, with a risk ratio of 1.79 for African American, 2.27 for White, and non-significant for the Asian group. A statistically significant difference in the risk ratio for uncontrolled diabetes was found only in the White cohort (0.843). White individuals without visual impairments were more likely to receive two A1c tests, a trend not significant in other racial groups. African Americans with visual impairments had a higher rate of glomerular filtration rate testing. However, White individuals with visual impairments were less likely to undergo GFR testing, indicating a disparity in kidney health monitoring. This pattern of disparity was not observed in the Asian cohort. Conclusions: This study uncovers pronounced disparities in diabetes incidence and management among individuals with visual impairments, particularly among White, Asian, and African American groups. Our DAG analysis illuminates the intricate interplay between SDoH, healthcare access, and frequency of crucial diabetes monitoring practices, highlighting visual impairment as both a medical and social issue.

Autonomous artificial intelligence increases screening and follow-up for diabetic retinopathy in youth: the ACCESS randomized control trial.

Diabetic retinopathy can be prevented with screening and early detection. We hypothesized that autonomous artificial intelligence (AI) diabetic eye exams at the point-of-care would increase diabetic eye exam completion rates in a racially and ethnically diverse youth population. AI for Children's diabetiC Eye ExamS (NCT05131451) is a parallel randomized controlled trial that randomized youth (ages 8-21 years) with type 1 and type 2 diabetes to intervention (autonomous artificial intelligence diabetic eye exam at the point of care), or control (scripted eye care provider referral and education) in an academic pediatric diabetes center. The primary outcome was diabetic eye exam completion rate within 6 months. The secondary outcome was the proportion of participants who completed follow-through with an eye care provider if deemed appropriate. Diabetic eye exam completion rate was significantly higher (100%, 95%CI: 95.5%, 100%) in the intervention group (n = 81) than the control group (n = 83) (22%, 95%CI: 14.2%, 32.4%)(p < 0.001). In the intervention arm, 25/81 participants had an abnormal result, of whom 64% (16/25) completed follow-through with an eye care provider, compared to 22% in the control arm (p < 0.001). Autonomous AI increases diabetic eye exam completion rates in youth with diabetes.

Advancing Toward a Common Data Model in Ophthalmology: Gap Analysis of General Eye Examination Concepts to Standard Observational Medical Outcomes Partnership (OMOP) Concepts.

Purpose: Evaluate the degree of concept coverage of the general eye examination in one widely used electronic health record (EHR) system using the Observational Health Data Sciences and Informatics Observational Medical Outcomes Partnership (OMOP) common data model (CDM). Design: Study of data elements. Participants: Not applicable. Methods: Data elements (field names and predefined entry values) from the general eye examination in the Epic foundation system were mapped to OMOP concepts and analyzed. Each mapping was given a Health Level 7 equivalence designation-equal when the OMOP concept had the same meaning as the source EHR concept, wider when it was missing information, narrower when it was overly specific, and unmatched when there was no match. Initial mappings were reviewed by 2 graders. Intergrader agreement for equivalence designation was calculated using Cohen's kappa. Agreement on the mapped OMOP concept was calculated as a percentage of total mappable concepts. Discrepancies were discussed and a final consensus created. Quantitative analysis was performed on wider and unmatched concepts. Main Outcome Measures: Gaps in OMOP concept coverage of EHR elements and intergrader agreement of mapped OMOP concepts. Results: A total of 698 data elements (210 fields, 488 values) from the EHR were analyzed. The intergrader kappa on the equivalence designation was 0.88 (standard error 0.03, P < 0.001). There was a 96% agreement on the mapped OMOP concept. In the final consensus mapping, 25% (1% fields, 31% values) of the EHR to OMOP concept mappings were considered equal, 50% (27% fields, 60% values) wider, 4% (8% fields, 2% values) narrower, and 21% (52% fields, 8% values) unmatched. Of the wider mapped elements, 46% were missing the laterality specification, 24% had other missing attributes, and 30% had both issues. Wider and unmatched EHR elements could be found in all areas of the general eye examination. Conclusions: Most data elements in the general eye examination could not be represented precisely using the OMOP CDM. Our work suggests multiple ways to improve the incorporation of important ophthalmology concepts in OMOP, including adding laterality to existing concepts. There exists a strong need to improve the coverage of ophthalmic concepts in source vocabularies so that the OMOP CDM can better accommodate vision research. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Artificial intelligence-based clinical decision support for liver transplant evaluation and considerations about fairness: A qualitative study.

BACKGROUND: The use of large-scale data and artificial intelligence (AI) to support complex transplantation decisions is in its infancy. Transplant candidate decision-making, which relies heavily on subjective assessment (ie, high variability), provides a ripe opportunity for AI-based clinical decision support (CDS). However, AI-CDS for transplant applications must consider important concerns regarding fairness (ie, health equity). The objective of this study was to use human-centered design methods to elicit providers' perceptions of AI-CDS for liver transplant listing decisions. METHODS: In this multicenter qualitative study conducted from December 2020 to July 2021, we performed semistructured interviews with 53 multidisciplinary liver transplant providers from 2 transplant centers. We used inductive coding and constant comparison analysis of interview data. RESULTS: Analysis yielded 6 themes important for the design of fair AI-CDS for liver transplant listing decisions: (1) transparency in the creators behind the AI-CDS and their motivations; (2) understanding how the AI-CDS uses data to support recommendations (ie, interpretability); (3) acknowledgment that AI-CDS could mitigate emotions and biases; (4) AI-CDS as a member of the transplant team, not a replacement; (5) identifying patient resource needs; and (6) including the patient's role in the AI-CDS. CONCLUSIONS: Overall, providers interviewed were cautiously optimistic about the potential for AI-CDS to improve clinical and equitable outcomes for patients. These findings can guide multidisciplinary developers in the design and implementation of AI-CDS that deliberately considers health equity.

Associations between sleep health and obesity and weight change in adults: The Daily24 Multisite Cohort Study.

OBJECTIVES: To examine cross-sectional and longitudinal associations of individual sleep domains and multidimensional sleep health with current overweight or obesity and 5-year weight change in adults. METHODS: We estimated sleep regularity, quality, timing, onset latency, sleep interruptions, duration, and napping using validated questionnaires. We calculated multidimensional sleep health using a composite score (total number of "good" sleep health indicators) and sleep phenotypes derived from latent class analysis. Logistic regression was used to examine associations between sleep and overweight or obesity. Multinomial regression was used to examine associations between sleep and weight change (gain, loss, or maintenance) over a median of 1.66 years. RESULTS: The sample included 1016 participants with a median age of 52 (IQR = 37-65), who primarily identified as female (78%), White (79%), and college-educated (74%). We identified 3 phenotypes: good, moderate, and poor sleep. More regularity of sleep, sleep quality, and shorter sleep onset latency were associated with 37%, 38%, and 45% lower odds of overweight or obesity, respectively. The addition of each good sleep health dimension was associated with 16% lower adjusted odds of having overweight or obesity. The adjusted odds of overweight or obesity were similar between sleep phenotypes. Sleep, individual or multidimensional sleep health, was not associated with weight change. CONCLUSIONS: Multidimensional sleep health showed cross-sectional, but not longitudinal, associations with overweight or obesity. Future research should advance our understanding of how to assess multidimensional sleep health to understand the relationship between all aspects of sleep health and weight over time.

Strengths, weaknesses, opportunities, and threats for the nation's public health information systems infrastructure: synthesis of discussions from the 2022 ACMI Symposium.

OBJECTIVE: The annual American College of Medical Informatics (ACMI) symposium focused discussion on the national public health information systems (PHIS) infrastructure to support public health goals. The objective of this article is to present the strengths, weaknesses, threats, and opportunities (SWOT) identified by public health and informatics leaders in attendance. MATERIALS AND METHODS: The Symposium provided a venue for experts in biomedical informatics and public health to brainstorm, identify, and discuss top PHIS challenges. Two conceptual frameworks, SWOT and the Informatics Stack, guided discussion and were used to organize factors and themes identified through a qualitative approach. RESULTS: A total of 57 unique factors related to the current PHIS were identified, including 9 strengths, 22 weaknesses, 14 opportunities, and 14 threats, which were consolidated into 22 themes according to the Stack. Most themes (68%) clustered at the top of the Stack. Three overarching opportunities were especially prominent: (1) addressing the needs for sustainable funding, (2) leveraging existing infrastructure and processes for information exchange and system development that meets public health goals, and (3) preparing the public health workforce to benefit from available resources. DISCUSSION: The PHIS is unarguably overdue for a strategically designed, technology-enabled, information infrastructure for delivering day-to-day essential public health services and to respond effectively to public health emergencies. CONCLUSION: Most of the themes identified concerned context, people, and processes rather than technical elements. We recommend that public health leadership consider the possible actions and leverage informatics expertise as we collectively prepare for the future.

Trends and opportunities in computable clinical phenotyping: A scoping review.

Identifying patient cohorts meeting the criteria of specific phenotypes is essential in biomedicine and particularly timely in precision medicine. Many research groups deliver pipelines that automatically retrieve and analyze data elements from one or more sources to automate this task and deliver high-performing computable phenotypes. We applied a systematic approach based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to conduct a thorough scoping review on computable clinical phenotyping. Five databases were searched using a query that combined the concepts of automation, clinical context, and phenotyping. Subsequently, four reviewers screened 7960 records (after removing over 4000 duplicates) and selected 139 that satisfied the inclusion criteria. This dataset was analyzed to extract information on target use cases, data-related topics, phenotyping methodologies, evaluation strategies, and portability of developed solutions. Most studies supported patient cohort selection without discussing the application to specific use cases, such as precision medicine. Electronic Health Records were the primary source in 87.1 % (N = 121) of all studies, and International Classification of Diseases codes were heavily used in 55.4 % (N = 77) of all studies, however, only 25.9 % (N = 36) of the records described compliance with a common data model. In terms of the presented methods, traditional Machine Learning (ML) was the dominant method, often combined with natural language processing and other approaches, while external validation and portability of computable phenotypes were pursued in many cases. These findings revealed that defining target use cases precisely, moving away from sole ML strategies, and evaluating the proposed solutions in the real setting are essential opportunities for future work. There is also momentum and an emerging need for computable phenotyping to support clinical and epidemiological research and precision medicine.

Effectiveness of artificial intelligence screening in preventing vision loss from diabetes: a policy model.

The effectiveness of using artificial intelligence (AI) systems to perform diabetic retinal exams ('screening') on preventing vision loss is not known. We designed the Care Process for Preventing Vision Loss from Diabetes (CAREVL), as a Markov model to compare the effectiveness of point-of-care autonomous AI-based screening with in-office clinical exam by an eye care provider (ECP), on preventing vision loss among patients with diabetes. The estimated incidence of vision loss at 5 years was 1535 per 100,000 in the AI-screened group compared to 1625 per 100,000 in the ECP group, leading to a modelled risk difference of 90 per 100,000. The base-case CAREVL model estimated that an autonomous AI-based screening strategy would result in 27,000 fewer Americans with vision loss at 5 years compared with ECP. Vision loss at 5 years remained lower in the AI-screened group compared to the ECP group, in a wide range of parameters including optimistic estimates biased toward ECP. Real-world modifiable factors associated with processes of care could further increase its effectiveness. Of these factors, increased adherence with treatment was estimated to have the greatest impact.

Enhancing the nation's public health information infrastructure: a report from the ACMI symposium.

The COVID-19 pandemic exposed multiple weaknesses in the nation's public health system. Therefore, the American College of Medical Informatics selected "Rebuilding the Nation's Public Health Informatics Infrastructure" as the theme for its annual symposium. Experts in biomedical informatics and public health discussed strategies to strengthen the US public health information infrastructure through policy, education, research, and development. This article summarizes policy recommendations for the biomedical informatics community postpandemic. First, the nation must perceive the health data infrastructure to be a matter of national security. The nation must further invest significantly more in its health data infrastructure. Investments should include the education and training of the public health workforce as informaticians in this domain are currently limited. Finally, investments should strengthen and expand health data utilities that increasingly play a critical role in exchanging information across public health and healthcare organizations.

Human-Centered Design of a Clinical Decision Support for Anemia Screening in Children with Inflammatory Bowel Disease.

BACKGROUND: Inflammatory bowel disease (IBD) commonly leads to iron deficiency anemia (IDA). Rates of screening and treatment of IDA are often low. A clinical decision support system (CDSS) embedded in an electronic health record could improve adherence to evidence-based care. Rates of CDSS adoption are often low due to poor usability and fit with work processes. One solution is to use human-centered design (HCD), which designs CDSS based on identified user needs and context of use and evaluates prototypes for usefulness and usability. OBJECTIVES: this study aimed to use HCD to design a CDSS tool called the IBD Anemia Diagnosis Tool, IADx. METHODS: Interviews with IBD practitioners informed creation of a process map of anemia care that was used by an interdisciplinary team that used HCD principles to create a prototype CDSS. The prototype was iteratively tested with "Think Aloud" usability evaluation with clinicians as well as semi-structured interviews, a survey, and observations. Feedback was coded and informed redesign. RESULTS: Process mapping showed that IADx should function at in-person encounters and asynchronous laboratory review. Clinicians desired full automation of clinical information acquisition such as laboratory trends and analysis such as calculation of iron deficit, less automation of clinical decision selection such as laboratory ordering, and no automation of action implementation such as signing medication orders. Providers preferred an interruptive alert over a noninterruptive reminder. CONCLUSION: Providers preferred an interruptive alert, perhaps due to the low likelihood of noticing a noninterruptive advisory. High levels of desire for automation of information acquisition and analysis with less automation of decision selection and action may be generalizable to other CDSSs designed for chronic disease management. This underlines the ways in which CDSSs have the potential to augment rather than replace provider cognitive work.

Healthcare utilization is a collider: an introduction to collider bias in EHR data reuse.

OBJECTIVES: Collider bias is a common threat to internal validity in clinical research but is rarely mentioned in informatics education or literature. Conditioning on a collider, which is a variable that is the shared causal descendant of an exposure and outcome, may result in spurious associations between the exposure and outcome. Our objective is to introduce readers to collider bias and its corollaries in the retrospective analysis of electronic health record (EHR) data. TARGET AUDIENCE: Collider bias is likely to arise in the reuse of EHR data, due to data-generating mechanisms and the nature of healthcare access and utilization in the United States. Therefore, this tutorial is aimed at informaticians and other EHR data consumers without a background in epidemiological methods or causal inference. SCOPE: We focus specifically on problems that may arise from conditioning on forms of healthcare utilization, a common collider that is an implicit selection criterion when one reuses EHR data. Directed acyclic graphs (DAGs) are introduced as a tool for identifying potential sources of bias during study design and planning. References for additional resources on causal inference and DAG construction are provided.

Development of a Digital Assistant to Support Teleconsultations Between Remote Physicians and Frontline Health Workers in India: User-Centered Design Approach.

BACKGROUND: Many low- and middle-income countries have adopted telemedicine programs that connect frontline health workers (FHWs) such as nurses, midwives, or community health workers in rural and remote areas with physicians in urban areas to deliver care to patients. By leveraging technology to reduce temporal, financial, and geographical barriers, these health worker-to-physician telemedicine programs have the potential to increase health care quality, expand the specialties available to patients, and reduce the time and cost required to deliver care. OBJECTIVE: We aimed to identify, validate, and prioritize unmet needs in the health care space of health worker-to-physician telemedicine programs and develop and refine a solution that addresses those needs. METHODS: We collected information regarding user needs through ethnographic research, direct observation, and semistructured interviews with 37 stakeholders (n=5, 14% physicians; n=1, 3% public health program manager; n=12, 32% community health workers; and n=19, 51% patients) at 2 telemedicine clinics in rural West Bengal, India. We used the Spiral-Iterative Innovation Model to design and develop a prototype solution to meet these needs. RESULTS: We identified 74 unmet needs through our immersion in health worker-to-physician telemedicine programs. We identified a critical unmet need that achieving optimal teleconsultations in low- and middle-income countries often requires shifting tasks such as history taking and physical examination from high-skilled remote physicians to FHWs. To meet this need, we developed a prototype digital assistant that would allow FHWs to assume some of the tasks carried out by remote clinicians. The user needs of multiple stakeholder groups (patients, FHWs, physicians, and health organizations) were incorporated into the design and features of the task-shifting tool. The final prototype was shared with the health workers, physicians, and public health program managers who expressed that the tool would be useful and valuable. CONCLUSIONS: The final prototype that was developed was released as an open-source digital public good and may improve the quality and efficiency of care delivery in health worker-to-physician telemedicine programs.

Association of Eating and Sleeping Intervals With Weight Change Over Time: The Daily24 Cohort.

Background We aim to evaluate the association between meal intervals and weight trajectory among adults from a clinical cohort. Methods and Results This is a multisite prospective cohort study of adults recruited from 3 health systems. Over the 6-month study period, 547 participants downloaded and used a mobile application to record the timing of meals and sleep for at least 1 day. We obtained information on weight and comorbidities at each outpatient visit from electronic health records for up to 10 years before until 10 months after baseline. We used mixed linear regression to model weight trajectories. Mean age was 51.1 (SD 15.0) years, and body mass index was 30.8 (SD 7.8) kg/m2; 77.9% were women, and 77.5% reported White race. Mean interval from first to last meal was 11.5 (2.3) hours and was not associated with weight change. The number of meals per day was positively associated with weight change. The average difference in annual weight change (95% CI) associated with an increase of 1 daily meal was 0.28 kg (0.02-0.53). Conclusions Number of daily meals was positively associated with weight change over 6 years. Our findings did not support the use of time-restricted eating as a strategy for long-term weight loss in a general medical population.

Applicability Area: A novel utility-based approach for evaluating predictive models, beyond discrimination.

Translating prediction models into practice and supporting clinicians' decision-making demand demonstration of clinical value. Existing approaches to evaluating machine learning models emphasize discriminatory power, which is only a part of the medical decision problem. We propose the Applicability Area (ApAr), a decision-analytic utility-based approach to evaluating predictive models that communicate the range of prior probability and test cutoffs for which the model has positive utility; larger ApArs suggest a broader potential use of the model. We assess ApAr with simulated datasets and with three published medical datasets. ApAr adds value beyond the typical area under the receiver operating characteristic curve (AUROC) metric analysis. As an example, in the diabetes dataset, the top model by ApAr was ranked as the 23rd best model by AUROC. Decision makers looking to adopt and implement models can leverage ApArs to assess if the local range of priors and utilities is within the respective ApArs.

Electronic Health Record-Based Recruitment and Retention and Mobile Health App Usage: Multisite Cohort Study.

BACKGROUND: To address the obesity epidemic, there is a need for novel paradigms, including those that address the timing of eating and sleep in relation to circadian rhythms. Electronic health records (EHRs) are an efficient way to identify potentially eligible participants for health research studies. Mobile health (mHealth) apps offer available and convenient data collection of health behaviors, such as timing of eating and sleep. OBJECTIVE: The aim of this descriptive analysis was to report on recruitment, retention, and app use from a 6-month cohort study using a mobile app called Daily24. METHODS: Using an EHR query, adult patients from three health care systems in the PaTH clinical research network were identified as potentially eligible, invited electronically to participate, and instructed to download and use the Daily24 mobile app, which focuses on eating and sleep timing. Online surveys were completed at baseline and 4 months. We described app use and identified predictors of app use, defined as 1 or more days of use, versus nonuse and usage categories (ie, immediate, consistent, and sustained) using multivariate regression analyses. RESULTS: Of 70,661 patients who were sent research invitations, 1021 (1.44%) completed electronic consent forms and online baseline surveys; 4 withdrew, leaving a total of 1017 participants in the analytic sample. A total of 53.79% (n=547) of the participants were app users and, of those, 75.3% (n=412), 50.1% (n=274), and 25.4% (n=139) were immediate, consistent, and sustained users, respectively. Median app use was 28 (IQR 7-75) days over 6 months. Younger age, White race, higher educational level, higher income, having no children younger than 18 years, and having used 1 to 5 health apps significantly predicted app use (vs nonuse) in adjusted models. Older age and lower BMI predicted early, consistent, and sustained use. About half (532/1017, 52.31%) of the participants completed the 4-month online surveys. A total of 33.5% (183/547), 29.3% (157/536), and 27.1% (143/527) of app users were still using the app for at least 2 days per month during months 4, 5, and 6 of the study, respectively. CONCLUSIONS: EHR recruitment offers an efficient (ie, high reach, low touch, and minimal participant burden) approach to recruiting participants from health care settings into mHealth research. Efforts to recruit and retain less engaged subgroups are needed to collect more generalizable data. Additionally, future app iterations should include more evidence-based features to increase participant use.

Harmonizing units and values of quantitative data elements in a very large nationally pooled electronic health record (EHR) dataset.

OBJECTIVE: The goals of this study were to harmonize data from electronic health records (EHRs) into common units, and impute units that were missing. MATERIALS AND METHODS: The National COVID Cohort Collaborative (N3C) table of laboratory measurement data-over 3.1 billion patient records and over 19 000 unique measurement concepts in the Observational Medical Outcomes Partnership (OMOP) common-data-model format from 55 data partners. We grouped ontologically similar OMOP concepts together for 52 variables relevant to COVID-19 research, and developed a unit-harmonization pipeline comprised of (1) selecting a canonical unit for each measurement variable, (2) arriving at a formula for conversion, (3) obtaining clinical review of each formula, (4) applying the formula to convert data values in each unit into the target canonical unit, and (5) removing any harmonized value that fell outside of accepted value ranges for the variable. For data with missing units for all the results within a lab test for a data partner, we compared values with pooled values of all data partners, using the Kolmogorov-Smirnov test. RESULTS: Of the concepts without missing values, we harmonized 88.1% of the values, and imputed units for 78.2% of records where units were absent (41% of contributors' records lacked units). DISCUSSION: The harmonization and inference methods developed herein can serve as a resource for initiatives aiming to extract insight from heterogeneous EHR collections. Unique properties of centralized data are harnessed to enable unit inference. CONCLUSION: The pipeline we developed for the pooled N3C data enables use of measurements that would otherwise be unavailable for analysis.