Don Lindberg's influence on future generations: The U.S. National Library of Medicine's biomedical informatics research training programs.

Through his visionary leadership as Director of the U.S. National Library of Medicine (NLM), Donald A. B. Lindberg M.D. influenced future generations of informatics professionals and the field of biomedical informatics itself. This chapter describes Dr. Lindberg's role in sponsoring and shaping the NLM's Institutional T15 training programs.

Don Lindberg's Influence on Future Generations: The U.S. National Library of Medicine's Biomedical Informatics Research Training Programs.

Through his visionary leadership as Director of the U.S. National Library of Medicine (NLM), Donald A.B. Lindberg M.D. influenced future generations of informatics professionals and the field of biomedical informatics itself. This chapter describes Dr. Lindberg's role in sponsoring and shaping the NLM's Institutional T15 training programs.

Categorizing metadata to help mobilize computable biomedical knowledge.

INTRODUCTION: Computable biomedical knowledge artifacts (CBKs) are digital objects conveying biomedical knowledge in machine-interpretable structures. As more CBKs are produced and their complexity increases, the value obtained from sharing CBKs grows. Mobilizing CBKs and sharing them widely can only be achieved if the CBKs are findable, accessible, interoperable, reusable, and trustable (FAIR+T). To help mobilize CBKs, we describe our efforts to outline metadata categories to make CBKs FAIR+T. METHODS: We examined the literature regarding metadata with the potential to make digital artifacts FAIR+T. We also examined metadata available online today for actual CBKs of 12 different types. With iterative refinement, we came to a consensus on key categories of metadata that, when taken together, can make CBKs FAIR+T. We use subject-predicate-object triples to more clearly differentiate metadata categories. RESULTS: We defined 13 categories of CBK metadata most relevant to making CBKs FAIR+T. Eleven of these categories (type, domain, purpose, identification, location, CBK-to-CBK relationships, technical, authorization and rights management, provenance, evidential basis, and evidence from use metadata) are evident today where CBKs are stored online. Two additional categories (preservation and integrity metadata) were not evident in our examples. We provide a research agenda to guide further study and development of these and other metadata categories. CONCLUSION: A wide variety of metadata elements in various categories is needed to make CBKs FAIR+T. More work is needed to develop a common framework for CBK metadata that can make CBKs FAIR+T for all stakeholders.

Summary of fourth annual MCBK public meeting: Mobilizing computable biomedical knowledge-metadata and trust.

The exponential growth of biomedical knowledge in computable formats challenges organizations to consider mobilizing artifacts in findable, accessible, interoperable, reusable, and trustable (FAIR+T) ways1. There is a growing need to apply biomedical knowledge artifacts to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations lack the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are insufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community formed in 2016 to address these needs. This report summarizes the main outputs of the Fourth Annual MCBK public meeting, which was held virtually July 20 to July 21, 2021 and convened over 100 participants spanning diverse domains to frame and address important dimensions for mobilizing CBK.

Machine learning approaches for the prediction of bone mineral density by using genomic and phenotypic data of 5130 older men.

The study aimed to utilize machine learning (ML) approaches and genomic data to develop a prediction model for bone mineral density (BMD) and identify the best modeling approach for BMD prediction. The genomic and phenotypic data of Osteoporotic Fractures in Men Study (n = 5130) was analyzed. Genetic risk score (GRS) was calculated from 1103 associated SNPs for each participant after a comprehensive genotype imputation. Data were normalized and divided into a training set (80%) and a validation set (20%) for analysis. Random forest, gradient boosting, neural network, and linear regression were used to develop BMD prediction models separately. Ten-fold cross-validation was used for hyper-parameters optimization. Mean square error and mean absolute error were used to assess model performance. When using GRS and phenotypic covariates as the predictors, all ML models' performance and linear regression in BMD prediction were similar. However, when replacing GRS with the 1103 individual SNPs in the model, ML models performed significantly better than linear regression (with lasso regularization), and the gradient boosting model performed the best. Our study suggested that ML models, especially gradient boosting, can improve BMD prediction in genomic data.

Summary of third annual MCBK public meeting: Mobilizing computable biomedical knowledge-Accelerating the second knowledge revolution.

The volume of biomedical knowledge is growing exponentially and much of this knowledge is represented in computer executable formats, such as models, algorithms, and programmatic code. There is a growing need to apply this knowledge to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations do not yet have the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are not sufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community was formed in 2016 to address these needs. This report summarizes the main outputs of the third annual MCBK public meeting, which was held virtually from June 30 to July 1, 2020 and brought together over 200 participants from various domains to frame and address important dimensions for mobilizing CBK.

Summary of second annual MCBK public meeting: Mobilizing Computable Biomedical Knowledge-A movement to accelerate translation of knowledge into action.

The volume of biomedical knowledge is growing exponentially and much of this knowledge is represented in computer executable formats, such as models, algorithms and programmatic code. There is a growing need to apply this knowledge to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations do not yet have the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are not sufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community formed in 2016 to address these needs. This report summarizes the main outputs of the Second Annual MCBK public meeting, which was held at the National Institutes of Health on July 18-19, 2019 and brought together over 150 participants from various domains to frame and address important dimensions for mobilizing CBK.

Preferences of older adults and their families for Meaningful Use clinical summaries.

BACKGROUND: This study assessed the perceptions of older adults regarding the plan of care (POC) contained in the clinical summary mandated by the Electronic Health Records (EHR) Incentive Program. METHODS: A qualitative descriptive design was selected for this study. Older adults (≥65) with chronic cardiac diagnoses were invited to participate. The investigator shadowed the physician during the patient encounter, interviewed the patients following their encounter, and asked patients to complete standard health literacy and cognitive screening tools and the Patient Activation Measure. Directed content analysis was used to analyze transcripts. RESULTS: Patients (n=40) found the clinical summary useful for sharing information with family members and other physicians, reminding and informing, and for engaging in behavior change. Seventy-six percent reported that they would not go online to access the clinical summary for multiple reasons, including not being "computer savvy" and privacy concerns. Participant recommendations for a re-designed, improved clinical summary are included. The clinical summary helps patients and families communicate among health care professionals in a complex, disjointed health care system that often burdens patients with that responsibility. The majority of participants were happy with the paper version and offered multiple reasons for not wanting online access that may help us to focus on more compelling reasons for patient portal use. CONCLUSIONS: Qualitatively, it appears that the clinical summary is a useful tool for engaging people with chronic disease in self-management. The participants in this study told us what many of us already know to be true; that the documentation we provide patients and families is less than ideal.

Clinical decision support models and frameworks: Seeking to address research issues underlying implementation successes and failures.

Computer-based clinical decision support (CDS) has been pursued for more than five decades. Despite notable accomplishments and successes, wide adoption and broad use of CDS in clinical practice has not been achieved. Many issues have been identified as being partially responsible for the relatively slow adoption and lack of impact, including deficiencies in leadership, recognition of purpose, understanding of human interaction and workflow implications of CDS, cognitive models of the role of CDS, and proprietary implementations with limited interoperability and sharing. To address limitations, many approaches have been proposed and evaluated, drawing on theoretical frameworks, as well as management, technical and other disciplines and experiences. It seems clear, because of the multiple perspectives involved, that no single model or framework is adequate to encompass these challenges. This Viewpoint paper seeks to review the various foci of CDS and to identify aspects in which theoretical models and frameworks for CDS have been explored or could be explored and where they might be expected to be most useful.

Granular Quality Reporting for Cervical Cytology Testing.

Quality reporting for cervical cancer prevention is focused on patients with normal cervical cytology, and excludes patients with cytological abnormalities that may be at higher risk. The major obstacles for granular reporting are the complexity of surveillance guidelines and free-text data. We performed automated chart review to compare the cytology testing rates for patients with 'atypical squamous cells of undetermined significance' (ASCUS) cytology, with the rates for patients with normal cytology. We modeled the surveillance guidelines, and extracted information from free-text cytology reports, to perform this study on 28101 female patients. Our results show that patients with ASCUS cytology had significantly higher adherence rates (94.9%) than those for patients with normal cytology (90.4%). Overall our study indicates that the quality of care varies significantly between the high and average risk patients. Our study demonstrates the use of health information technology for higher granularity of reporting for cervical cytology testing.

Evaluation of the effect of decision support on the efficiency of primary care providers in the outpatient practice.

BACKGROUND: Clinical decision support (CDS) for primary care has been shown to improve delivery of preventive services. However, there is little evidence for efficiency of physicians due to CDS assistance. In this article, we report a pilot study for measuring the impact of CDS on the time spent by physicians for deciding on preventive services and chronic disease management. METHODS: We randomly selected 30 patients from a primary care practice, and assigned them to 10 physicians. The physicians were requested to perform chart review to decide on preventive services and chronic disease management for the assigned patients. The patients assignment was done in a randomized crossover design, such that each patient received 2 sets of recommendations-one from a physician with CDS assistance and the other from a different physician without CDS assistance. We compared the physician recommendations made using CDS assistance, with the recommendations made without CDS assistance. RESULTS: The physicians required an average of 1 minute 44 seconds, when they were they had access to the decision support system and 5 minutes when they were unassisted. Hence the CDS assistance resulted in an estimated saving of 3 minutes 16 seconds (65%) of the physicians' time, which was statistically significant (P < .0001). There was no statistically significant difference in the number of recommendations. CONCLUSION: Our findings suggest that CDS assistance significantly reduced the time spent by physicians for deciding on preventive services and chronic disease management. The result needs to be confirmed by performing similar studies at other institutions.

Automated recommendation for cervical cancer screening and surveillance.

Because of the complexity of cervical cancer prevention guidelines, clinicians often fail to follow best-practice recommendations. Moreover, existing clinical decision support (CDS) systems generally recommend a cervical cytology every three years for all female patients, which is inappropriate for patients with abnormal findings that require surveillance at shorter intervals. To address this problem, we developed a decision tree-based CDS system that integrates national guidelines to provide comprehensive guidance to clinicians. Validation was performed in several iterations by comparing recommendations generated by the system with those of clinicians for 333 patients. The CDS system extracted relevant patient information from the electronic health record and applied the guideline model with an overall accuracy of 87%. Providers without CDS assistance needed an average of 1 minute 39 seconds to decide on recommendations for management of abnormal findings. Overall, our work demonstrates the feasibility and potential utility of automated recommendation system for cervical cancer screening and surveillance.

Workflow-based Data Reconciliation for Clinical Decision Support: Case of Colorectal Cancer Screening and Surveillance.

A major barrier for computer-based clinical decision support (CDS), is the difficulty in obtaining the patient information required for decision making. The information gap is often due to deficiencies in the clinical documentation. One approach to address this gap is to gather and reconcile data from related documents or data sources. In this paper we consider the case of a CDS system for colorectal cancer screening and surveillance. We describe the use of workflow analysis to design data reconciliation processes. Further, we perform a quantitative analysis of the impact of these processes on system performance using a dataset of 106 patients. Results show that data reconciliation considerably improves the performance of the system. Our study demonstrates that, workflow-based data reconciliation can play a vital role in designing new-generation CDS systems that are based on complex guideline models and use natural language processing (NLP) to obtain patient data.

Formative evaluation of the accuracy of a clinical decision support system for cervical cancer screening.

OBJECTIVES: We previously developed and reported on a prototype clinical decision support system (CDSS) for cervical cancer screening. However, the system is complex as it is based on multiple guidelines and free-text processing. Therefore, the system is susceptible to failures. This report describes a formative evaluation of the system, which is a necessary step to ensure deployment readiness of the system. MATERIALS AND METHODS: Care providers who are potential end-users of the CDSS were invited to provide their recommendations for a random set of patients that represented diverse decision scenarios. The recommendations of the care providers and those generated by the CDSS were compared. Mismatched recommendations were reviewed by two independent experts. RESULTS: A total of 25 users participated in this study and provided recommendations for 175 cases. The CDSS had an accuracy of 87% and 12 types of CDSS errors were identified, which were mainly due to deficiencies in the system's guideline rules. When the deficiencies were rectified, the CDSS generated optimal recommendations for all failure cases, except one with incomplete documentation. DISCUSSION AND CONCLUSIONS: The crowd-sourcing approach for construction of the reference set, coupled with the expert review of mismatched recommendations, facilitated an effective evaluation and enhancement of the system, by identifying decision scenarios that were missed by the system's developers. The described methodology will be useful for other researchers who seek rapidly to evaluate and enhance the deployment readiness of complex decision support systems.

Healthcare information technology and economics.

At the 2011 American College of Medical Informatics (ACMI) Winter Symposium we studied the overlap between health IT and economics and what leading healthcare delivery organizations are achieving today using IT that might offer paths for the nation to follow for using health IT in healthcare reform. We recognized that health IT by itself can improve health value, but its main contribution to health value may be that it can make possible new care delivery models to achieve much larger value. Health IT is a critically important enabler to fundamental healthcare system changes that may be a way out of our current, severe problem of rising costs and national deficit. We review the current state of healthcare costs, federal health IT stimulus programs, and experiences of several leading organizations, and offer a model for how health IT fits into our health economic future.

Clinical decision support with automated text processing for cervical cancer screening.

OBJECTIVE: To develop a computerized clinical decision support system (CDSS) for cervical cancer screening that can interpret free-text Papanicolaou (Pap) reports. MATERIALS AND METHODS: The CDSS was constituted by two rulebases: the free-text rulebase for interpreting Pap reports and a guideline rulebase. The free-text rulebase was developed by analyzing a corpus of 49 293 Pap reports. The guideline rulebase was constructed using national cervical cancer screening guidelines. The CDSS accesses the electronic medical record (EMR) system to generate patient-specific recommendations. For evaluation, the screening recommendations made by the CDSS for 74 patients were reviewed by a physician. RESULTS AND DISCUSSION: Evaluation revealed that the CDSS outputs the optimal screening recommendations for 73 out of 74 test patients and it identified two cases for gynecology referral that were missed by the physician. The CDSS aided the physician to amend recommendations in six cases. The failure case was because human papillomavirus (HPV) testing was sometimes performed separately from the Pap test and these results were reported by a laboratory system that was not queried by the CDSS. Subsequently, the CDSS was upgraded to look up the HPV results missed earlier and it generated the optimal recommendations for all 74 test cases. LIMITATIONS: Single institution and single expert study. CONCLUSION: An accurate CDSS system could be constructed for cervical cancer screening given the standardized reporting of Pap tests and the availability of explicit guidelines. Overall, the study demonstrates that free text in the EMR can be effectively utilized through natural language processing to develop clinical decision support tools.

Informatics and a health care strategy for the future--general directions.

Central problems in health care involve availability, access, quality, and cost. A major part of a health care strategy also involves disease prevention and promotion of healthy lifestyles, which go well beyond the purview of the health care system itself. Implementing any strategy involves health policy, finance, and management expertise. What then is the role of informatics? We take the position here that informatics is a key enabler both for addressing availability, access, quality, and cost, and also for supporting the work of health policy, finance, and management experts. Informatics provides the necessary information technology (IT) infrastructure, standards, tools, and data to be able to address these key topics and for carrying out the work of the experts. In that sense, we regard informatics as a means for social engineering--the availability of these capabilities brings stakeholders to the table who might otherwise not have reason to or be able to work together.

Reducing diagnostic error with computer-based clinical decision support.

Information technology approaches to delivering diagnostic clinical decision support (CDS) are the subject of the papers to follow in the proceedings. These will address the history of CDS and present day approaches (Miller), evaluation of diagnostic CDS methods (Friedman), and the role of clinical documentation in supporting diagnostic decision making (Schiff). In addition, several other considerations relating to this topic are interesting to ponder. We are moving toward increased understanding of gene regulation and gene expression, identification of biomarkers, and the ability to predict patient response to disease and to tailor treatments to these individual variations-referred to as "personalized" or, more recently, "predictive" medicine. Consequently, diagnostic decision making is more and more linked to management decision making, and generic diagnostic labels like "diabetes" or "colon cancer" will no longer be sufficient, because they don't tell us what to do. Ultimately, if we have more complete data including more structured capture of phenomic data as well as the characterization of the patient's genome, direct prediction from responses of highly refined subsets of similar patients in a database can be used to select appropriate management, the effectiveness of which was demonstrated in projects in selected limited domains as early as the 1970s. In general, there are six classes of methodologies, including the above, which can be applied to delivering CDS. In addition, patients are becoming more knowledgeable and should be regarded as active participants, not only in helping to obtain data but also in their own status assessment and as recipients of decision support. With the above advances, this is a very promising time to be engaged in pursuit of methods of CDS.

Commentary: Informatics in biomedicine and health care.

During the last two decades, biomedical informatics (BMI) has become a critical component in biomedical research and health care delivery, as evidenced by two recent phenomena. One, as discussed in the article by Bernstam and colleagues in this issue, has been the introduction of Clinical and Translational Science Awards. Perhaps even more important has been the recent, arguably long overdue, emphasis on deployment of health information technology (IT) nationally. BMI utilizes IT and computer science as tools and methods for improving data acquisition, data management, data analysis, and knowledge generation, but it is driven by a focus on applications based in deep understanding of the science and practice, problems, interactions, culture, and milieu of biomedicine and health. Building from Bernstam and colleagues' distinction between BMI and other IT disciplines, the authors discuss the evolving role of BMI professionals as individuals uniquely positioned to work within the human and organizational context and culture in which the IT is being applied. The focus is not on the IT but on the combination--the interactions of IT systems, human beings, and organizations aimed at achieving a particular purpose. There has never been a time when the need for individuals well trained in BMI--those who understand the complexities of the human, social, and organizational milieu of biomedicine and health--has been more critical than it is now, as the nation seeks to develop a national infrastructure for biomedicine and health care, and as these fields seek to broadly deploy IT wisely and appropriately.