Don Lindberg's influence on future generations: The U.S. National Library of Medicine's biomedical informatics research training programs.

Through his visionary leadership as Director of the U.S. National Library of Medicine (NLM), Donald A. B. Lindberg M.D. influenced future generations of informatics professionals and the field of biomedical informatics itself. This chapter describes Dr. Lindberg's role in sponsoring and shaping the NLM's Institutional T15 training programs.

Don Lindberg's Influence on Future Generations: The U.S. National Library of Medicine's Biomedical Informatics Research Training Programs.

Through his visionary leadership as Director of the U.S. National Library of Medicine (NLM), Donald A.B. Lindberg M.D. influenced future generations of informatics professionals and the field of biomedical informatics itself. This chapter describes Dr. Lindberg's role in sponsoring and shaping the NLM's Institutional T15 training programs.

Categorizing metadata to help mobilize computable biomedical knowledge.

INTRODUCTION: Computable biomedical knowledge artifacts (CBKs) are digital objects conveying biomedical knowledge in machine-interpretable structures. As more CBKs are produced and their complexity increases, the value obtained from sharing CBKs grows. Mobilizing CBKs and sharing them widely can only be achieved if the CBKs are findable, accessible, interoperable, reusable, and trustable (FAIR+T). To help mobilize CBKs, we describe our efforts to outline metadata categories to make CBKs FAIR+T. METHODS: We examined the literature regarding metadata with the potential to make digital artifacts FAIR+T. We also examined metadata available online today for actual CBKs of 12 different types. With iterative refinement, we came to a consensus on key categories of metadata that, when taken together, can make CBKs FAIR+T. We use subject-predicate-object triples to more clearly differentiate metadata categories. RESULTS: We defined 13 categories of CBK metadata most relevant to making CBKs FAIR+T. Eleven of these categories (type, domain, purpose, identification, location, CBK-to-CBK relationships, technical, authorization and rights management, provenance, evidential basis, and evidence from use metadata) are evident today where CBKs are stored online. Two additional categories (preservation and integrity metadata) were not evident in our examples. We provide a research agenda to guide further study and development of these and other metadata categories. CONCLUSION: A wide variety of metadata elements in various categories is needed to make CBKs FAIR+T. More work is needed to develop a common framework for CBK metadata that can make CBKs FAIR+T for all stakeholders.

Summary of fourth annual MCBK public meeting: Mobilizing computable biomedical knowledge-metadata and trust.

The exponential growth of biomedical knowledge in computable formats challenges organizations to consider mobilizing artifacts in findable, accessible, interoperable, reusable, and trustable (FAIR+T) ways1. There is a growing need to apply biomedical knowledge artifacts to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations lack the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are insufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community formed in 2016 to address these needs. This report summarizes the main outputs of the Fourth Annual MCBK public meeting, which was held virtually July 20 to July 21, 2021 and convened over 100 participants spanning diverse domains to frame and address important dimensions for mobilizing CBK.

Machine learning approaches for the prediction of bone mineral density by using genomic and phenotypic data of 5130 older men.

The study aimed to utilize machine learning (ML) approaches and genomic data to develop a prediction model for bone mineral density (BMD) and identify the best modeling approach for BMD prediction. The genomic and phenotypic data of Osteoporotic Fractures in Men Study (n = 5130) was analyzed. Genetic risk score (GRS) was calculated from 1103 associated SNPs for each participant after a comprehensive genotype imputation. Data were normalized and divided into a training set (80%) and a validation set (20%) for analysis. Random forest, gradient boosting, neural network, and linear regression were used to develop BMD prediction models separately. Ten-fold cross-validation was used for hyper-parameters optimization. Mean square error and mean absolute error were used to assess model performance. When using GRS and phenotypic covariates as the predictors, all ML models' performance and linear regression in BMD prediction were similar. However, when replacing GRS with the 1103 individual SNPs in the model, ML models performed significantly better than linear regression (with lasso regularization), and the gradient boosting model performed the best. Our study suggested that ML models, especially gradient boosting, can improve BMD prediction in genomic data.

Summary of third annual MCBK public meeting: Mobilizing computable biomedical knowledge-Accelerating the second knowledge revolution.

The volume of biomedical knowledge is growing exponentially and much of this knowledge is represented in computer executable formats, such as models, algorithms, and programmatic code. There is a growing need to apply this knowledge to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations do not yet have the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are not sufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community was formed in 2016 to address these needs. This report summarizes the main outputs of the third annual MCBK public meeting, which was held virtually from June 30 to July 1, 2020 and brought together over 200 participants from various domains to frame and address important dimensions for mobilizing CBK.

Summary of second annual MCBK public meeting: Mobilizing Computable Biomedical Knowledge-A movement to accelerate translation of knowledge into action.

The volume of biomedical knowledge is growing exponentially and much of this knowledge is represented in computer executable formats, such as models, algorithms and programmatic code. There is a growing need to apply this knowledge to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations do not yet have the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are not sufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community formed in 2016 to address these needs. This report summarizes the main outputs of the Second Annual MCBK public meeting, which was held at the National Institutes of Health on July 18-19, 2019 and brought together over 150 participants from various domains to frame and address important dimensions for mobilizing CBK.

Preferences of older adults and their families for Meaningful Use clinical summaries.

BACKGROUND: This study assessed the perceptions of older adults regarding the plan of care (POC) contained in the clinical summary mandated by the Electronic Health Records (EHR) Incentive Program. METHODS: A qualitative descriptive design was selected for this study. Older adults (≥65) with chronic cardiac diagnoses were invited to participate. The investigator shadowed the physician during the patient encounter, interviewed the patients following their encounter, and asked patients to complete standard health literacy and cognitive screening tools and the Patient Activation Measure. Directed content analysis was used to analyze transcripts. RESULTS: Patients (n=40) found the clinical summary useful for sharing information with family members and other physicians, reminding and informing, and for engaging in behavior change. Seventy-six percent reported that they would not go online to access the clinical summary for multiple reasons, including not being "computer savvy" and privacy concerns. Participant recommendations for a re-designed, improved clinical summary are included. The clinical summary helps patients and families communicate among health care professionals in a complex, disjointed health care system that often burdens patients with that responsibility. The majority of participants were happy with the paper version and offered multiple reasons for not wanting online access that may help us to focus on more compelling reasons for patient portal use. CONCLUSIONS: Qualitatively, it appears that the clinical summary is a useful tool for engaging people with chronic disease in self-management. The participants in this study told us what many of us already know to be true; that the documentation we provide patients and families is less than ideal.

Clinical decision support models and frameworks: Seeking to address research issues underlying implementation successes and failures.

Computer-based clinical decision support (CDS) has been pursued for more than five decades. Despite notable accomplishments and successes, wide adoption and broad use of CDS in clinical practice has not been achieved. Many issues have been identified as being partially responsible for the relatively slow adoption and lack of impact, including deficiencies in leadership, recognition of purpose, understanding of human interaction and workflow implications of CDS, cognitive models of the role of CDS, and proprietary implementations with limited interoperability and sharing. To address limitations, many approaches have been proposed and evaluated, drawing on theoretical frameworks, as well as management, technical and other disciplines and experiences. It seems clear, because of the multiple perspectives involved, that no single model or framework is adequate to encompass these challenges. This Viewpoint paper seeks to review the various foci of CDS and to identify aspects in which theoretical models and frameworks for CDS have been explored or could be explored and where they might be expected to be most useful.

50th Anniversary International Medical Informatics Association (IMIA) History Working Group and Its Projects.

The IMIA History Working Group has as its first goal the editing of a volume of contributions from pioneers and leaders in the field of biomedical and health informatics (BMHI) to commemorate the 50th anniversary of IMIA's predecessor IFIP-TC4. This paper describes how the IMIA History WG evolved from an earlier Taskforce, and has focused on producing the edited book of original contributions. We describe its proposed outline of objectives for the personal stories, and national and regional society narratives, together with some comments on the evolution of Medinfo meeting contributions over the years, to provide a reference source for the early motivations of the scientific, clinical, educational, and professional changes that have influenced the historical course of our field.

Granular Quality Reporting for Cervical Cytology Testing.

Quality reporting for cervical cancer prevention is focused on patients with normal cervical cytology, and excludes patients with cytological abnormalities that may be at higher risk. The major obstacles for granular reporting are the complexity of surveillance guidelines and free-text data. We performed automated chart review to compare the cytology testing rates for patients with 'atypical squamous cells of undetermined significance' (ASCUS) cytology, with the rates for patients with normal cytology. We modeled the surveillance guidelines, and extracted information from free-text cytology reports, to perform this study on 28101 female patients. Our results show that patients with ASCUS cytology had significantly higher adherence rates (94.9%) than those for patients with normal cytology (90.4%). Overall our study indicates that the quality of care varies significantly between the high and average risk patients. Our study demonstrates the use of health information technology for higher granularity of reporting for cervical cytology testing.

Evaluation of the effect of decision support on the efficiency of primary care providers in the outpatient practice.

BACKGROUND: Clinical decision support (CDS) for primary care has been shown to improve delivery of preventive services. However, there is little evidence for efficiency of physicians due to CDS assistance. In this article, we report a pilot study for measuring the impact of CDS on the time spent by physicians for deciding on preventive services and chronic disease management. METHODS: We randomly selected 30 patients from a primary care practice, and assigned them to 10 physicians. The physicians were requested to perform chart review to decide on preventive services and chronic disease management for the assigned patients. The patients assignment was done in a randomized crossover design, such that each patient received 2 sets of recommendations-one from a physician with CDS assistance and the other from a different physician without CDS assistance. We compared the physician recommendations made using CDS assistance, with the recommendations made without CDS assistance. RESULTS: The physicians required an average of 1 minute 44 seconds, when they were they had access to the decision support system and 5 minutes when they were unassisted. Hence the CDS assistance resulted in an estimated saving of 3 minutes 16 seconds (65%) of the physicians' time, which was statistically significant (P < .0001). There was no statistically significant difference in the number of recommendations. CONCLUSION: Our findings suggest that CDS assistance significantly reduced the time spent by physicians for deciding on preventive services and chronic disease management. The result needs to be confirmed by performing similar studies at other institutions.

Automated recommendation for cervical cancer screening and surveillance.

Because of the complexity of cervical cancer prevention guidelines, clinicians often fail to follow best-practice recommendations. Moreover, existing clinical decision support (CDS) systems generally recommend a cervical cytology every three years for all female patients, which is inappropriate for patients with abnormal findings that require surveillance at shorter intervals. To address this problem, we developed a decision tree-based CDS system that integrates national guidelines to provide comprehensive guidance to clinicians. Validation was performed in several iterations by comparing recommendations generated by the system with those of clinicians for 333 patients. The CDS system extracted relevant patient information from the electronic health record and applied the guideline model with an overall accuracy of 87%. Providers without CDS assistance needed an average of 1 minute 39 seconds to decide on recommendations for management of abnormal findings. Overall, our work demonstrates the feasibility and potential utility of automated recommendation system for cervical cancer screening and surveillance.

Workflow-based Data Reconciliation for Clinical Decision Support: Case of Colorectal Cancer Screening and Surveillance.

A major barrier for computer-based clinical decision support (CDS), is the difficulty in obtaining the patient information required for decision making. The information gap is often due to deficiencies in the clinical documentation. One approach to address this gap is to gather and reconcile data from related documents or data sources. In this paper we consider the case of a CDS system for colorectal cancer screening and surveillance. We describe the use of workflow analysis to design data reconciliation processes. Further, we perform a quantitative analysis of the impact of these processes on system performance using a dataset of 106 patients. Results show that data reconciliation considerably improves the performance of the system. Our study demonstrates that, workflow-based data reconciliation can play a vital role in designing new-generation CDS systems that are based on complex guideline models and use natural language processing (NLP) to obtain patient data.

Formative evaluation of the accuracy of a clinical decision support system for cervical cancer screening.

OBJECTIVES: We previously developed and reported on a prototype clinical decision support system (CDSS) for cervical cancer screening. However, the system is complex as it is based on multiple guidelines and free-text processing. Therefore, the system is susceptible to failures. This report describes a formative evaluation of the system, which is a necessary step to ensure deployment readiness of the system. MATERIALS AND METHODS: Care providers who are potential end-users of the CDSS were invited to provide their recommendations for a random set of patients that represented diverse decision scenarios. The recommendations of the care providers and those generated by the CDSS were compared. Mismatched recommendations were reviewed by two independent experts. RESULTS: A total of 25 users participated in this study and provided recommendations for 175 cases. The CDSS had an accuracy of 87% and 12 types of CDSS errors were identified, which were mainly due to deficiencies in the system's guideline rules. When the deficiencies were rectified, the CDSS generated optimal recommendations for all failure cases, except one with incomplete documentation. DISCUSSION AND CONCLUSIONS: The crowd-sourcing approach for construction of the reference set, coupled with the expert review of mismatched recommendations, facilitated an effective evaluation and enhancement of the system, by identifying decision scenarios that were missed by the system's developers. The described methodology will be useful for other researchers who seek rapidly to evaluate and enhance the deployment readiness of complex decision support systems.

Healthcare information technology and economics.

At the 2011 American College of Medical Informatics (ACMI) Winter Symposium we studied the overlap between health IT and economics and what leading healthcare delivery organizations are achieving today using IT that might offer paths for the nation to follow for using health IT in healthcare reform. We recognized that health IT by itself can improve health value, but its main contribution to health value may be that it can make possible new care delivery models to achieve much larger value. Health IT is a critically important enabler to fundamental healthcare system changes that may be a way out of our current, severe problem of rising costs and national deficit. We review the current state of healthcare costs, federal health IT stimulus programs, and experiences of several leading organizations, and offer a model for how health IT fits into our health economic future.

Clinical decision support with automated text processing for cervical cancer screening.

OBJECTIVE: To develop a computerized clinical decision support system (CDSS) for cervical cancer screening that can interpret free-text Papanicolaou (Pap) reports. MATERIALS AND METHODS: The CDSS was constituted by two rulebases: the free-text rulebase for interpreting Pap reports and a guideline rulebase. The free-text rulebase was developed by analyzing a corpus of 49 293 Pap reports. The guideline rulebase was constructed using national cervical cancer screening guidelines. The CDSS accesses the electronic medical record (EMR) system to generate patient-specific recommendations. For evaluation, the screening recommendations made by the CDSS for 74 patients were reviewed by a physician. RESULTS AND DISCUSSION: Evaluation revealed that the CDSS outputs the optimal screening recommendations for 73 out of 74 test patients and it identified two cases for gynecology referral that were missed by the physician. The CDSS aided the physician to amend recommendations in six cases. The failure case was because human papillomavirus (HPV) testing was sometimes performed separately from the Pap test and these results were reported by a laboratory system that was not queried by the CDSS. Subsequently, the CDSS was upgraded to look up the HPV results missed earlier and it generated the optimal recommendations for all 74 test cases. LIMITATIONS: Single institution and single expert study. CONCLUSION: An accurate CDSS system could be constructed for cervical cancer screening given the standardized reporting of Pap tests and the availability of explicit guidelines. Overall, the study demonstrates that free text in the EMR can be effectively utilized through natural language processing to develop clinical decision support tools.

The morningside initiative: collaborative development of a knowledge repository to accelerate adoption of clinical decision support.

The Morningside Initiative is a public-private activity that has evolved from an August, 2007, meeting at the Morningside Inn, in Frederick, MD, sponsored by the Telemedicine and Advanced Technology Research Center (TATRC) of the US Army Medical Research Materiel Command. Participants were subject matter experts in clinical decision support (CDS) and included representatives from the Department of Defense, Veterans Health Administration, Kaiser Permanente, Partners Healthcare System, Henry Ford Health System, Arizona State University, and the American Medical Informatics Association (AMIA). The Morningside Initiative was convened in response to the AMIA Roadmap for National Action on Clinical Decision Support and on the basis of other considerations and experiences of the participants. Its formation was the unanimous recommendation of participants at the 2007 meeting which called for creating a shared repository of executable knowledge for diverse health care organizations and practices, as well as health care system vendors. The rationale is based on the recognition that sharing of clinical knowledge needed for CDS across organizations is currently virtually non-existent, and that, given the considerable investment needed for creating, maintaining and updating authoritative knowledge, which only larger organizations have been able to undertake, this is an impediment to widespread adoption and use of CDS. The Morningside Initiative intends to develop and refine (1) an organizational framework, (2) a technical approach, and (3) CDS content acquisition and management processes for sharing CDS knowledge content, tools, and experience that will scale with growing numbers of participants and can be expanded in scope of content and capabilities. Intermountain Healthcare joined the initial set of participants shortly after its formation. The efforts of the Morningside Initiative are intended to serve as the basis for a series of next steps in a national agenda for CDS. It is based on the belief that sharing of knowledge can be highly effective as is the case in other competitive domains such as genomics. Participants in the Morningside Initiative believe that a coordinated effort between the private and public sectors is needed to accomplish this goal and that a small number of highly visible and respected health care organizations in the public and private sector can lead by example. Ultimately, a future collaborative knowledge sharing organization must have a sustainable long-term business model for financial support.