Toward Real-World Reproducibility: Verifying Value Sets for Clinical Research.

Standardized operational definitions are an important tool to improve reproducibility of research using secondary real-world healthcare data. This approach was leveraged for studies evaluating the effectiveness of AZD7442 as COVID-19 pre-exposure prophylaxis across multiple healthcare systems. Value sets were defined, grouped, and mapped. Results of this exercise were reviewed and recorded. Value sets were updated to reflect findings.

Creating a Real-World Data, United States Healthcare Claims-Based Adaptation of Kurtzke Functional Systems Scores for Assessing Multiple Sclerosis Severity and Progression.

INTRODUCTION: This article describes the development of a unique mapping of the Kurtzke Functional Systems Scores (KFSS) from International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) codes among multiple sclerosis (MS) patients within a US Integrated Delivery Network (IDN). Valid identification of increasing disability may allow deeper insight into MS progression and possible treatments. METHODS: This cohort study identified MS patients in the IDN, Intermountain Healthcare. Experienced clinicians and informaticists mapped electronic health record ICD-9-CM codes to KFSS components generating a modified Kurtzke Expanded Disability Status Scale (EDSS). Modified EDSS scores were used to assess disability progression by calculating means, medians, ranges, and changes in KFSS and modified EDSS scores. RESULTS: Overall, 608/2960 (20.5%) patients were identified as having MS progression and presented a wide range of scores on the EDSS 10-point scale. The median (range) first and second EDSS scores were 0 (0-6) and 5 (1-8), respectively. The median (range) change from first to second score was 5 (1-7.5). The median first KFSS score for all systems was 0, and the mean differed among components. The highest mean first KFSS score (1.06) was measured for sensory function and lowest (0.12) for cerebellar functions. Of the 544 patients with their first EDSS scores in the ≤ 2.5 group, 75.2% and 15.1% had their second EDSS scores in group 3-5.5 and ≥ 6, respectively. Of the 62 patients with their first EDSS score in the 3-5.5 group, 58.1% had their second EDSS scores in group ≥ 6. CONCLUSION: This innovative mapping technique is a promising method for future comparative effectiveness and safety research of Disease-Modifying Therapy in Real-World Data repositories. Future research to validate and expand on this method in another healthcare database is encouraged.

Risperdal® CONSTA® Needle Detachment. Incidence Rates Before and After Kit Redesign: A Retrospective Study using Electronic Health Records and Natural Language Processing in the Department of Veterans Affairs.

INTRODUCTION: Janssen received reports of needle detachments for Risperdal® CONSTA® and, in response, redesigned the kit. OBJECTIVE: The study objective was to estimate the rate of Risperdal® CONSTA® needle detachments prior to and after the introduction of a redesigned kit. METHODS: This retrospective study used record abstraction in the US Department of Veterans Affairs (VA). The 3 phases included: (1) a pilot study for methods evaluation in a sample of 6 hospitals with previously reported detachments; (2) a baseline study to ascertain the baseline detachment rate; and (3) a follow-up study to ascertain the rate for the redesigned kit. Administrative codes and natural language processing with clinical review were used to identify detachments. RESULTS: Pilot: we identified a subset of spontaneously reported detachments and several previously unreported events. In the baseline study (original device), from January through December 2013, 22 needle detachments were identified among 47,934 administrations of the drug in a census of administrations in the VA; an incidence of 0.0459%. In the follow-up study (redesigned device), from December 2015 through December 2016, there were 14 reported detachments in 41,819 injections, 0.0335%. This represents a reduction of 27% from the baseline. CONCLUSION: This approach enabled us to identify needle detachments we would not have otherwise found ("solicited"). However, it likely resulted in incomplete outcome ascertainment. While this may have resulted in lower overall rates, it did not bias the comparison of the baseline and follow-up studies. The results showed that the redesigned Risperdal® CONSTA® kit reduced the incidence of needle detachment events in the VA. FUNDING: Janssen Pharmaceuticals, Inc.

Identifying Patients With Relapsing-Remitting Multiple Sclerosis Using Algorithms Applied to US Integrated Delivery Network Healthcare Data.

BACKGROUND: Relapsing-remitting multiple sclerosis (RRMS) has a major impact on affected patients; therefore, improved understanding of RRMS is important, particularly in the context of real-world evidence. OBJECTIVES: To develop and validate algorithms for identifying patients with RRMS in both unstructured clinical notes found in electronic health records (EHRs) and structured/coded health care claims data. METHODS: US Integrated Delivery Network data (2010-2014) were queried for study inclusion criteria (possible multiple sclerosis [MS] base cohort): one or more MS diagnosis code, patients aged 18 years or older, 1 year or more baseline history, and no other demyelinating diseases. Sets of algorithms were developed to search narrative text of unstructured clinical notes (EHR clinical notes-based algorithms) and structured/coded data (claims-based algorithms) to identify adult patients with RRMS, excluding patients with evidence of progressive MS. Medical records were reviewed manually for algorithm validation. Positive predictive value was calculated for both EHR clinical notes-based and claims-based algorithms. RESULTS: From a sample of 5308 patients with possible MS, 837 patients with RRMS were identified using only the EHR clinical notes-based algorithms and 2271 patients were identified using only the claims-based algorithms; 779 patients were identified using both algorithms. The positive predictive value was 99.1% (95% confidence interval [CI], 94.2%-100%) for the EHR clinical notes-based algorithms and 94.6% (95% CI, 89.1%-97.8%) to 94.9% (95% CI, 89.8%-97.9%) for the claims-based algorithms. CONCLUSIONS: The algorithms evaluated in this study identified a real-world cohort of patients with RRMS without evidence of progressive MS that can be studied in clinical research with confidence.

Characterization and survival of patients with metastatic basal cell carcinoma in the Department of Veterans Affairs: a retrospective electronic health record review.

Available descriptive statistics for patients with metastatic basal cell carcinoma (mBCC) are limited. To describe disease characteristics, treatment patterns, survival outcomes, and prognostic factors of patients with mBCC, we conducted a retrospective review of electronic health records in the Department of Veterans Affairs (VA). The primary outcome was survival. Data were also collected on demographics, comorbidities, medications, and procedures. Median (IQR) age of patients with mBCC (n = 475) was 72.0 (17.0) years; 97.9% of patients were male. Almost two-thirds of patients received no initial therapy for mBCC. Median overall survival was 40.5 months [95% CI (confidence interval) 4.8-140.0], and was shorter in patients with distant metastases (17.1 months; 95% CI 2.8-58.0) than in those with regional metastases (59.4 months; 95% CI 17.6-140.0). Because the VA mBCC population is largely male and elderly, the generalizability of these results in other populations is limited and must be interpreted cautiously. Data from this large cohort add valuable information on a rare and poorly researched disease and refine previously wide estimates of overall survival for mBCC.

Interdisciplinary pharmacometrics linking oseltamivir pharmacology, influenza epidemiology and health economics to inform antiviral use in pandemics.

AIMS: A modular interdisciplinary platform was developed to investigate the economic impact of oseltamivir treatment by dosage regimen under simulated influenza pandemic scenarios. METHODS: The pharmacology module consisted of a pharmacokinetic distribution of oseltamivir carboxylate daily area under the concentration-time curve at steady state (simulated for 75 mg and 150 mg twice daily regimens for 5 days) and a pharmacodynamic distribution of viral shedding duration obtained from phase II influenza inoculation data. The epidemiological module comprised a susceptible, exposed, infected, recovered (SEIR) model to which drug effect on the basic reproductive number (R0 ), a measure of transmissibility, was linked by reduction of viral shedding duration. The number of infected patients per population of 100 000 susceptible individuals was simulated for a series of pandemic scenarios, varying oseltamivir dose, R0 (1.9 vs. 2.7), and drug uptake (25%, 50%, and 80%). The number of infected patients for each scenario was entered into the health economics module, a decision analytic model populated with branch probabilities, disease utility, costs of hospitalized patients developing complications, and case-fatality rates. Change in quality-adjusted life years was determined relative to base case. RESULTS: Oseltamivir 75 mg relative to no treatment reduced the median number of infected patients, increased change in quality-adjusted life years by deaths averted, and was cost-saving under all scenarios; 150 mg relative to 75 mg was not cost effective in low transmissibility scenarios but was cost saving in high transmissibility scenarios. CONCLUSION: This methodological study demonstrates proof of concept that the disciplines of pharmacology, disease epidemiology and health economics can be linked in a single quantitative framework.

Determining Multiple Sclerosis Phenotype from Electronic Medical Records.

BACKGROUND: Multiple sclerosis (MS), a central nervous system disease in which nerve signals are disrupted by scarring and demyelination, is classified into phenotypes depending on the patterns of cognitive or physical impairment progression: relapsing-remitting MS (RRMS), primary-progressive MS (PPMS), secondary-progressive MS (SPMS), or progressive-relapsing MS (PRMS). The phenotype is important in managing the disease and determining appropriate treatment. The ICD-9-CM code 340.0 is uninformative about MS phenotype, which increases the difficulty of studying the effects of phenotype on disease. OBJECTIVE: To identify MS phenotype using natural language processing (NLP) techniques on progress notes and other clinical text in the electronic medical record (EMR). METHODS: Patients with at least 2 ICD-9-CM codes for MS (340.0) from 1999 through 2010 were identified from nationwide EMR data in the Department of Veterans Affairs. Clinical experts were interviewed for possible keywords and phrases denoting MS phenotype in order to develop a data dictionary for NLP. For each patient, NLP was used to search EMR clinical notes, since the first MS diagnosis date for these keywords and phrases. Presence of phenotype-related keywords and phrases were analyzed in context to remove mentions that were negated (e.g., "not relapsing-remitting") or unrelated to MS (e.g., "RR" meaning "respiratory rate"). One thousand mentions of MS phenotype were validated, and all records of 150 patients were reviewed for missed mentions. RESULTS: There were 7,756 MS patients identified by ICD-9-CM code 340.0. MS phenotype was identified for 2,854 (36.8%) patients, with 1,836 (64.3%) of those having just 1 phenotype mentioned in their EMR clinical notes: 1,118 (39.2%) RRMS, 325 (11.4%) PPMS, 374 (13.1%) SPMS, and 19 (0.7%) PRMS. A total of 747 patients (26.2%) had 2 phenotypes, the most common being 459 patients (16.1%) with RRMS and SPMS. A total of 213 patients (7.5%) had 3 phenotypes, and 58 patients (2.0%) had 4 phenotypes mentioned in their EMR clinical notes. Positive predictive value of phenotype identification was 93.8% with sensitivity of 94.0%. CONCLUSIONS: Phenotype was documented for slightly more than one third of MS patients, an important but disappointing finding that sets a limit on studying the effects of phenotype on MS in general. However, for cases where the phenotype was documented, NLP accurately identified the phenotypes. Having multiple phenotypes documented is consistent with disease progression. The most common misidentification was because of ambiguity while clinicians were trying to determine phenotype. This study brings attention to the need for care providers to document MS phenotype more consistently and provides a solution for capturing phenotype from clinical text. DISCLOSURES: This study was funded by Anolinx and F. Hoffman-La Roche. Nelson serves as a consultant for Anolinx. Kamauu is owner of Anolinx, which has received multiple research grants from pharmaceutical and biotechnology companies. LaFleur has received a Novartis grant for ongoing work. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the U.S. government. Study concept and design were contributed by Butler, LaFleur, Kamauu, DuVall, and Nelson. DuVall collected the data, and interpretation was performed by Nelson, DuVall, and Kamauu, along with Butler, LaFleur, and Knippenberg. The manuscript was written primarily by Nelson, along with Knippenberg and assisted by the other authors, and revised by Knippenberg, Nelson, and DuVall, along with the other authors.

Characteristics and use of treatment modalities of patients with binge-eating disorder in the Department of Veterans Affairs.

OBJECTIVE: In 2013 binge-eating disorder (BED) was recognized as a formal diagnosis, but was historically included under the diagnosis code for eating disorder not otherwise specified (EDNOS). This study compared the characteristics and use of treatment modalities in BED patients to those with EDNOS without BED (EDNOS-only) and to matched-patients with no eating disorders (NED). METHODS: Patients were identified for this study from electronic health records in the Department of Veterans Affairs from 2000 to 2011. Patients with BED were identified using natural language processing and patients with EDNOS-only were identified by ICD-9 code (307.50). First diagnosis defined index date for these groups. NED patients were frequency matched to BED patients up to 4:1, as available, on age, sex, BMI, depression, and index month encounter. Baseline characteristics and use of treatment modalities during the post-index year were compared using t-tests or chi-square tests. RESULTS: There were 593 BED, 1354 EDNOS-only, and 1895 matched-NED patients identified. Only 68 patients with BED had an EDNOS diagnosis. BED patients were younger (48.7 vs. 49.8years, p=0.04), more were male (72.2% vs. 62.8%, p<0.001) and obese (BMI 40.2 vs. 37.0, p<0.001) than EDNOS-only patients. In the follow-up period fewer BED (68.0%) than EDNOS-only patients (87.6%, p<0.001), but more BED than NED patients (51.9%, p<0.001) used at least one treatment modality. DISCUSSION: The characteristics of BED patients were different from those with EDNOS-only and NED as was their use of treatment modalities. These differences highlight the need for a separate identifier of BED.

Multiple Sclerosis and Risk of Infection-Related Hospitalization and Death in US Veterans.

BACKGROUND: This study estimated the risk of infection-related hospitalizations and death in patients with and without multiple sclerosis (MS). METHODS: We identified adults with MS in the US Department of Veterans Affairs (VA) system between 1999 and 2010. Each veteran with MS was matched, on age and sex, with up to four veterans without MS. Multivariable Cox proportional hazards regression models were performed to assess the influence of MS on the development of serious and fatal infections. RESULTS: The cohort included 7743 veterans with MS and 30,972 veterans without MS. Mean (SD) age was 53.8 (13.3) years, and 80.8% were male. The incidence per 1000 person-years of overall serious infections was 19.2 (95% confidence interval [CI], 17.6-20.8) for those with MS and 10.3 (95% CI, 9.8-10.9) for those without MS. Fatal infection incidence rates were 1.2 (95% CI, 0.8-1.7) for patients with MS and 0.5 (95% CI, 0.3-0.6) for patients without MS. Regression models showed that veterans with MS were at greater risk for overall serious (hazard ratio [HR] = 1.52, P < .01) and fatal (HR = 1.85, P = .03) infections and serious respiratory (HR = 1.31, P = .01), urinary tract (HR = 4.44, P < .01), and sepsis-related infections (HR = 2.56, P < .01). CONCLUSIONS: This study provides evidence that VA patients with MS are more likely than those without MS to be hospitalized and die of infection.

Healthcare costs and resource utilization of patients with binge-eating disorder and eating disorder not otherwise specified in the Department of Veterans Affairs.

OBJECTIVE: The objective of this study was to compare the one-year healthcare costs and utilization of patients with binge-eating disorder (BED) to patients with eating disorder not otherwise specified without BED (EDNOS-only) and to matched patients without an eating disorder (NED). METHODS: A natural language processing (NLP) algorithm identified adults with BED from clinical notes in the Department of Veterans Affairs (VA) electronic health record database from 2000 to 2011. Patients with EDNOS-only were identified using ICD-9 code (307.50) and those with NLP-identified BED were excluded. First diagnosis date defined the index date for both groups. Patients with NED were randomly matched 4:1, as available, to patients with BED on age, sex, BMI, depression diagnosis, and index month. Patients with cost data (2005-2011) were included. Total healthcare, inpatient, outpatient, and pharmacy costs were examined. Generalized linear models were used to compare total one-year healthcare costs while adjusting for baseline patient characteristics. RESULTS: There were 257 BED, 743 EDNOS-only, and 823 matched NED patients identified. The mean (SD) total unadjusted one-year costs, in 2011 US dollars, were $33,716 ($38,928) for BED, $37,052 ($40,719) for EDNOS-only, and $19,548 ($35,780) for NED patients. When adjusting for patient characteristics, BED patients had one-year total healthcare costs $5,589 higher than EDNOS-only (p = 0.06) and $18,152 higher than matched NED patients (p < 0.001). DISCUSSION: This study is the first to use NLP to identify BED patients and quantify their healthcare costs and utilization. Patients with BED had similar one-year total healthcare costs to EDNOS-only patients, but significantly higher costs than patients with NED.

Automated identification of patients with a diagnosis of binge eating disorder from narrative electronic health records.

Binge eating disorder (BED) does not have an International Classification of Diseases, 9th or 10th edition code, but is included under 'eating disorder not otherwise specified' (EDNOS). This historical cohort study identified patients with clinician-diagnosed BED from electronic health records (EHR) in the Department of Veterans Affairs between 2000 and 2011 using natural language processing (NLP) and compared their characteristics to patients identified by EDNOS diagnosis codes. NLP identified 1487 BED patients with classification accuracy of 91.8% and sensitivity of 96.2% compared to human review. After applying study inclusion criteria, 525 patients had NLP-identified BED only, 1354 had EDNOS only, and 68 had both BED and EDNOS. Patient characteristics were similar between the groups. This is the first study to use NLP as a method to identify BED patients from EHR data and will allow further epidemiological study of patients with BED in systems with adequate clinical notes.

Conformity between protocol eligibility criteria for electronic patient identification: a comparison of clinical trials.

Clinical trial eligibility criteria define the target patient population for research studies. We assessed the eligibility criteria from 40 different protocols for Type II Diabetes Mellitus and depression (20 protocols each), to determine the extent to which protocol eligibility criteria were similar at three levels (test, test-value, and test-value-time clause). This was done to determine criteria that could be standardized to aid in identification of eligible patients from electronic health records. It was found that Type II Diabetes Mellitus had 36.9% similar and depression protocols had 53.1% similar at the test-value-clause level. This study demonstrates the need for more standardization of study protocol criteria as well as the associated query definitions to be run against the electronic healthcare data. Standardizing criteria based on the similar eligibility criteria between protocols will aid in patient recruitment by being able to reuse criteria and minimizing the time and money it takes to recruit patients.

Using Java to generate globally unique identifiers for DICOM objects.

Digital imaging and communication in medicine (DICOM) specifies that all DICOM objects have globally unique identifiers (UIDs). Creating these UIDs can be a difficult task due to the variety of techniques in use and the requirement to ensure global uniqueness. We present a simple technique of combining a root organization identifier, assigned descriptive identifiers, and JAVA generated unique identifiers to construct DICOM compliant UIDs.

Informatics in radiology: IHE teaching file and clinical trial export integration profile: functional examples.

The digital revolution in radiology introduced the need for electronic export of medical images. However, the current export process is complicated and time consuming. In response to this continued difficulty, the Integrating the Healthcare Enterprise (IHE) initiative published the Teaching File and Clinical Trial Export (TCE) integration profile. The IHE TCE profile describes a method for using existing standards to simplify the export of key medical images for education, research, and publication. This article reviews the authors' experience in implementing the TCE profile in the following three processes: (a) the retrieval of images for a typical teaching file application within a TCE-compliant picture archiving and communication system (PACS); (b) the export of images, independent of TCE compliance of the PACS, to a typical teaching file application; and (c) the TCE-compliant transfer of images for publication. These examples demonstrate methods with which the TCE profile can be implemented to ease the burden of collecting key medical images from the PACS.

Using applet-servlet communication for optimizing window, level and crop for DICOM to JPEG conversion.

In the creation of interesting radiological cases in a digital teaching file, it is necessary to adjust the window and level settings of an image to effectively display the educational focus. The web-based applet described in this paper presents an effective solution for real-time window and level adjustments without leaving the picture archiving and communications system workstation. Optimized images are created, as user-defined parameters are passed between the applet and a servlet on the Health Insurance Portability and Accountability Act-compliant teaching file server.

Translating the IHE Teaching File and Clinical Trial Export (TCE) profile document templates into functional DICOM structured report objects.

The Integrating the Healthcare Enterprise (IHE) Teaching File and Clinical Trial Export (TCE) integration profile describes a standard workflow for exporting key images from an image manager/archive to a teaching file, clinical trial, or electronic publication application. Two specific digital imaging and communication in medicine (DICOM) structured reports (SR) reference the key images and contain associated case information. This paper presents step-by-step instructions for translating the TCE document templates into functional and complete DICOM SR objects. Others will benefit from these instructions in developing TCE compliant applications.

Building knowledge: a system for batch loading biomedical terminologies.

Ontologies provide knowledge that supports health care applications. Biomedical ontologies must include a vast number of both standard and proprietary terminology concepts. Conventional loading methods are labor-intensive and inefficient. Thus, a system was developed to simultaneously load a large number of terminology concepts into a biomedical ontology. Such a robust ontology can support a variety of health care applications.

Requirements for an efficient biomedical terminology browser.

Biomedical ontologies provide knowledge in support of health care applications. Knowledge engineers require tools to develop and manage a rich biomedical ontology. An efficient terminology browser is necessary for knowledge engineers to develop and manage a rich biomedical ontology that supports a variety of health care applications.

Informatics in radiology (infoRAD): Vendor-neutral case input into a server-based digital teaching file system.

Although digital teaching files are important to radiology education, there are no current satisfactory solutions for export of Digital Imaging and Communications in Medicine (DICOM) images from picture archiving and communication systems (PACS) in desktop publishing format. A vendor-neutral digital teaching file, the Radiology Interesting Case Server (RadICS), offers an efficient tool for harvesting interesting cases from PACS without requiring modifications of the PACS configurations. Radiologists push imaging studies from PACS to RadICS via the standard DICOM Send process, and the RadICS server automatically converts the DICOM images into the Joint Photographic Experts Group format, a common desktop publishing format. They can then select key images and create an interesting case series at the PACS workstation. RadICS was tested successfully against multiple unmodified commercial PACS. Using RadICS, radiologists are able to harvest and author interesting cases at the point of clinical interpretation with minimal disruption in clinical work flow.

Academic podcasting: quality media delivery.

A video podcast of the CME-approved University of Utah Department of Biomedical Informatics seminar was created in order to address issues with streaming video quality, take advantage of popular web-based syndication methods, and make the files available for convenient, subscription-based download. An RSS feed, which is automatically generated, contains links to the media files and allows viewers to easily subscribe to the weekly seminars in a format that guarantees consistent video quality.