Effects of a Patient Portal Intervention to Address Diabetes Care Gaps: Protocol for a Pragmatic Randomized Controlled Trial.

BACKGROUND: Despite the potential to significantly reduce complications, many patients do not consistently receive diabetes preventive care. Our research team recently applied user-centered design sprint methodology to develop a patient portal intervention empowering patients to address selected diabetes care gaps (eg, no diabetes eye examination in last 12 months). OBJECTIVE: This study aims to evaluate the effect of our novel diabetes care gap intervention on completion of selected evidence-based diabetes preventive care services and secondary outcomes. METHODS: We are conducting a pragmatic randomized controlled trial of the effect of the intervention on diabetes care gaps. Adult patients with diabetes mellitus (DM) are recruited from primary care clinics affiliated with Vanderbilt University Medical Center. Participants are eligible if they have type 1 or 2 DM, can read in English, are aged 18-75 years, have a current patient portal account, and have reliable access to a mobile device with internet access. We exclude patients with medical conditions that prevent them from using a mobile device, severe difficulty seeing, pregnant women or women who plan to become pregnant during the study period, and patients on dialysis. Participants will be randomly assigned to the intervention or usual care. The primary outcome measure will be the number of diabetes care gaps among 4 DM preventive care services (diabetes eye examination, pneumococcal vaccination, hemoglobin A1c, and urine microalbumin) at 12 months after randomization. Secondary outcomes will include diabetes self-efficacy, confidence managing diabetes in general, understanding of diabetes preventive care, diabetes distress, patient portal satisfaction, and patient-initiated orders at baseline, 3 months, 6 months, and 12 months after randomization. An ordinal logistic regression model will be used to quantify the effect of the intervention on the number of diabetes care gaps at the 12-month follow-up. For dichotomous secondary outcomes, a logistic regression model will be used with random effects for the clinic and provider variables as needed. For continuous secondary outcomes, a regression model will be used. RESULTS: This study is ongoing. Recruitment was closed in February 2022; a total of 433 patients were randomized. Of those randomized, most (n=288, 66.5%) were non-Hispanic White, 33.5% (n=145) were racial or ethnic minorities, 33.9% (n=147) were aged 65 years or older, and 30.7% (n=133) indicated limited health literacy. CONCLUSIONS: The study directly tests the hypothesis that a patient portal intervention-alerting patients about selected diabetes care gaps, fostering understanding of their significance, and allowing patients to initiate care-will reduce diabetes care gaps compared with usual care. The insights gained from this study may have broad implications for developing future interventions to address various care gaps, such as gaps in cancer screening, and contribute to the development of effective, scalable, and sustainable approaches to engage patients in chronic disease management and prevention. TRIAL REGISTRATION: ClinicalTrials.gov NCT04894903; https://classic.clinicaltrials.gov/ct2/show/NCT04894903. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/56123.

Implementation of a Quality Metrics Competition to Improve Panel Management Among Internal Medicine Residents.

Background Panel management is essential for residents to learn, yet challenging to teach. To our knowledge, prior literature has not described curricula utilizing a financially incentivized competition to improve resident primary care metrics. Objective We developed a panel management curriculum, including a financially incentivized quality competition, to improve resident performance on quality metrics. Methods We developed a cancer screening and diabetes metric quality competition for internal medicine residents at Vanderbilt University Medical Center for their primary care clinics for the 2020-2021 (pilot) and 2021-2022 academic years. Residents received several educational tools, including a 1-hour introduction to the health maintenance dashboard within the electronic medical record (EMR) and instructions on how to access the quality dashboard outside the EMR, and were encouraged to discuss panel management with preceptors. Chief residents distributed measures to trainees 3 times annually, so residents were aware of their competition ranking. Residents' composite metrics at year end were compared to baseline to determine top performers. The top 15 performers received $100 gift cards as incentives. We also assessed the curriculum's impact on the residents' metrics in aggregate. Results At curriculum completion, residents (n=100) demonstrated an average improvement of 1.9% from baseline composite metrics for the percent of patients receiving screening. In aggregate, residents improved in every measure except HbA1c testing. Breast cancer screening had the largest improvement from 69.5% (1518 of 2183) to 75.6% (1646 of 2178) of all patients receiving recommended screening. Conclusions The curriculum resulted in more patients receiving recommended cancer and diabetes screenings.

Empowering patients to address diabetes care gaps: formative usability testing of a novel patient portal intervention.

Objective: The aim of this study was to design and assess the formative usability of a novel patient portal intervention designed to empower patients with diabetes to initiate orders for diabetes-related monitoring and preventive services. Materials and Methods: We used a user-centered Design Sprint methodology to create our intervention prototype and assess its usability with 3 rounds of iterative testing. Participants (5/round) were presented with the prototype and asked to perform common, standardized tasks using think-aloud procedures. A facilitator rated task performance using a scale: (1) completed with ease, (2) completed with difficulty, and (3) failed. Participants completed the System Usability Scale (SUS) scored 0-worst to 100-best. All testing occurred remotely via Zoom. Results: We identified 3 main categories of usability issues: distrust about the automated system, content concerns, and layout difficulties. Changes included improving clarity about the ordering process and simplifying language; however, design constraints inherent to the electronic health record system limited our ability to respond to all usability issues (eg, could not modify fixed elements in layout). Percent of tasks completed with ease across each round were 67%, 60%, and 80%, respectively. Average SUS scores were 87, 74, and 93, respectively. Across rounds, participants found the intervention valuable and appreciated the concept of patient-initiated ordering. Conclusions: Through iterative user-centered design and testing, we improved the usability of the patient portal intervention. A tool that empowers patients to initiate orders for disease-specific services as part of their existing patient portal account has potential to enhance the completion of recommended health services and improve clinical outcomes.

ACR Appropriateness Criteria® Lung Cancer Screening: 2022 Update.

Lung cancer remains the leading cause of cancer-related mortality for men and women in the United States. Screening for lung cancer with annual low-dose CT is saving lives, and the continued implementation of lung screening can save many more. In 2015, the CMS began covering annual lung screening for those who qualified based on the original United States Preventive Services Task Force (USPSTF) lung screening criteria, which included patients 55 to 77 year of age with a 30 pack-year history of smoking, who were either currently using tobacco or who had smoked within the previous 15 years. In 2021, the USPSTF issued new screening guidelines, decreasing the age of eligibility to 80 years of age and pack-years to 20. Lung screening remains controversial for those who do not meet the updated USPSTF criteria, but who have additional risk factors for the development of lung cancer. The American College of Radiology Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed annually by a multidisciplinary expert panel. The guideline development and revision process support the systematic analysis of the medical literature from peer reviewed journals. Established methodology principles such as Grading of Recommendations Assessment, Development, and Evaluation or GRADE are adapted to evaluate the evidence. The RAND/UCLA Appropriateness Method User Manual provides the methodology to determine the appropriateness of imaging and treatment procedures for specific clinical scenarios. In those instances where peer reviewed literature is lacking or equivocal, experts may be the primary evidentiary source available to formulate a recommendation.

Validating drug repurposing signals using electronic health records: a case study of metformin associated with reduced cancer mortality.

OBJECTIVES: Drug repurposing, which finds new indications for existing drugs, has received great attention recently. The goal of our work is to assess the feasibility of using electronic health records (EHRs) and automated informatics methods to efficiently validate a recent drug repurposing association of metformin with reduced cancer mortality. METHODS: By linking two large EHRs from Vanderbilt University Medical Center and Mayo Clinic to their tumor registries, we constructed a cohort including 32,415 adults with a cancer diagnosis at Vanderbilt and 79,258 cancer patients at Mayo from 1995 to 2010. Using automated informatics methods, we further identified type 2 diabetes patients within the cancer cohort and determined their drug exposure information, as well as other covariates such as smoking status. We then estimated HRs for all-cause mortality and their associated 95% CIs using stratified Cox proportional hazard models. HRs were estimated according to metformin exposure, adjusted for age at diagnosis, sex, race, body mass index, tobacco use, insulin use, cancer type, and non-cancer Charlson comorbidity index. RESULTS: Among all Vanderbilt cancer patients, metformin was associated with a 22% decrease in overall mortality compared to other oral hypoglycemic medications (HR 0.78; 95% CI 0.69 to 0.88) and with a 39% decrease compared to type 2 diabetes patients on insulin only (HR 0.61; 95% CI 0.50 to 0.73). Diabetic patients on metformin also had a 23% improved survival compared with non-diabetic patients (HR 0.77; 95% CI 0.71 to 0.85). These associations were replicated using the Mayo Clinic EHR data. Many site-specific cancers including breast, colorectal, lung, and prostate demonstrated reduced mortality with metformin use in at least one EHR. CONCLUSIONS: EHR data suggested that the use of metformin was associated with decreased mortality after a cancer diagnosis compared with diabetic and non-diabetic cancer patients not on metformin, indicating its potential as a chemotherapeutic regimen. This study serves as a model for robust and inexpensive validation studies for drug repurposing signals using EHR data.

Type 2 diabetes and mammographic breast density among underserved women.

PURPOSE: We conducted a study of women recruited at Meharry Medical College, a historically black medical school, to investigate the relationship between diabetes and mammographic breast density. METHODS: A total of 476 women completed in-person interviews, body measurements, and full-field digital mammograms on a Hologic mammography unit from December 2011 to February 2014. Average percent breast density for the left and right breasts combined was estimated using Quantra, an automated algorithm for volumetric assessment of breast tissue. The prevalence of type 2 diabetes was determined by self-report. RESULTS: After adjustment for confounding variables, the mean percent breast density among premenopausal women with type 2 diabetes [[Formula: see text] 13.8 %, 95 % confidence interval (CI) 11.6-15.9] was nonsignificantly lower than that of women without type 2 diabetes ([Formula: see text] 15.9 %, 95 % CI 15.0-16.8) (p = 0.07); however, there was no association among postmenopausal women. The effect of type 2 diabetes in severely obese women (BMI ≥ 35) appeared to differ by menopausal status with a reduction in mean percent breast density in premenopausal women, but an increase in mean percent breast density in postmenopausal women which could have been due to chance. CONCLUSIONS: Confirmation of our findings in larger studies may assist in clarifying the role of the insulin signaling breast cancer pathway in women with high breast density.

Adverse drug events during AKI and its recovery.

BACKGROUND AND OBJECTIVES: The impact of AKI on adverse drug events and therapeutic failures and the medication errors leading to these events have not been well described. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: A single-center observational study of 396 hospitalized patients with a minimum 0.5 mg/dl change in serum creatinine who were prescribed a nephrotoxic or renally eliminated medication was conducted. The population was stratified into two groups by the direction of their initial serum creatinine change: AKI and AKI recovery. Adverse drug events, potential adverse drug events, therapeutic failures, and potential therapeutic failures for 148 drugs and 46 outcomes were retrospectively measured. Events were classified for preventability and severity by expert adjudication. Multivariable analysis identified medication classes predisposing AKI patients to adverse drug events. RESULTS: Forty-three percent of patients experienced a potential adverse drug event, adverse drug event, therapeutic failure, or potential therapeutic failure; 66% of study events were preventable. Failure to adjust for kidney function (63%) and use of nephrotoxic medications during AKI (28%) were the most common potential adverse drug events. Worsening AKI and hypotension were the most common preventable adverse drug events. Most adverse drug events were considered serious (63%) or life-threatening (31%), with one fatal adverse drug event. Among AKI patients, administration of angiotensin-converting enzyme inhibitors/angiotensin receptor blockers, antibiotics, and antithrombotics was most strongly associated with the development of an adverse drug event or potential adverse drug event. CONCLUSIONS: Adverse drug events and potential therapeutic failures are common and frequently severe in patients with AKI exposed to nephrotoxic or renally eliminated medications.

The effect of reducing maximum shift lengths to 16 hours on internal medicine interns' educational opportunities.

PURPOSE: To evaluate educational experiences of internal medicine interns before and after maximum shift lengths were decreased from 30 hours to 16 hours. METHOD: The authors compared educational experiences of internal medicine interns at Vanderbilt University Medical Center before (2010; 47 interns) and after (2011; 50 interns) duty hours restrictions were implemented in July 2011. The authors compared number of inpatient encounters, breadth of concepts in notes, exposure to five common presenting problems, procedural experience, and attendance at teaching conferences. RESULTS: Following the duty hours restrictions, interns cared for more unique patients (mean 118 versus 140 patients per intern, P = .005) and wrote more history and physicals (mean 73 versus 88, P = .005). Documentation included more total concepts after the 16-hour maximum shift implementation, with a 14% increase for history and physicals (338 versus 387, P < .001) and a 10% increase for progress notes (316 versus 349, P < .001). There was no difference in the median number of selected procedures performed (6 versus 6, P = 0.94). Attendance was higher at the weekly chief resident conference (60% versus 68% of expected attendees, P < .001) but unchanged at morning report conferences (79% versus 78%, P = .49). CONCLUSIONS: Intern clinical exposure did not decrease after implementation of the 16-hour shift length restriction. In fact, interns saw more patients, produced more detailed notes, and attended more conferences following duty hours restrictions.

Extracting epidemiologic exposure and outcome terms from literature using machine learning approaches.

Much epidemiologic information resides in literature, which is not in a computable format. To extract information and build knowledge bases of epidemiologic studies, we developed a system to extract noun phrases about epidemiologic exposures and outcomes. The system consists of two components: a natural language processing (NLP) engine; a machine learning (ML) based classifier. Four ML algorithms were applied and compared over different feature sets. To evaluate the performance of the system, we manually constructed an annotated dataset. The system achieved the highest F-measure of 82.0% for extracting exposure terms, and 70% for extracting outcome terms.

Real-time pharmacy surveillance and clinical decision support to reduce adverse drug events in acute kidney injury: a randomized, controlled trial.

OBJECTIVES: Clinical decision support (CDS), such as computerized alerts, improves prescribing in the setting of acute kidney injury (AKI), but considerable opportunity remains to improve patient safety. The authors sought to determine whether pharmacy surveillance of AKI patients could detect and prevent medication errors that are not corrected by automated interventions. METHODS: The authors conducted a randomized clinical trial among 396 patients admitted to an academic, tertiary care hospital between June 1, 2010 and August 31, 2010 with an acute 0.5 mg/dl change in serum creatinine over 48 hours and a nephrotoxic or renally cleared medication order. Patients randomly assigned to the intervention group received surveillance from a clinical pharmacist using a web-based surveillance tool to monitor drug prescribing and kidney function trends. CDS alerting and standard pharmacy services were active in both study arms. Outcome measures included blinded adjudication of potential adverse drug events (pADEs), adverse drug events (ADEs) and time to provider modification or discontinuation of targeted nephrotoxic or renally cleared medications. RESULTS: Potential ADEs or ADEs occurred for 104 (8.0%) of control and 99 (7.1%) of intervention patient-medication pairs (p=0.4). Additionally, the time to provider modification or discontinuation of targeted nephrotoxic or renally cleared medications did not differ between control and intervention patients (33.4 hrs vs. 30.3 hrs, p=0.3). CONCLUSIONS: Pharmacy surveillance had no incremental benefit over previously implemented CDS alerts.

A study of transportability of an existing smoking status detection module across institutions.

Electronic Medical Records (EMRs) are valuable resources for clinical observational studies. Smoking status of a patient is one of the key factors for many diseases, but it is often embedded in narrative text. Natural language processing (NLP) systems have been developed for this specific task, such as the smoking status detection module in the clinical Text Analysis and Knowledge Extraction System (cTAKES). This study examined transportability of the smoking module in cTAKES on the Vanderbilt University Hospital's EMR data. Our evaluation demonstrated that modest effort of change is necessary to achieve desirable performance. We modified the system by filtering notes, annotating new data for training the machine learning classifier, and adding rules to the rule-based classifiers. Our results showed that the customized module achieved significantly higher F-measures at all levels of classification (i.e., sentence, document, patient) compared to the direct application of the cTAKES module to the Vanderbilt data.

Natural language processing improves identification of colorectal cancer testing in the electronic medical record.

BACKGROUND: Difficulty identifying patients in need of colorectal cancer (CRC) screening contributes to low screening rates. OBJECTIVE: To use Electronic Health Record (EHR) data to identify patients with prior CRC testing. DESIGN: A clinical natural language processing (NLP) system was modified to identify 4 CRC tests (colonoscopy, flexible sigmoidoscopy, fecal occult blood testing, and double contrast barium enema) within electronic clinical documentation. Text phrases in clinical notes referencing CRC tests were interpreted by the system to determine whether testing was planned or completed and to estimate the date of completed tests. SETTING: Large academic medical center. PATIENTS: 200 patients ≥ 50 years old who had completed ≥ 2 non-acute primary care visits within a 1-year period. MEASURES: Recall and precision of the NLP system, billing records, and human chart review were compared to a reference standard of human review of all available information sources. RESULTS: For identification of all CRC tests, recall and precision were as follows: NLP system (recall 93%, precision 94%), chart review (74%, 98%), and billing records review (44%, 83%). Recall and precision for identification of patients in need of screening were: NLP system (recall 95%, precision 88%), chart review (99%, 82%), and billing records (99%, 67%). LIMITATIONS: Small sample size and requirement for a robust EHR. CONCLUSIONS: Applying NLP to EHR records detected more CRC tests than either manual chart review or billing records review alone. NLP had better precision but marginally lower recall to identify patients who were due for CRC screening than billing record review.

Closing the quality gap: revisiting the state of the science (vol. 3: quality improvement interventions to address health disparities).

OBJECTIVE: This review evaluates the effectiveness of quality improvement (QI) strategies in reducing disparities in health and health care. DATA SOURCES: We identified papers published in English between 1983 and 2011 from the MEDLINE® database, the Cumulative Index of Nursing and Allied Health Literature (CINAHL), Web of Science Social Science Index, and PsycINFO. REVIEW METHODS: All abstracts and full-text articles were dually reviewed. Studies were eligible if they reported data on effectiveness of QI interventions on processes or health outcomes in the United States such that the impact on a health disparity could be measured. The review focused on the following clinical conditions: breast cancer, colorectal cancer, diabetes, heart failure, hypertension, coronary artery disease, asthma, major depressive disorder, cystic fibrosis, pneumonia, pregnancy, and end-stage renal disease. It assessed health disparities associated with race or ethnicity, socioeconomic status, insurance status, sexual orientation, health literacy/numeracy, and language barrier. We evaluated the risk of bias of individual studies and the overall strength of the body of evidence based on risk of bias, consistency, directness, and precision. RESULTS: Nineteen papers, representing 14 primary research studies, met criteria for inclusion. All but one of the studies incorporated multiple components into their QI approach. Patient education was part of most interventions (12 of 14), although the specific approach differed substantially across the studies. Ten of the studies incorporated self-management; this would include, for example, teaching individuals with diabetes to check their blood sugar regularly. Most (8 of 14) included some sort of provider education, which may have focused on the clinical issue or on raising awareness about disparities affecting the target population. Studies evaluated the effect of these strategies on disparities in the prevention or treatment of breast or colorectal cancer, cardiovascular disease, depression, or diabetes. Overall, QI interventions were not shown to reduce disparities. Most studies have focused on racial or ethnic disparities, with some targeted interventions demonstrating greater effect in racial minorities--specifically, supporting individuals in tracking their blood pressure at home to reduce blood pressure and collaborative care to improve depression care. In one study, the effect of a language-concordant breast cancer screening intervention was helpful in promoting mammography in Spanish-speaking women. For some depression care outcomes, the collaborative care model was more effective in less-educated individuals than in those with more education and in women than in men. CONCLUSIONS: The literature on QI interventions generally and their ability to improve health and health care is large. Whether those interventions are effective at reducing disparities remains unclear. This report should not be construed to assess the general effectiveness of QI in the health care setting; rather, QI has not been shown specifically to reduce known disparities in health care or health outcomes. In a few instances, some increased effect is seen in disadvantaged populations; these studies should be replicated and the interventions studied further as having potential to address disparities.

Extracting and integrating data from entire electronic health records for detecting colorectal cancer cases.

Identification of a cohort of patients with specific diseases is an important step for clinical research that is based on electronic health records (EHRs). Informatics approaches combining structured EHR data, such as billing records, with narrative text data have demonstrated utility for such tasks. This paper describes an algorithm combining machine learning and natural language processing to detect patients with colorectal cancer (CRC) from entire EHRs at Vanderbilt University Hospital. We developed a general case detection method that consists of two steps: 1) extraction of positive CRC concepts from all clinical notes (document-level concept identification); and 2) determination of CRC cases using aggregated information from both clinical narratives and structured billing data (patient-level case determination). For each step, we compared performance of rule-based and machine-learning-based approaches. Using a manually reviewed data set containing 300 possible CRC patients (150 for training and 150 for testing), we showed that our method achieved F-measures of 0.996 for document level concept identification, and 0.93 for patient level case detection.

Patient numeracy, perceptions of provider communication, and colorectal cancer screening utilization.

Patients with poor numeracy skills may have difficulty participating in shared-decision making, affecting their utilization of colorectal cancer (CRC) screening. We explored the relationship between numeracy, provider communication, and CRC screening. Data were from the 2007 National Cancer Institute Health Information Trends Survey. Individuals age 50 years or older responded via mail or phone to items measuring numeracy, perceptions of provider communication quality, and CRC screening. After accounting for national sampling weights, multivariate logistic regression models examined the association between these factors. A total of 1,436 subjects responded to an objective numeracy item via mail, and 3,286 responded to a subjective numeracy item via mail or phone; 22.6% had low objective numeracy, and 39.4% had low subjective numeracy. Low subjective numeracy was associated with a lower likelihood of perceiving high quality provider communication (OR 0.63-0.73), but for low objective numeracy, the opposite was observed (OR 1.51-1.64). Low objective or subjective numeracy was associated with less CRC screening. There was significant interaction between subjective numeracy, perceptions of provider communication, and CRC screening. Patient numeracy is associated with perceptions of provider communication quality. For individuals with low subjective numeracy, perceiving high quality communication offset the association between low numeracy and underutilization of CRC screening.

Extracting timing and status descriptors for colonoscopy testing from electronic medical records.

Colorectal cancer (CRC) screening rates are low despite confirmed benefits. The authors investigated the use of natural language processing (NLP) to identify previous colonoscopy screening in electronic records from a random sample of 200 patients at least 50 years old. The authors developed algorithms to recognize temporal expressions and 'status indicators', such as 'patient refused', or 'test scheduled'. The new methods were added to the existing KnowledgeMap concept identifier system, and the resulting system was used to parse electronic medical records (EMR) to detect completed colonoscopies. Using as the 'gold standard' expert physicians' manual review of EMR notes, the system identified timing references with a recall of 0.91 and precision of 0.95, colonoscopy status indicators with a recall of 0.82 and precision of 0.95, and references to actually completed colonoscopies with recall of 0.93 and precision of 0.95. The system was superior to using colonoscopy billing codes alone. Health services researchers and clinicians may find NLP a useful adjunct to traditional methods to detect CRC screening status. Further investigations must validate extension of NLP approaches for other types of CRC screening applications.

Association of COMT haplotypes and breast cancer risk in caucasian women.

Catechol-O-methyl transferase (COMT) is an important estrogen-metabolizing enzyme, and common genetic variants in this gene could affect breast cancer risk. We conducted a large population-based case control study in Massachusetts, New Hampshire, and Wisconsin to examine six strategically selected COMT haplotype-tagging (ht) single nucleotide polymorphism (SNPs), including the val158met polymorphism (rs4680), in relation to breast cancer risk. Analyses were based on 1,655 Caucasian women with invasive breast cancer and 1,470 Caucasian controls. None of the six individual SNPs were associated with breast cancer risk. The global test for haplotype associations was nonsignificant (p-value=0.097), although two uncommon haplotypes present in 6% of the study population showed statistically significant inverse associations with risk. These results suggest that genetic variation in COMT has no significant association with breast cancer risk among Caucasian women.

Mining Biomedical Literature for Terms related to Epidemiologic Exposures.

Epidemiologic studies contribute greatly to evidence-based medicine by identifying risk factors for diseases and determining optimal treatments for clinical practice. However, there is very limited effort on automatic extraction of knowledge from epidemiologic articles, such as exposures, outcomes, and their relations. In this initial study, we developed a system that consists of a natural language processing (NLP) engine and a rule-based classifier, to automatically extract exposure-related terms from titles of epidemiologic articles. The evaluation using 450 titles annotated by an epidemiologist showed the highest F-measure of 0.646 (Precision 0.610 and Recall 0.688) using in-exact matching, which indicated the feasibility of automated methods on mining epidemiologic literature. Further analysis of terms related to epidemiologic exposures suggested that although UMLS would have reasonable coverage, more appropriate semantic classifications of epidemiologic exposures would be required.

Evaluation of a method to identify and categorize section headers in clinical documents.

OBJECTIVE: Clinical notes, typically written in natural language, often contain substructure that divides them into sections, such as "History of Present Illness" or "Family Medical History." The authors designed and evaluated an algorithm ("SecTag") to identify both labeled and unlabeled (implied) note section headers in "history and physical examination" documents ("H&P notes"). DESIGN: The SecTag algorithm uses a combination of natural language processing techniques, word variant recognition with spelling correction, terminology-based rules, and naive Bayesian scoring methods to identify note section headers. Eleven physicians evaluated SecTag's performance on 319 randomly chosen H&P notes. MEASUREMENTS: The primary outcomes were the algorithm's recall and precision in identifying all document sections and a predefined list of twenty-nine major sections. A secondary outcome was to evaluate the algorithm's ability to recognize the correct start and end boundaries of identified sections. RESULTS: The SecTag algorithm identified 16,036 total sections and 7,858 major sections. Physician evaluators classified 15,329 as true positives and identified 160 sections omitted by SecTag. The recall and precision of the SecTag algorithm were 99.0 and 95.6% for all sections, 98.6 and 96.2% for major sections, and 96.6 and 86.8% for unlabeled sections. The algorithm determined the correct starting and ending text boundaries for 94.8% of labeled sections and 85.9% of unlabeled sections. CONCLUSIONS: The SecTag algorithm accurately identified both labeled and unlabeled sections in history and physical documents. This type of algorithm may assist in natural language processing applications, such as clinical decision support systems or competency assessment for medical trainees.