Electronic Health Record-Based Nudge Intervention and Axillary Surgery in Older Women With Breast Cancer: A Nonrandomized Controlled Trial.

Importance: Choosing Wisely recommendations advocate against routine use of axillary staging in older women with early-stage, clinically node-negative (cN0), hormone receptor-positive (HR+), and HER2-negative breast cancer. However, rates of sentinel lymph node biopsy (SLNB) in this population remain persistently high. Objective: To evaluate whether an electronic health record (EHR)-based nudge intervention targeting surgeons in their first outpatient visit with patients meeting Choosing Wisely criteria decreases rates of SLNB. Design, Setting, and Participants: This nonrandomized controlled trial was a hybrid type 1 effectiveness-implementation study with subsequent postintervention semistructured interviews and lasted from October 2021 to October 2023. Data came from EHRs at 8 outpatient clinics within an integrated health care system; participants included 7 breast surgical oncologists. Data were collected for female patients meeting Choosing Wisely criteria for omission of SLNB (aged ≥70 years with cT1 and cT2, cN0, HR+/HER2- breast cancer). The study included a 12-month preintervention control period; baseline surveys assessing perceived acceptability, appropriateness, and feasibility of the designed intervention; and a 12-month intervention period. Intervention: A column nudge was embedded into the surgeon's schedule in the EHR identifying patients meeting Choosing Wisely criteria for potential SLNB omission. Main Outcomes and Measures: The primary outcome was rate of SLNB following nudge deployment into the EHR. Results: Similar baseline demographic and tumor characteristics were observed before (control period, n = 194) and after (intervention period, n = 193) nudge deployment. Patients in both the control and intervention period had a median (IQR) age of 75 (72-79) years. Compared with the control period, unadjusted rates of SLNB decreased by 23.1 percentage points (46.9% SLNB rate prenudge to 23.8% after; 95% CI, -32.9 to -13.8) in the intervention period. An interrupted time series model showed a reduction in the rate of SLNB following nudge deployment (adjusted odds ratio, 0.26; 95% CI, 0.07 to 0.90; P = .03). The participating surgeons scored the intervention highly on acceptability, appropriateness, and feasibility. Dominant themes from semistructured interviews indicated that the intervention helped remind the surgeons of potential Choosing Wisely applicability without the need for additional clicks or actions on the day of the patient visit, which facilitated use. Conclusions and Relevance: This study showed that a nudge intervention in the EHR significantly decreased low-value axillary surgery in older women with early-stage, cN0, HR+/HER2- breast cancer. This user-friendly and easily implementable EHR-based intervention could be a beneficial approach for decreasing low-value care in other practice settings or patient populations. Trial Registration: ClinicalTrials.gov Identifier: NCT06006910.

Use of Natural Language Understanding to Facilitate Surgical De-Escalation of Axillary Staging in Patients With Breast Cancer.

PURPOSE: Natural language understanding (NLU) may be particularly well equipped for enhanced data capture from the electronic health record given its examination of both content-driven and context-driven extraction. METHODS: We developed and applied a NLU model to examine rates of pathological node positivity (pN+) and rates of lymphedema to determine whether omission of routine axillary staging could be extended to younger patients with estrogen receptor-positive (ER+)/cN0 disease. RESULTS: We found that rates of pN+ and arm lymphedema were similar between patients age 55-69 years and ≥70 years, with rates of lymphedema exceeding rates of pN+ for clinical stage T1c and smaller disease. CONCLUSION: Data from our NLU model suggest that omission of sentinel lymph node biopsy might be extended beyond Choosing Wisely recommendations, limited to those older than 70 years and to all postmenopausal women with early-stage ER+/cN0 disease. These data support the recently reported SOUND trial results and provide additional granularity to facilitate surgical de-escalation.

Use of natural language understanding to facilitate surgical de-escalation of axillary staging in patients with breast cancer.

Natural language understanding (NLU) may be particularly well-equipped for enhanced data capture from the electronic health record (EHR) given its examination of both content- and context-driven extraction. We developed and applied a NLU model to examine rates of pathological node positivity (pN+) and rates of lymphedema to determine if omission of routine axillary staging could be extended to younger patients with ER+/cN0 disease. We found that rates of pN+ and arm lymphedema were similar between patients 55-69yo and ≥70yo, with rates of lymphedema exceeding rates of pN+ for clinical stage T1c and smaller disease. Data from our NLU model suggest that omission of SLNB might be extended beyond Choosing Wisely recommendations, limited to those over 70 years old, to all postmenopausal women with early-stage ER+/cN0 disease. These data support the recently-reported SOUND trial results and provide additional granularity to facilitate surgical de-escalation.

Automatic Lung-RADS™ classification with a natural language processing system.

BACKGROUND: Our aim was to train a natural language processing (NLP) algorithm to capture imaging characteristics of lung nodules reported in a structured CT report and suggest the applicable Lung-RADS™ (LR) category. METHODS: Our study included structured, clinical reports of consecutive CT lung screening (CTLS) exams performed from 08/2014 to 08/2015 at an ACR accredited Lung Cancer Screening Center. All patients screened were at high-risk for lung cancer according to the NCCN Guidelines®. All exams were interpreted by one of three radiologists credentialed to read CTLS exams using LR using a standard reporting template. Training and test sets consisted of consecutive exams. Lung screening exams were divided into two groups: three training sets (500, 120, and 383 reports each) and one final evaluation set (498 reports). NLP algorithm results were compared with the gold standard of LR category assigned by the radiologist. RESULTS: The sensitivity/specificity of the NLP algorithm to correctly assign LR categories for suspicious nodules (LR 4) and positive nodules (LR 3/4) were 74.1%/98.6% and 75.0%/98.8% respectively. The majority of mismatches occurred in cases where pulmonary findings were present not currently addressed by LR. Misclassifications also resulted from the failure to identify exams as follow-up and the failure to completely characterize part-solid nodules. In a sub-group analysis among structured reports with standardized language, the sensitivity and specificity to detect LR 4 nodules were 87.0% and 99.5%, respectively. CONCLUSIONS: An NLP system can accurately suggest the appropriate LR category from CTLS exam findings when standardized reporting is used.

The development and effectiveness of a health information website designed to improve parents' self-efficacy in managing risk for obesity in preschoolers.

PURPOSE: To evaluate the effects of web-based information on parental self-efficacy in managing obesity risk in preschoolers. DESIGN AND METHODS: The project included a literature review and the development and field testing of an information website that presented information on how to manage nine obesity risk factors for childhood obesity. RESULTS: Parents stated that they had no problems using the website, and 69% reported improved self-efficacy on at least two risk factors. PRACTICE IMPLICATIONS: Many parents access the Internet to obtain health information. A website that offers practical information on managing childhood obesity risk factors is a valuable resource for obesity prevention efforts.

Factors affecting feeling-of-knowing in a medical intelligent tutoring system: the role of immediate feedback as a metacognitive scaffold.

Previous studies in our laboratory have shown the benefits of immediate feedback on cognitive performance for pathology residents using an intelligent tutoring system (ITS) in pathology. In this study, we examined the effect of immediate feedback on metacognitive performance, and investigated whether other metacognitive scaffolds will support metacognitive gains when immediate feedback is faded. Twenty-three participants were randomized into intervention and control groups. For both groups, periods working with the ITS under varying conditions were alternated with independent computer-based assessments. On day 1, a within-subjects design was used to evaluate the effect of immediate feedback on cognitive and metacognitive performance. On day 2, a between-subjects design was used to compare the use of other metacognitive scaffolds (intervention group) against no metacognitive scaffolds (control group) on cognitive and metacognitive performance, as immediate feedback was faded. Measurements included learning gains (a measure of cognitive performance), as well as several measures of metacognitive performance, including Goodman-Kruskal gamma correlation (G), bias, and discrimination. For the intervention group, we also computed metacognitive measures during tutoring sessions. Results showed that immediate feedback in an intelligent tutoring system had a statistically significant positive effect on learning gains, G and discrimination. Removal of immediate feedback was associated with decreasing metacognitive performance, and this decline was not prevented when students used a version of the tutoring system that provided other metacognitive scaffolds. Results obtained directly from the ITS suggest that other metacognitive scaffolds do have a positive effect on G and discrimination, as immediate feedback is faded. We conclude that immediate feedback had a positive effect on both metacognitive and cognitive gains in a medical tutoring system. Other metacognitive scaffolds were not sufficient to replace immediate feedback in this study. However, results obtained directly from the tutoring system are not consistent with results obtained from assessments. In order to facilitate transfer to real-world tasks, further research will be needed to determine the optimum methods for supporting metacognition as immediate feedback is faded.

A natural language intelligent tutoring system for training pathologists: implementation and evaluation.

INTRODUCTION: We developed and evaluated a Natural Language Interface (NLI) for an Intelligent Tutoring System (ITS) in Diagnostic Pathology. The system teaches residents to examine pathologic slides and write accurate pathology reports while providing immediate feedback on errors they make in their slide review and diagnostic reports. Residents can ask for help at any point in the case, and will receive context-specific feedback. RESEARCH QUESTIONS: We evaluated (1) the performance of our natural language system, (2) the effect of the system on learning (3) the effect of feedback timing on learning gains and (4) the effect of ReportTutor on performance to self-assessment correlations. METHODS: The study uses a crossover 2 x 2 factorial design. We recruited 20 subjects from 4 academic programs. Subjects were randomly assigned to one of the four conditions--two conditions for the immediate interface, and two for the delayed interface. An expert dermatopathologist created a reference standard and 2 board certified AP/CP pathology fellows manually coded the residents' assessment reports. Subjects were given the opportunity to self grade their performance and we used a survey to determine student response to both interfaces. RESULTS: Our results show a highly significant improvement in report writing after one tutoring session with 4-fold increase in the learning gains with both interfaces but no effect of feedback timing on performance gains. Residents who used the immediate feedback interface first experienced a feature learning gain that is correlated with the number of cases they viewed. There was no correlation between performance and self-assessment in either condition.

Definition of an XML markup language for clinical laboratory procedures and comparison with generic XML markup.

BACKGROUND: Clinical laboratory procedure manuals are typically maintained as word processor files and are inefficient to store and search, require substantial effort for review and updating, and integrate poorly with other laboratory information. Electronic document management systems could improve procedure management and utility. As a first step toward building such systems, we have developed a prototype electronic format for laboratory procedures using Extensible Markup Language (XML). METHODS: Representative laboratory procedures were analyzed to identify document structure and data elements. This information was used to create a markup vocabulary, CLP-ML, expressed as an XML Document Type Definition (DTD). To determine whether this markup provided advantages over generic markup, we compared procedures structured with CLP-ML or with the vocabulary of the Health Level Seven, Inc. (HL7) Clinical Document Architecture (CDA) narrative block. RESULTS: CLP-ML includes 124 XML tags and supports a variety of procedure types across different laboratory sections. When compared with a general-purpose markup vocabulary (CDA narrative block), CLP-ML documents were easier to edit and read, less complex structurally, and simpler to traverse for searching and retrieval. CONCLUSION: In combination with appropriate software, CLP-ML is designed to support electronic authoring, reviewing, distributing, and searching of clinical laboratory procedures from a central repository, decreasing procedure maintenance effort and increasing the utility of procedure information. A standard electronic procedure format could also allow laboratories and vendors to share procedures and procedure layouts, minimizing duplicative word processor editing. Our results suggest that laboratory-specific markup such as CLP-ML will provide greater benefit for such systems than generic markup.

Caring connection: developing an Internet resource for family caregivers of children with cancer.

This article describes the development of a family-focused Internet resource, the Caring Connection Web site, designed to provide individualized clinical information, patient-centered information, and access to online communication for caregivers of children with cancer. Data from surveys with family caregivers described the scope and characteristics of their current Internet use. Web site prototype development and user testing provided additional insights that inform the ongoing design of the Caring Connection Web site. The family caregivers and oncology healthcare providers who participated are active users of Internet health information. The preliminary data they provided support the feasibility and usability of the Caring Connection prototype. Ongoing development of the Caring Connection Web site will provide an innovative approach to assist these family caregivers in meeting currently unmet information and communication needs, and will provide evidence regarding "best practice" in design and development of Internet resources to support communication and information sharing.

A method for automated detection of usability problems from client user interface events.

Think-aloud usability analysis provides extremely useful data but is very time-consuming and expensive to perform because of the extensive manual video analysis that is required. We describe a simple method for automated detection of usability problems from client user interface events for a developing medical intelligent tutoring system. The method incorporates (1) an agent-based method for communication that funnels all interface events and system responses to a centralized database, (2) a simple schema for representing interface events and higher order subgoals, and (3) an algorithm that reproduces the criteria used for manual coding of usability problems. A correction factor was empirically determining to account for the slower task performance of users when thinking aloud. We tested the validity of the method by simultaneously identifying usability problems using TAU and manually computing them from stored interface event data using the proposed algorithm. All usability problems that did not rely on verbal utterances were detectable with the proposed method.

Caring Connection: developing an Internet resource for family caregivers of children with cancer.

This poster will describe the development of a family-focused, web-based resource, the "Caring Connection", designed to provide individualized clinical information, patient-centered information resources, and access to online family-provider and family-to-family communication for caregivers of children with cancer. Data from structured interviews with family caregivers and healthcare providers describe their current Internet use and information needs. Website prototype development and user testing provide additional insights that are informing the ongoing design of the "Caring Connection" website.

XML syntax for clinical laboratory procedure manuals.

We have developed a document type description (DTD) in Extensable Markup Language (XML) for clinical laboratory procedures. Our XML syntax can adequately structure a variety of procedure types across different laboratories and is compatible with current procedure standards. The combination of this format with an XML content management system and appropriate style sheets will allow efficient procedure maintenance, distributed access, customized display and effective searching across a large body of test information.