Automatic Lung-RADS™ classification with a natural language processing system.

BACKGROUND: Our aim was to train a natural language processing (NLP) algorithm to capture imaging characteristics of lung nodules reported in a structured CT report and suggest the applicable Lung-RADS™ (LR) category. METHODS: Our study included structured, clinical reports of consecutive CT lung screening (CTLS) exams performed from 08/2014 to 08/2015 at an ACR accredited Lung Cancer Screening Center. All patients screened were at high-risk for lung cancer according to the NCCN Guidelines®. All exams were interpreted by one of three radiologists credentialed to read CTLS exams using LR using a standard reporting template. Training and test sets consisted of consecutive exams. Lung screening exams were divided into two groups: three training sets (500, 120, and 383 reports each) and one final evaluation set (498 reports). NLP algorithm results were compared with the gold standard of LR category assigned by the radiologist. RESULTS: The sensitivity/specificity of the NLP algorithm to correctly assign LR categories for suspicious nodules (LR 4) and positive nodules (LR 3/4) were 74.1%/98.6% and 75.0%/98.8% respectively. The majority of mismatches occurred in cases where pulmonary findings were present not currently addressed by LR. Misclassifications also resulted from the failure to identify exams as follow-up and the failure to completely characterize part-solid nodules. In a sub-group analysis among structured reports with standardized language, the sensitivity and specificity to detect LR 4 nodules were 87.0% and 99.5%, respectively. CONCLUSIONS: An NLP system can accurately suggest the appropriate LR category from CTLS exam findings when standardized reporting is used.

The development and effectiveness of a health information website designed to improve parents' self-efficacy in managing risk for obesity in preschoolers.

PURPOSE: To evaluate the effects of web-based information on parental self-efficacy in managing obesity risk in preschoolers. DESIGN AND METHODS: The project included a literature review and the development and field testing of an information website that presented information on how to manage nine obesity risk factors for childhood obesity. RESULTS: Parents stated that they had no problems using the website, and 69% reported improved self-efficacy on at least two risk factors. PRACTICE IMPLICATIONS: Many parents access the Internet to obtain health information. A website that offers practical information on managing childhood obesity risk factors is a valuable resource for obesity prevention efforts.

Factors affecting feeling-of-knowing in a medical intelligent tutoring system: the role of immediate feedback as a metacognitive scaffold.

Previous studies in our laboratory have shown the benefits of immediate feedback on cognitive performance for pathology residents using an intelligent tutoring system (ITS) in pathology. In this study, we examined the effect of immediate feedback on metacognitive performance, and investigated whether other metacognitive scaffolds will support metacognitive gains when immediate feedback is faded. Twenty-three participants were randomized into intervention and control groups. For both groups, periods working with the ITS under varying conditions were alternated with independent computer-based assessments. On day 1, a within-subjects design was used to evaluate the effect of immediate feedback on cognitive and metacognitive performance. On day 2, a between-subjects design was used to compare the use of other metacognitive scaffolds (intervention group) against no metacognitive scaffolds (control group) on cognitive and metacognitive performance, as immediate feedback was faded. Measurements included learning gains (a measure of cognitive performance), as well as several measures of metacognitive performance, including Goodman-Kruskal gamma correlation (G), bias, and discrimination. For the intervention group, we also computed metacognitive measures during tutoring sessions. Results showed that immediate feedback in an intelligent tutoring system had a statistically significant positive effect on learning gains, G and discrimination. Removal of immediate feedback was associated with decreasing metacognitive performance, and this decline was not prevented when students used a version of the tutoring system that provided other metacognitive scaffolds. Results obtained directly from the ITS suggest that other metacognitive scaffolds do have a positive effect on G and discrimination, as immediate feedback is faded. We conclude that immediate feedback had a positive effect on both metacognitive and cognitive gains in a medical tutoring system. Other metacognitive scaffolds were not sufficient to replace immediate feedback in this study. However, results obtained directly from the tutoring system are not consistent with results obtained from assessments. In order to facilitate transfer to real-world tasks, further research will be needed to determine the optimum methods for supporting metacognition as immediate feedback is faded.

A natural language intelligent tutoring system for training pathologists: implementation and evaluation.

INTRODUCTION: We developed and evaluated a Natural Language Interface (NLI) for an Intelligent Tutoring System (ITS) in Diagnostic Pathology. The system teaches residents to examine pathologic slides and write accurate pathology reports while providing immediate feedback on errors they make in their slide review and diagnostic reports. Residents can ask for help at any point in the case, and will receive context-specific feedback. RESEARCH QUESTIONS: We evaluated (1) the performance of our natural language system, (2) the effect of the system on learning (3) the effect of feedback timing on learning gains and (4) the effect of ReportTutor on performance to self-assessment correlations. METHODS: The study uses a crossover 2 x 2 factorial design. We recruited 20 subjects from 4 academic programs. Subjects were randomly assigned to one of the four conditions--two conditions for the immediate interface, and two for the delayed interface. An expert dermatopathologist created a reference standard and 2 board certified AP/CP pathology fellows manually coded the residents' assessment reports. Subjects were given the opportunity to self grade their performance and we used a survey to determine student response to both interfaces. RESULTS: Our results show a highly significant improvement in report writing after one tutoring session with 4-fold increase in the learning gains with both interfaces but no effect of feedback timing on performance gains. Residents who used the immediate feedback interface first experienced a feature learning gain that is correlated with the number of cases they viewed. There was no correlation between performance and self-assessment in either condition.

Caring connection: developing an Internet resource for family caregivers of children with cancer.

This article describes the development of a family-focused Internet resource, the Caring Connection Web site, designed to provide individualized clinical information, patient-centered information, and access to online communication for caregivers of children with cancer. Data from surveys with family caregivers described the scope and characteristics of their current Internet use. Web site prototype development and user testing provided additional insights that inform the ongoing design of the Caring Connection Web site. The family caregivers and oncology healthcare providers who participated are active users of Internet health information. The preliminary data they provided support the feasibility and usability of the Caring Connection prototype. Ongoing development of the Caring Connection Web site will provide an innovative approach to assist these family caregivers in meeting currently unmet information and communication needs, and will provide evidence regarding "best practice" in design and development of Internet resources to support communication and information sharing.

Caring Connection: developing an Internet resource for family caregivers of children with cancer.

This poster will describe the development of a family-focused, web-based resource, the "Caring Connection", designed to provide individualized clinical information, patient-centered information resources, and access to online family-provider and family-to-family communication for caregivers of children with cancer. Data from structured interviews with family caregivers and healthcare providers describe their current Internet use and information needs. Website prototype development and user testing provide additional insights that are informing the ongoing design of the "Caring Connection" website.