Knowledge-Infused Abstractive Summarization of Clinical Diagnostic Interviews: Framework Development Study.

BACKGROUND: In clinical diagnostic interviews, mental health professionals (MHPs) implement a care practice that involves asking open questions (eg, "What do you want from your life?" "What have you tried before to bring change in your life?") while listening empathetically to patients. During these interviews, MHPs attempted to build a trusting human-centered relationship while collecting data necessary for professional medical and psychiatric care. Often, because of the social stigma of mental health disorders, patient discomfort in discussing their presenting problem may add additional complexities and nuances to the language they use, that is, hidden signals among noisy content. Therefore, a focused, well-formed, and elaborative summary of clinical interviews is critical to MHPs in making informed decisions by enabling a more profound exploration of a patient's behavior, especially when it endangers life. OBJECTIVE: The aim of this study is to propose an unsupervised, knowledge-infused abstractive summarization (KiAS) approach that generates summaries to enable MHPs to perform a well-informed follow-up with patients to improve the existing summarization methods built on frequency heuristics by creating more informative summaries. METHODS: Our approach incorporated domain knowledge from the Patient Health Questionnaire-9 lexicon into an integer linear programming framework that optimizes linguistic quality and informativeness. We used 3 baseline approaches: extractive summarization using the SumBasic algorithm, abstractive summarization using integer linear programming without the infusion of knowledge, and abstraction over extractive summarization to evaluate the performance of KiAS. The capability of KiAS on the Distress Analysis Interview Corpus-Wizard of Oz data set was demonstrated through interpretable qualitative and quantitative evaluations. RESULTS: KiAS generates summaries (7 sentences on average) that capture informative questions and responses exchanged during long (58 sentences on average), ambiguous, and sparse clinical diagnostic interviews. The summaries generated using KiAS improved upon the 3 baselines by 23.3%, 4.4%, 2.5%, and 2.2% for thematic overlap, Flesch Reading Ease, contextual similarity, and Jensen Shannon divergence, respectively. On the Recall-Oriented Understudy for Gisting Evaluation-2 and Recall-Oriented Understudy for Gisting Evaluation-L metrics, KiAS showed an improvement of 61% and 49%, respectively. We validated the quality of the generated summaries through visual inspection and substantial interrater agreement from MHPs. CONCLUSIONS: Our collaborator MHPs observed the potential utility and significant impact of KiAS in leveraging valuable but voluminous communications that take place outside of normally scheduled clinical appointments. This study shows promise in generating semantically relevant summaries that will help MHPs make informed decisions about patient status.

Enhanced Training Benefits of Video Recording Surgery With Automated Hand Motion Analysis.

BACKGROUND: Hand motion analysis by video recording during surgery has potential for evaluation of surgical performance. The aim was to identify how technical skill during open surgery can be measured unobtrusively by video recording during a surgical procedure. We hypothesized that procedural-step timing, hand movements, instrument use and Shannon entropy differ with expertise and training and are concordant with a performance-based validated individual procedure score. METHODS: Surgeon and non-surgeon participants with varying training and levels of expertise were video recorded performing axillary artery exposure and control (AA) on un-preserved cadavers. Color-coded gloves permitted motion-tracking and automated extraction of entropy data from recordings. Timing and instrument-use metrics were obtained through observational video reviews. Shannon entropy measured speed, acceleration and direction by computer-vision algorithms. Findings were compared with individual procedure score for AA performance RESULTS: Experts had lowest entropy values, idle time, active time and shorter time to divide pectoralis minor, using fewer instruments. Residents improved with training, without reaching expert levels, and showed deterioration 12-18 months later. Individual procedure scores mirrored these results. Non-surgeons differed substantially. CONCLUSIONS: Hand motion entropy and timing metrics discriminate levels of surgical skill and training, and these findings are congruent with individual procedure score evaluations. These measures can be collected using consumer-level cameras and analyzed automatically with free software. Hand motion with video timing data may have widespread application to evaluate resident performance and can contribute to the range of evaluation and testing modalities available to educators, training course designers and surgical quality assurance programs.

Analyzing and learning the language for different types of harassment.

THIS ARTICLE USES WORDS OR LANGUAGE THAT IS CONSIDERED PROFANE, VULGAR, OR OFFENSIVE BY SOME READERS. The presence of a significant amount of harassment in user-generated content and its negative impact calls for robust automatic detection approaches. This requires the identification of different types of harassment. Earlier work has classified harassing language in terms of hurtfulness, abusiveness, sentiment, and profanity. However, to identify and understand harassment more accurately, it is essential to determine the contextual type that captures the interrelated conditions in which harassing language occurs. In this paper we introduce the notion of contextual type in harassment by distinguishing between five contextual types: (i) sexual, (ii) racial, (iii) appearance-related, (iv) intellectual and (v) political. We utilize an annotated corpus from Twitter distinguishing these types of harassment. We study the context of each kind to shed light on the linguistic meaning, interpretation, and distribution, with results from two lines of investigation: an extensive linguistic analysis, and the statistical distribution of uni-grams. We then build type- aware classifiers to automate the identification of type-specific harassment. Our experiments demonstrate that these classifiers provide competitive accuracy for identifying and analyzing harassment on social media. We present extensive discussion and significant observations about the effectiveness of type-aware classifiers using a detailed comparison setup, providing insight into the role of type-dependent features.

Cadaver-Based Trauma Procedural Skills Training: Skills Retention 30 Months after Training among Practicing Surgeons in Comparison to Experts or More Recently Trained Residents.

BACKGROUND: Long-term retention of trauma procedural core-competency skills and need for re-training after a 1-day cadaver-based course remains unknown. We measured and compared technical skills for trauma core competencies at mean 14 months (38 residents), 30 months (35 practicing surgeons), and 46 months (10 experts) after training to determine if skill degradation occurs with time. Technical performance during extremity vascular exposures and lower-extremity fasciotomy in fresh cadavers measured by validated individual procedure score (IPS) was the primary outcome. STUDY DESIGN: We performed a prospective study between May 2013 and September 2016. RESULTS: Practicing surgeons had lower IPS and IPS component scores (p = 0.02 to 0.001) than residents (p < 0.05) and experts (p < 0.002) for vascular procedures. Frequencies of errors were no different among residents and experts. Practicing surgeons made more critical errors (p < 0.05) than experts or residents. Experts had shortest time to proximal vascular control. Fasciotomy procedural errors occurred in all participants. Cluster analysis of anatomy vs procedural steps identified tertiles of performance and wide variance (32.5% practicing surgeons, 26.5% residents vs 13% experts) for vascular procedures. Vascular control duration > 20 minutes (n = 21) and failure to decompress fasciotomy compartments were correlated with incorrect landmarks and skin incisions. Modeling found interval trauma skills experience, not time since training, was associated with lower IPS. CONCLUSIONS: Practicing surgeons with low trauma skills experience since training had lower IPS and component scores (p = 0.02 to 0.001) and more errors compared with experts and residents (p < 0.05). Surgeons, including experts with low interval experience performing trauma procedures, may benefit from refreshing of correct landmarks and skin incision placement identification.

Assessment of Anatomical Knowledge and Core Trauma Competency Vascular Skills.

Objectives: Surgical residents express confidence in performing specific vascular exposures before training, but such self-reported confidence did not correlate with co-located evaluator ratings. This study reports residents' self-confidence evaluated before and after Advanced Surgical Skills for Exposure in Trauma (ASSET) cadaver-based training, and 12-18 mo later. We hypothesize that residents will better judge their own skill after ASSET than before when compared with evaluator ratings. Methods: Forty PGY2-7 surgical residents performed four procedures: axillary artery (AA), brachial artery (BA), femoral artery exposure and control (FA), and lower extremity fasciotomy (FAS) at the three evaluations. Using 5-point Likert scales, surgeons self-assessed their confidence in anatomical understanding and procedure performance after each procedure and evaluators rated each surgeon accordingly. Results: For all the three evaluations, residents consistently rated their anatomical understanding (p < 0.04) and surgical performance (p < 0.03) higher than evaluators for both FA and FAS. Residents rated their anatomical understanding and surgical performance higher (p < 0.005) than evaluators for BA after training and up to 18 mo later. Only for third AA evaluation were there no rating differences. Conclusions: Residents overrate their anatomical understanding and performance abilities for BA, FA, and FAS even after performing the procedures and being debriefed three times in 18 mo.

Can hyper-realistic physical models of peripheral vessel exposure and fasciotomy replace cadavers for performance assessment?

BACKGROUND: Work-hour restrictions have reduced operative experience for residents. The Advanced Surgical Skills for Exposure in Trauma (ASSET) course fills this training gap. Cadaver use has limitations including cost and availability. Hyper-realistic synthetic models may provide an alternative to cadavers. We compared same surgeon performance between synthetic and cadaveric models to determine interchangeability for formative evaluation. METHODS: Forty residents (<4 weeks after ASSET) and 35 faculty (mean, 2.5 ± 1.3 years after ASSET) exposed axillary, brachial, and femoral arteries, and performed lower extremity fasciotomy. Separate evaluators and random starting order between models were used for participants. Individual procedure scores and aggregate procedure scores, a trauma readiness index, evaluated participants. Student's t and χ tests were used where appropriate. p Values less than 0.05 were considered significant. RESULTS: For same surgeons, faculty, but not residents, had higher trauma readiness index on the synthetic model (0.63 vs. 0.70, p < 0.01; 0.63 vs. 0.67, p = 0.06, respectively). Scores were not significantly different between models for residents except for the brachial artery exposure (0.68 vs. 0.75, p < 0.01), which was the least realistic of all procedures. Faculty did significantly better on the synthetic model in all procedures. All participants completed procedures nearly twice as quickly (5.61 ± 3.21 vs. 10.08 ± 4.66 minutes) and performed fewer errors on the synthetic model (113 vs. 53, p < 0.01; 118 vs. 76, p = 0.03, respectively). CONCLUSION: Same surgeons performed procedures quicker and with fewer errors on the synthetic model. Residents performed similarly on both model types, this likely represents the unfamiliarity neophytes bring to new procedures. This suggests that the synthetic model, with easily discernible and standardized anatomy, may be useful in the early stages of training to understand critical procedural steps. The difficulty of the cadaver is more apt to assess and evaluate the experienced surgeon and identify opportunities for improvement. LEVEL OF EVIDENCE: Prognostic, level III.

Sentinels of Breach: Lexical Choice as a Measure of Urgency in Social Media.

Objective This paper identifies general properties of language style in social media to help identify areas of need in disasters. Background In the search for metrics of need in social media data, much of the existing literature ignores processes of language usage. Psychological concepts, such as narrative breach, Gricean maxims, and lexical marking in cognition, may assist the recovery of disaster-relevant metrics from altered patterns of word prevalence. Method We analyzed several hundred thousand location-specific microblogs from Twitter for Hurricane Sandy, Oklahoma tornadoes, and the Boston Marathon bombing along with a fantasy football control corpus, examining the relative frequency of words in 36 antonym pairs. We compared the ratio of words within these pairs to the corresponding ratios recovered from an online word norm database. Results Partial rank correlation values between observed antonym ratios demonstrate consistent patterns across disasters. For Hurricane Sandy data, 25 antonym pairs have moderate to large effect sizes for discrepancies between observed and normative ratios. Across disasters, 7 pairs are stable and meet effect size criteria. Sentiment analysis, supplementary word frequency counts with respect to disaster proximity, and examples support a "breach" account for the observed results. Conclusion Lexical choice between antonyms, only somewhat related to sentiment, suggests that social media capture wide-ranging breaches of normal functioning. Application Antonym selection contributes to screening tools based on language style for identifying relevant content and quantifying disruption using social media without the a priori specification of content keywords.

Navigating the Decision Space: Shared Medical Decision Making as Distributed Cognition.

Despite increasing prominence, little is known about the cognitive processes underlying shared decision making. To investigate these processes, we conceptualize shared decision making as a form of distributed cognition. We introduce a Decision Space Model to identify physical and social influences on decision making. Using field observations and interviews, we demonstrate that patients and physicians in both acute and chronic care consider these influences when identifying the need for a decision, searching for decision parameters, making actionable decisions Based on the distribution of access to information and actions, we then identify four related patterns: physician dominated; physician-defined, patient-made; patient-defined, physician-made; and patient-dominated decisions. Results suggests that (a) decision making is necessarily distributed between physicians and patients, (b) differential access to information and action over time requires participants to transform a distributed task into a shared decision, and (c) adverse outcomes may result from failures to integrate physician and patient reasoning. Our analysis unifies disparate findings in the medical decision-making literature and has implications for improving care and medical training.

Using an Individual Procedure Score Before and After the Advanced Surgical Skills Exposure for Trauma Course Training to Benchmark a Hemorrhage-Control Performance Metric.

OBJECTIVE: Test with an individual procedure score (IPS) to assess whether an unpreserved cadaver trauma training course, including upper and lower limb vascular exposure, improves correct identification of surgical landmarks, underlying anatomy, and shortens time to vascular control. DESIGN: Prospective study of performance of 3 vascular exposure and control procedures (axillary, brachial, and femoral arteries) using IPS metrics by 2 colocated and trained evaluators before and after training with the Advanced Surgical Skills Exposure for Trauma (ASSET) course. IPS, including identification of anatomical landmarks, incisions, underlying structures, and time to completion of each procedure was compared before and after training using repeated measurement models. SETTING: Audio-video instrumented cadaver laboratory at University of Maryland School of Medicine. PARTICIPANTS: A total of 41 second to sixth year surgical residents from surgical programs throughout Mid-Atlantic States who had not previously taken the ASSET course were enrolled, 40 completed the pre- and post-ASSET performance evaluations. RESULTS: After ASSET training, all components of IPS increased and time shortened for each of the 3 artery exposures. Procedure steps performed correctly increased 57%, anatomical knowledge increased 43% and skin incision to passage of a vessel loop twice around the correct vessel decreased by a mean of 2.5 minutes. An overall vascular trauma readiness index, a comprehensive IPS score for 3 procedures increased 28% with ASSET Training. CONCLUSIONS: Improved knowledge of surface landmarks and underlying anatomy is associated with increased IPS, faster procedures, more accurate incision placement, and successful vascular control. Structural recognition during specific procedural steps and anatomical knowledge were key points learned during the ASSET course. Such training may accelerate acquisition of specific trauma surgery skills to compensate for shortened training hours, infrequent exposure to major vascular injuries, or when just-in-time training is necessary. IPS is a benchmark for competence in extremity vascular control.

Development and validation of trauma surgical skills metrics: Preliminary assessment of performance after training.

BACKGROUND: Maintaining trauma-specific surgical skills is an ongoing challenge for surgical training programs. An objective assessment of surgical skills is needed. We hypothesized that a validated surgical performance assessment tool could detect differences following a training intervention. METHODS: We developed surgical performance assessment metrics based on discussion with expert trauma surgeons, video review of 10 experts and 10 novice surgeons performing three vascular exposure procedures and lower extremity fasciotomy on cadavers, and validated the metrics with interrater reliability testing by five reviewers blinded to level of expertise and a consensus conference. We tested these performance metrics in 12 surgical residents (Year 3-7) before and 2 weeks after vascular exposure skills training in the Advanced Surgical Skills for Exposure in Trauma (ASSET) course. Performance was assessed in three areas as follows: knowledge (anatomic, management), procedure steps, and technical skills. Time to completion of procedures was recorded, and these metrics were combined into a single performance score, the Trauma Readiness Index (TRI). Wilcoxon matched-pairs signed-ranks test compared pretraining/posttraining effects. RESULTS: Mean time to complete procedures decreased by 4.3 minutes (from 13.4 minutes to 9.1 minutes). The performance component most improved by the 1-day skills training was procedure steps, completion of which increased by 21%. Technical skill scores improved by 12%. Overall knowledge improved by 3%, with 18% improvement in anatomic knowledge. TRI increased significantly from 50% to 64% with ASSET training. Interrater reliability of the surgical performance assessment metrics was validated with single intraclass correlation coefficient of 0.7 to 0.98. CONCLUSION: A trauma-relevant surgical performance assessment detected improvements in specific procedure steps and anatomic knowledge taught during a 1-day course, quantified by the TRI. ASSET training reduced time to complete vascular control by one third. Future applications include assessing specific skills in a larger surgeon cohort, assessing military surgical readiness, and quantifying skill degradation with time since training.

The impact of medical record technologies on collaboration in emergency medicine.

PURPOSE: Electronic medical records (EMR) promise potential benefits for the practice of medical care. However, individual technologies such as EMR must interact with the work system as a whole - including people, technology and work practices - to enable or hinder the coordination of dynamic work demands. Based on this extended perspective, we address in this paper how support technologies (should) impact the coordination of work across multiple agents, controlling a dynamic domain with multiple, interacting processes. The technology we address is the medical record and the dynamic domain is emergency medicine as it is practiced in the U.S. METHOD: We performed 500 hours of naturalistic observations of physicians in two different hospital emergency departments in the Midwestern U.S differing in their reliance on paper or electronic medical records. RESULTS AND CONCLUSIONS: An analysis of work practice across the two hospitals revealed the role of medical records in facilitating or hindering the coordination of time sensitive and context dependent distributed work, as well as the specific influence of EMR. Recognizing that work practice compensates for the limitations of technology, we suggest four requirements for the design of EMR to promote workplace efficiency: facilitation of locally customized data presentations; support for integration of hitherto fragmented record systems and data formats; support for effective multi-user coordination of control tasks; and guidance for standardizing a level of detail in planning and documenting care.

Everyday expertise: cognitive demands in diabetes selfmanagement.

OBJECTIVE: To assess the relationship between decision making and successful diabetes self-management. BACKGROUND: Patients with type II diabetes make routine but critical self-management decisions. METHOD: We conducted cognitive task analysis interviews with 18 patients to examine problem detection, functional relationships, problem-solving strategies, and types of knowledge used to make self-management decisions. We expected that these decision processes would be related to behavioral adherence and glycemic control. RESULTS: Verbal reports displaying problem detection skills, knowledge of functional relationships, and effective problem-solving strategies were all related to better adherence. Problem detection skill was linked to greater glycemic control. Participants differed in declarative and applied knowledge. CONCLUSION: Diabetes self-management draws on the same cognitive skills found in experts from diverse professional domains. Considering diabetes self-management as a form of expertise may support adherence. APPLICATION: Human factors approaches that support professional expertise may be useful for the decision making of patients with diabetes and other chronic diseases.

The identification of knowledge content and function in manual labour.

Calls for an alternative conceptualization of cognition for applied concerns retain the core commitment of the basic research community to abstract cognition detached from a physical environment. The present paper attempts to break out of the dominant, narrow view of cognition and cognitive domains, with a cognitive analysis of digging ditches for the utility industry. To illustrate knowledge-based cognition in manual labour excerpts are presented from the journal entries of a moderately experienced student working a summer job, organized with a representation that distinguishes between the goals and methods of work. The journal entries illustrate the functions of knowledge for interacting with a physical environment; knowledge enables the selection, execution and monitoring of work methods, the interpretation of perceptual information, the application of task completion criteria and the ability for explanation and generalization. To emphasize the generality of the functions of cognition in ditch digging, comparable functions are indicated in a domain rarely regarded as a form of manual labour: the practice of internal medicine. Discussion of the results includes the implications for cognitive theory as well as practical implications for productivity, training and task analysis.