RESUMO
Attention mechanisms are now a mainstay architecture in neural networks and improve the performance of biomedical text classification tasks. In particular, models that perform automated medical encoding of clinical documents make extensive use of the label-wise attention mechanism. A label-wise attention mechanism increases a model's discriminatory ability by using label-specific reference information. This information can either be implicitly learned during training or explicitly provided through embedded textual code descriptions or information on the code hierarchy; however, contemporary studies arbitrarily select the type of label-specific reference information. To address this shortcoming, we evaluated label-wise attention initialized with either implicit or explicit label-specific reference information against two common baseline methods-target-attention and text-encoder architecture-specific methods-to generate document embeddings across four text-encoder architectures-a convolutional neural network, two recurrent neural networks, and a transformer. We also present an extension of label-wise attention that can embed the information on the code hierarchy. We performed our experiments on the MIMIC III dataset, which is a standard dataset in the clinical text classification domain. Our experiments showed that using pretrained reference information and the hierarchical design helped improve classification performance. These performance improvements had less impact on larger datasets and label spaces across all text-encoder architectures. In our analysis, we used an attention mechanism's energy scores to explain the perceived differences in performance and interpretability between the text-encoder architectures and types of label-attention.
RESUMO
Introduction: Translating narrative clinical guidelines to computable knowledge is a long-standing challenge that has seen a diverse range of approaches. The UK National Institute for Health and Care Excellence (NICE) Content Advisory Board (CAB) aims ultimately to (1) guide clinical decision support and other software developers to increase traceability, fidelity and consistency in supporting clinical use of NICE recommendations, (2) guide local practice audit and intervention to reduce unwarranted variation, (3) provide feedback to NICE on how future recommendations should be developed. Objectives: The first phase of work was to explore a range of technical approaches to transition NICE toward the production of natively digital content. Methods: Following an initial 'collaborathon' in November 2022, the NICE Computable Implementation Guidance project (NCIG) was established. We held a series of workstream calls approximately fortnightly, focusing on (1) user stories and trigger events, (2) information model and definitions, (3) horizon-scanning and output format. A second collaborathon was held in March 2023 to consolidate progress across the workstreams and agree residual actions to complete. Results: While we initially focussed on technical implementation standards, we decided that an intermediate logical model was a more achievable first step in the journey from narrative to fully computable representation. NCIG adopted the WHO Digital Adaptation Kit (DAK) as a technology-agnostic method to model user scenarios, personae, processes and workflow, core data elements and decision-support logic. Further work will address indicators, such as prescribing compliance, and implementation in document templates for primary care patient record systems. Conclusions: The project has shown that the WHO DAK, with some modification, is a promising approach to build technology-neutral logical specifications of NICE recommendations. Implementation of concurrent computable modelling by multidisciplinary teams during guideline development poses methodological and cultural questions that are complex but tractable given suitable will and leadership.