Monte Carlo Inverse RNA Folding.

The inverse RNA folding problem deals with designing a sequence of nucleotides that will fold into a desired target structure. Generalized Nested Rollout Policy Adaptation (GNRPA) is a Monte Carlo search algorithm for optimizing a sequence of choices. It learns a playout policy to intensify the search of the state space near the current best sequence. The algorithm uses a prior on the possible actions so as to perform non uniform playouts when learning the instance of problem at hand. We trained a transformer neural network on the inverse RNA folding problem using the Rfam database. This network is used to generate a prior for every Eterna100 puzzle. GNRPA is used with this prior to solve some of the instances of the Eterna100 dataset. The transformer prior gives better result than handcrafted heuristics.

Vision transformer introduces a new vitality to the classification of renal pathology.

Recent advancements in computer vision within the field of artificial intelligence (AI) have made significant inroads into the medical domain. However, the application of AI for classifying renal pathology remains challenging due to the subtle variations in multiple renal pathological classifications. Vision Transformers (ViT), an adaptation of the Transformer model for image recognition, have demonstrated superior capabilities in capturing global features and providing greater explainability. In our study, we developed a ViT model using a diverse set of stained renal histopathology images to evaluate its effectiveness in classifying renal pathology. A total of 1861 whole slide images (WSI) stained with HE, MASSON, PAS, and PASM were collected from 635 patients. Renal tissue images were then extracted, tiled, and categorized into 14 classes on the basis of renal pathology. We employed the classic ViT model from the Timm library, utilizing images sized 384 × 384 pixels with 16 × 16 pixel patches, to train the classification model. A comparative analysis was conducted to evaluate the performance of the ViT model against traditional convolutional neural network (CNN) models. The results indicated that the ViT model demonstrated superior recognition ability (accuracy: 0.96-0.99). Furthermore, we visualized the identification process of the ViT models to investigate potentially significant pathological ultrastructures. Our study demonstrated that ViT models outperformed CNN models in accurately classifying renal pathology. Additionally, ViT models are able to focus on specific, significant structures within renal histopathology, which could be crucial for identifying novel and meaningful pathological features in the diagnosis and treatment of renal disease.

Ensemble of vision transformer architectures for efficient Alzheimer's Disease classification.

Transformers have dominated the landscape of Natural Language Processing (NLP) and revolutionalized generative AI applications. Vision Transformers (VT) have recently become a new state-of-the-art for computer vision applications. Motivated by the success of VTs in capturing short and long-range dependencies and their ability to handle class imbalance, this paper proposes an ensemble framework of VTs for the efficient classification of Alzheimer's Disease (AD). The framework consists of four vanilla VTs, and ensembles formed using hard and soft-voting approaches. The proposed model was tested using two popular AD datasets: OASIS and ADNI. The ADNI dataset was employed to assess the models' efficacy under imbalanced and data-scarce conditions. The ensemble of VT saw an improvement of around 2% compared to individual models. Furthermore, the results are compared with state-of-the-art and custom-built Convolutional Neural Network (CNN) architectures and Machine Learning (ML) models under varying data conditions. The experimental results demonstrated an overall performance gain of 4.14% and 4.72% accuracy over the ML and CNN algorithms, respectively. The study has also identified specific limitations and proposes avenues for future research. The codes used in the study are made publicly available.

GBERT: A hybrid deep learning model based on GPT-BERT for fake news detection.

The digital era has expanded social exposure with easy internet access for mobile users, allowing for global communication. Now, people can get to know what is going on around the globe with just a click; however, this has also resulted in the issue of fake news. Fake news is content that pretends to be true but is actually false and is disseminated to defraud. Fake news poses a threat to harmony, politics, the economy, and public opinion. As a result, bogus news detection has become an emerging research domain to identify a given piece of text as genuine or fraudulent. In this paper, a new framework called Generative Bidirectional Encoder Representations from Transformers (GBERT) is proposed that leverages a combination of Generative pre-trained transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT) and addresses the fake news classification problem. This framework combines the best features of both cutting-edge techniques-BERT's deep contextual understanding and the generative capabilities of GPT-to create a comprehensive representation of a given text. Both GPT and BERT are fine-tuned on two real-world benchmark corpora and have attained 95.30 % accuracy, 95.13 % precision, 97.35 % sensitivity, and a 96.23 % F1 score. The statistical test results indicate the effectiveness of the fine-tuned framework for fake news detection and suggest that it can be a promising approach for eradicating this global issue of fake news in the digital landscape.

Classification of Veterinary Subjects in Medical Literature and Clinical Summaries.

INTRODUCTION: Human and veterinary medicine are practiced separately, but literature databases such as Pubmed include articles from both fields. This impedes supporting clinical decisions with automated information retrieval, because treatment considerations would not ignore the discipline of mixed sources. Here we investigate data-driven methods from computational linguistics for automatically distinguishing between human and veterinary medical texts. METHODS: For our experiments, we selected language models after a literature review of benchmark datasets and reported performances. We generated a dataset of around 48,000 samples for binary text classification, specifically designed to differentiate between human medical and veterinary subjects. Using this dataset, we trained and fine-tuned classifiers based on selected transformer-based models as well as support vector machines (SVM). RESULTS: All trained classifiers achieved more than 99% accuracy, even though the transformer-based classifiers moderately outperformed the SVM-based one. DISCUSSION: Such classifiers could be applicable in clinical decision support functions that build on automated information retrieval.

Revolutionizing health monitoring: Integrating transformer models with multi-head attention for precise human activity recognition using wearable devices.

BACKGROUND: A daily activity routine is vital for overall health and well-being, supporting physical and mental fitness. Consistent physical activity is linked to a multitude of benefits for the body, mind, and emotions, playing a key role in raising a healthy lifestyle. The use of wearable devices has become essential in the realm of health and fitness, facilitating the monitoring of daily activities. While convolutional neural networks (CNN) have proven effective, challenges remain in quickly adapting to a variety of activities. OBJECTIVE: This study aimed to develop a model for precise recognition of human activities to revolutionize health monitoring by integrating transformer models with multi-head attention for precise human activity recognition using wearable devices. METHODS: The Human Activity Recognition (HAR) algorithm uses deep learning to classify human activities using spectrogram data. It uses a pretrained convolution neural network (CNN) with a MobileNetV2 model to extract features, a dense residual transformer network (DRTN), and a multi-head multi-level attention architecture (MH-MLA) to capture time-related patterns. The model then blends information from both layers through an adaptive attention mechanism and uses a SoftMax function to provide classification probabilities for various human activities. RESULTS: The integrated approach, combining pretrained CNN with transformer models to create a thorough and effective system for recognizing human activities from spectrogram data, outperformed these methods in various datasets - HARTH, KU-HAR, and HuGaDB produced accuracies of 92.81%, 97.98%, and 95.32%, respectively. This suggests that the integration of diverse methodologies yields good results in capturing nuanced human activities across different activities. The comparison analysis showed that the integrated system consistently performs better for dynamic human activity recognition datasets. CONCLUSION: In conclusion, maintaining a routine of daily activities is crucial for overall health and well-being. Regular physical activity contributes substantially to a healthy lifestyle, benefiting both the body and the mind. The integration of wearable devices has simplified the monitoring of daily routines. This research introduces an innovative approach to human activity recognition, combining the CNN model with a dense residual transformer network (DRTN) with multi-head multi-level attention (MH-MLA) within the transformer architecture to enhance its capability.

Effect of Primary Cable Position on Accuracy in Non-Toroidal-Shaped Pass-Through Current Transformer.

Non-toroidal-shaped primary pass-through protection current transformers (CTs) are used to measure high currents. Their design provides them with a big airgap that allow the passing of several cables per phase though them, which is the main advantage versus toroidal types, as the number of CTs required to measure the whole phase current is drastically reduced. The cables passed through the transformer window can be in several positions. As the isolines of the magnetic field generated by the primary currents are centered in the cables, if these cables are not centered in the transformer window, then the magnetic field will be non-uniform along the transformer core. Consequently, local saturations can appear if the cables are not properly disposed, causing the malfunction of the CT. In this paper, the performance of a non-toroidal-shaped protection CT is studied. This research is focused on the influence of the cable position on possible partial saturations of the CT when it is operating near to its accuracy limit. Depending on the cable position, the ratio of the primary and secondary currents can depart from the assigned ratio. The validation of this phenomenon was carried out via finite element analysis (FEA), showing that partial transformer core saturations appear in areas of the magnetic core close to the cable. By applying FEA, the admissible accuracy region for cable positioning inside the CT is also delimited. Finally, the simulation results are ratified with experimental tests performed in non-toroidal protection CTs, varying the primary cables' positions, which are subjected to currents up to 5 kA, achieving satisfactory results. From this analysis, installation recommendations are given.

RA-XTNet: A Novel CNN Model to Predict Rheumatoid Arthritis from Hand Radiographs and Thermal Images: A Comparison with CNN Transformer and Quantum Computing.

The aim and objective of the research are to develop an automated diagnosis system for the prediction of rheumatoid arthritis (RA) based on artificial intelligence (AI) and quantum computing for hand radiographs and thermal images. The hand radiographs and thermal images were segmented using a UNet++ model and color-based k-means clustering technique, respectively. The attributes from the segmented regions were generated using the Speeded-Up Robust Features (SURF) feature extractor and classification was performed using k-star and Hoeffding classifiers. For the ground truth and the predicted test image, the study utilizing UNet++ segmentation achieved a pixel-wise accuracy of 98.75%, an intersection over union (IoU) of 0.87, and a dice coefficient of 0.86, indicating a high level of similarity. The custom RA-X-ray thermal imaging (XTNet) surpassed all the models for the detection of RA with a classification accuracy of 90% and 93% for X-ray and thermal imaging modalities, respectively. Furthermore, the study employed quantum support vector machine (QSVM) as a quantum computing approach which yielded an accuracy of 93.75% and 87.5% for the detection of RA from hand X-ray and thermal images. In addition, vision transformer (ViT) was employed to classify RA which obtained an accuracy of 80% for hand X-rays and 90% for thermal images. Thus, depending on the performance measures, the RA-XTNet model can be used as an effective automated diagnostic method to diagnose RA accurately and rapidly in hand radiographs and thermal images.

Investigation of Partial Discharge Transformation Characteristics in Polyimide (PI) Insulations under High-Frequency Electric Stress.

This research delves into the primary issue of polyimide (PI) insulation failures in high-frequency power transformers (HFPTs) by scrutinizing partial discharge development under high-frequency electrical stress. This study employs an experimental approach coupled with a plasma simulation model for a ball-sphere electrode structure. The simulation model integrates the particle transport equation, Poisson equation, and complex chemical reactions to ascertain microscopic parameters, including plasma distribution, electric field, electron density, electron temperature, surface, and space charge distribution. The effect of the voltage polarity and electrical energy on the PD process is also discussed. The contact point plays a pivotal role in triggering partial discharges and culminating in the breakdown of PI insulation. Asymmetry phenomena were found between positive and negative half-cycles by analyzing the PD data stage by stage. A significant number of PDs increased at every stage and the PD amplitude was higher during the negative cycle at the initial stage, but in later stages, the PD amplitude was found to be higher in the positive half-cycle, and scanning electron microscopy (SEM) revealed that the maximum damage occurred near the contact point junction. The simulation results show that the plasma initially accumulates the electron density near the contact point junction. Under the action of the electric field, plasma starts traveling at the PI surface outward from the contact point. Before the PD activity, all parameters have higher values in the plasma head. The microscopic parameters reveal maximum values near the contact point junction, during PD activities where significant damage takes place. These parameter distributions exhibit a decreasing trend over time as when the PD activity ends. The model's predictions are consistent with the experimental data. The paper lays the foundation for future research in polymer insulation design under high-frequency electrical stress.

The Potential of Gemini and GPTs for Structured Report Generation based on Free-Text 18F-FDG PET/CT Breast Cancer Reports.

RATIONALE AND OBJECTIVE: To compare the performance of large language model (LLM) based Gemini and Generative Pre-trained Transformers (GPTs) in data mining and generating structured reports based on free-text PET/CT reports for breast cancer after user-defined tasks. MATERIALS AND METHODS: Breast cancer patients (mean age, 50 years ± 11 [SD]; all female) who underwent consecutive 18F-FDG PET/CT for follow-up between July 2005 and October 2023 were retrospectively included in the study. A total of twenty reports from 10 patients were used to train user-defined text prompts for Gemini and GPTs, by which structured PET/CT reports were generated. The natural language processing (NLP) generated structured reports and the structured reports annotated by nuclear medicine physicians were compared in terms of data extraction accuracy and capacity of progress decision-making. Statistical methods, including chi-square test, McNemar test and paired samples t-test, were employed in the study. RESULTS: The structured PET/CT reports for 131 patients were generated by using the two NLP techniques, including Gemini and GPTs. In general, GPTs exhibited superiority over Gemini in data mining in terms of primary lesion size (89.6% vs. 53.8%, p < 0.001) and metastatic lesions (96.3% vs 89.6%, p < 0.001). Moreover, GPTs outperformed Gemini in making decision for progress (p < 0.001) and semantic similarity (F1 score 0.930 vs 0.907, p < 0.001) for reports. CONCLUSION: GPTs outperformed Gemini in generating structured reports based on free-text PET/CT reports, which is potentially applied in clinical practice. DATA AVAILABILITY: The data used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Augmented histopathology: Enhancing colon cancer detection through deep learning and ensemble techniques.

Colon cancer poses a significant threat to human life with a high global mortality rate. Early and accurate detection is crucial for improving treatment quality and the survival rate. This paper presents a comprehensive approach to enhance colon cancer detection and classification. The histopathological images are gathered from the CRC-VAL-HE-7K dataset. The images undergo preprocessing to improve quality, followed by augmentation to increase dataset size and enhance model generalization. A deep learning based transformer model is designed for efficient feature extraction and enhancing classification by incorporating a convolutional neural network (CNN). A cross-transformation model captures long-range dependencies between regions, and an attention mechanism assigns weights to highlight crucial features. To boost classification accuracy, a Siamese network distinguishes colon cancer tissue classes based on probabilities. Optimization algorithms fine-tune model parameters, categorizing colon cancer tissues into different classes. The multi-class classification performance is evaluated in the experimental evaluation, which demonstrates that the proposed model provided highest accuracy rate of 98.84%. In this research article, the proposed method achieved better performance in all analyses by comparing with other existing methods. RESEARCH HIGHLIGHTS: Deep learning-based techniques are proposed. DL methods are used to enhance colon cancer detection and classification. CRC-VAL-HE-7K dataset is utilized to enhance image quality. Hybrid particle swarm optimization (PSO) and dwarf mongoose optimization (DMO) are used. The deep learning models are tuned by implementing the PSO-DMO algorithm.

AVaTER: Fusing Audio, Visual, and Textual Modalities Using Cross-Modal Attention for Emotion Recognition.

Multimodal emotion classification (MEC) involves analyzing and identifying human emotions by integrating data from multiple sources, such as audio, video, and text. This approach leverages the complementary strengths of each modality to enhance the accuracy and robustness of emotion recognition systems. However, one significant challenge is effectively integrating these diverse data sources, each with unique characteristics and levels of noise. Additionally, the scarcity of large, annotated multimodal datasets in Bangla limits the training and evaluation of models. In this work, we unveiled a pioneering multimodal Bangla dataset, MAViT-Bangla (Multimodal Audio Video Text Bangla dataset). This dataset, comprising 1002 samples across audio, video, and text modalities, is a unique resource for emotion recognition studies in the Bangla language. It features emotional categories such as anger, fear, joy, and sadness, providing a comprehensive platform for research. Additionally, we developed a framework for audio, video and textual emotion recognition (i.e., AVaTER) that employs a cross-modal attention mechanism among unimodal features. This mechanism fosters the interaction and fusion of features from different modalities, enhancing the model's ability to capture nuanced emotional cues. The effectiveness of this approach was demonstrated by achieving an F1-score of 0.64, a significant improvement over unimodal methods.

First-Principles Investigation on Ru-Doped Janus WSSe Monolayer for Adsorption of Dissolved Gases in Transformer Oil: A Novel Sensing Candidate Exploration.

Using first-principles theory, this work purposes Ru-doped Janus WSSe (Ru-WSSe) monolayer as a potential gas sensor for detection of three typical gas species (CO, C2H2, and C2H4), in order to evaluate the operation status of the oil-immersed transformers. The Ru-doping behavior on the WSSe surface is analyzed, giving rise to the preferred doping site by the replacement of a Se atom with the formation energy of 0.01 eV. The gas adsorption of three gas species onto the Ru-WSSe monolayer is conducted, and chemisorption is identified for all three gas systems with the adsorption energy following the order: CO (-2.22 eV) > C2H2 (-2.01 eV) > C2H4 (-1.70 eV). Also, the modulated electronic properties and the frontier molecular orbital are investigated to uncover the sensing mechanism of Ru-WSSe monolayer upon three typical gases. Results reveal that the sensing responses of the Ru-WSSe monolayer, based on the variation of energy gap, to CO, C2H2, and C2H4 molecules are calculated to be 1.67 × 106, 2.10 × 105, and 9.61 × 103, respectively. Finally, the impact of the existence of O2 molecule for gas adsorption and sensing is also analyzed to uncover the potential of Ru-WSSe monolayer for practical application in the air atmosphere. The obtained high electrical responses manifest strong potential as a resistive sensor for detection of three gases. The findings hold practical implications for the development of novel gas sensing materials based on Janus WSSe monolayer. We anticipate that our results will inspire further research in this domain, particularly for applications in electrical engineering where the reliable detection of fault gases is paramount for maintaining the integrity and safety of power systems.

Dmg2Former-AR: Vision Transformers with Adaptive Rescaling for High-Resolution Structural Visual Inspection.

Developments in drones and imaging hardware technology have opened up countless possibilities for enhancing structural condition assessments and visual inspections. However, processing the inspection images requires considerable work hours, leading to delays in the assessment process. This study presents a semantic segmentation architecture that integrates vision transformers with Laplacian pyramid scaling networks, enabling rapid and accurate pixel-level damage detection. Unlike conventional methods that often lose critical details through resampling or cropping high-resolution images, our approach preserves essential inspection-related information such as microcracks and edges using non-uniform image rescaling networks. This innovation allows for detailed damage identification of high-resolution images while significantly reducing the computational demands. Our main contributions in this study are: (1) proposing two rescaling networks that together allow for processing high-resolution images while significantly reducing the computational demands; and (2) proposing Dmg2Former, a low-resolution segmentation network with a Swin Transformer backbone that leverages the saved computational resources to produce detailed visual inspection masks. We validate our method through a series of experiments on publicly available visual inspection datasets, addressing various tasks such as crack detection and material identification. Finally, we examine the computational efficiency of the adaptive rescalers in terms of multiply-accumulate operations and GPU-memory requirements.

Ensemble Techniques for Robust Fake News Detection: Integrating Transformers, Natural Language Processing, and Machine Learning.

The proliferation of fake news across multiple modalities has emerged as a critical challenge in the modern information landscape, necessitating advanced detection methods. This study proposes a comprehensive framework for fake news detection integrating text, images, and videos using machine learning and deep learning techniques. The research employs a dual-phased methodology, first analyzing textual data using various classifiers, then developing a multimodal approach combining BERT for text analysis and a modified CNN for visual data. Experiments on the ISOT fake news dataset and MediaEval 2016 image verification corpus demonstrate the effectiveness of the proposed models. For textual data, the Random Forest classifier achieved 99% accuracy, outperforming other algorithms. The multimodal approach showed superior performance compared to baseline models, with a 3.1% accuracy improvement over existing multimodal techniques. This research contributes to the ongoing efforts to combat misinformation by providing a robust, adaptable framework for detecting fake news across different media formats, addressing the complexities of modern information dissemination and manipulation.

Neural Conversational Agent for Weight Loss Counseling: Protocol for an Implementation and Feasibility Study.

BACKGROUND: Obesity is a common, serious and costly chronic disease. Current clinical practice guidelines recommend that providers augment the longitudinal care of people living with obesity with consistent support for the development of self-efficacy and motivation to modify their lifestyle behaviors. Lifestyle behavior change aligns with the goals of motivational interviewing (MI), a client-centered yet directive counseling modality. However, training health care providers to be proficient in MI is expensive and time-consuming, resulting in a lack of trained counselors and limiting the widespread adoption of MI in clinical practice. Artificial intelligence (AI) counselors accessible via the internet can help circumvent these barriers. OBJECTIVE: The primary objective is to explore the feasibility of conducting unscripted MI-consistent counseling using Neural Agent for Obesity Motivational Interviewing (NAOMI), a large language model (LLM)-based web app for weight loss counseling. The secondary objectives are to test the acceptability and usability of NAOMI's counseling and examine its ability to shift motivational precursors in a sample of patients with overweight and obesity recruited from primary care clinics. METHODS: NAOMI will be developed based on recent advances in deep learning in four stages. In stages 1 and 2, NAOMI will be implemented using an open-source foundation LLM and (1) few-shot learning based on a prompt with task-specific instructions and (2) domain adaptation strategy based on fine-tuning LLM using a large corpus of general psychotherapy and MI treatment transcripts. In stages 3 and 4, we will refine the best of these 2 approaches. Each NAOMI version will be evaluated using a mixed methods approach in which 10 adults (18-65 years) meeting the criteria for overweight or obesity (25.0≥BMI≤39.9) interact with NAOMI and provide feedback. NAOMI's fidelity to the MI framework will be assessed using the Motivational Interviewing Treatment Integrity scale. Participants' general perceptions of AI conversational agents and NAOMI specifically will be assessed via Pre- and Post-Interaction Questionnaires. Motivational precursors, such as participants' confidence, importance, and readiness for changing lifestyle behaviors (eg, diet and activity), will be measured before and after the interaction, and 1 week later. A qualitative analysis of changes in the measures of perceptions of AI agents and counselors and motivational precursors will be performed. Participants will rate NAOMI's usability and empathic skills post interaction via questionnaire-based assessments along with providing feedback about their experience with NAOMI via a qualitative interview. RESULTS: NAOMI (version 1.0) has been developed. Participant recruitment will commence in September 2024. Data collection activities are expected to conclude in May 2025. CONCLUSIONS: If proven effective, LLM-based counseling agents can become a cost-effective approach for addressing the obesity epidemic at a public health level. They can also have a broad, transformative impact on the delivery of MI and other psychotherapeutic treatment modalities extending their reach and broadening access. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/60361.

Assessing the utility of large language models for phenotype-driven gene prioritization in the diagnosis of rare genetic disease.

Phenotype-driven gene prioritization is fundamental to diagnosing rare genetic disorders. While traditional approaches rely on curated knowledge graphs with phenotype-gene relations, recent advancements in large language models (LLMs) promise a streamlined text-to-gene solution. In this study, we evaluated five LLMs, including two generative pre-trained transformers (GPT) series and three Llama2 series, assessing their performance across task completeness, gene prediction accuracy, and adherence to required output structures. We conducted experiments, exploring various combinations of models, prompts, phenotypic input types, and task difficulty levels. Our findings revealed that the best-performed LLM, GPT-4, achieved an average accuracy of 17.0% in identifying diagnosed genes within the top 50 predictions, which still falls behind traditional tools. However, accuracy increased with the model size. Consistent results were observed over time, as shown in the dataset curated after 2023. Advanced techniques such as retrieval-augmented generation (RAG) and few-shot learning did not improve the accuracy. Sophisticated prompts were more likely to enhance task completeness, especially in smaller models. Conversely, complicated prompts tended to decrease output structure compliance rate. LLMs also achieved better-than-random prediction accuracy with free-text input, though performance was slightly lower than with standardized concept input. Bias analysis showed that highly cited genes, such as BRCA1, TP53, and PTEN, are more likely to be predicted. Our study provides valuable insights into integrating LLMs with genomic analysis, contributing to the ongoing discussion on their utilization in clinical workflows.

Quantifying decision support level of explainable automatic classification of diagnoses in Spanish medical records.

BACKGROUND AND OBJECTIVE: In the realm of automatic Electronic Health Records (EHR) classification according to the International Classification of Diseases (ICD) there is a notable gap of non-black box approaches and more in Spanish, which is also frequently ignored in clinical language classification. An additional gap in explainability pertains to the lack of standardized metrics for evaluating the degree of explainability offered by distinct techniques. METHODS: We address the classification of Spanish electronic health records, using methods to explain the predictions and improve the decision support level. We also propose Leberage a novel metric to quantify the decision support level of the explainable predictions. We aim to assess the explanatory ability derived from three model-independent methods based on different theoretical frameworks: SHapley Additive exPlanations (SHAP), Local Interpretable Model-agnostic Explanations (LIME), and Integrated Gradients (IG). We develop a system based on longformers that can process long documents and then use the explainability methods to extract the relevant segments of text in the EHR that motivated each ICD. We then measure the outcome of the different explainability methods by implementing a novel metric. RESULTS: Our results beat those that carry out the same task by 7%. In terms of explainability degree LIME appears as a stronger technique compared to IG and SHAP. DISCUSSION: Our research reveals that the explored techniques are useful for explaining the output of black box models as the longformer. In addition, the proposed metric emerges as a good choice to quantify the contribution of explainability techniques.

Mammography classification with multi-view deep learning techniques: Investigating graph and transformer-based architectures.

The potential and promise of deep learning systems to provide an independent assessment and relieve radiologists' burden in screening mammography have been recognized in several studies. However, the low cancer prevalence, the need to process high-resolution images, and the need to combine information from multiple views and scales still pose technical challenges. Multi-view architectures that combine information from the four mammographic views to produce an exam-level classification score are a promising approach to the automated processing of screening mammography. However, training such architectures from exam-level labels, without relying on pixel-level supervision, requires very large datasets and may result in suboptimal accuracy. Emerging architectures such as Visual Transformers (ViT) and graph-based architectures can potentially integrate ipsi-lateral and contra-lateral breast views better than traditional convolutional neural networks, thanks to their stronger ability of modeling long-range dependencies. In this paper, we extensively evaluate novel transformer-based and graph-based architectures against state-of-the-art multi-view convolutional neural networks, trained in a weakly-supervised setting on a middle-scale dataset, both in terms of performance and interpretability. Extensive experiments on the CSAW dataset suggest that, while transformer-based architecture outperform other architectures, different inductive biases lead to complementary strengths and weaknesses, as each architecture is sensitive to different signs and mammographic features. Hence, an ensemble of different architectures should be preferred over a winner-takes-all approach to achieve more accurate and robust results. Overall, the findings highlight the potential of a wide range of multi-view architectures for breast cancer classification, even in datasets of relatively modest size, although the detection of small lesions remains challenging without pixel-wise supervision or ad-hoc networks.

Arabic dialect identification in social media: A hybrid model with transformer models and BiLSTM.

Arabic Dialect Identification (ADI) is a challenging task in natural language processing applications due to its diversity and regional variations. Despite previous efforts, this task is still difficult. Therefore, this study aims to use transformers to address the issue of ADI on social media. A combination of two hybrid models is proposed in this study: one that combines Bidirectional Long Short-Term Memory (BiLSTM) with CAMeLBERT, and the second model that combines the BiLSTM model with AlBERT. In addition, a novel dataset comprising 121,289 user-generated comments from various social media network platforms and four major Arabic dialects (Egyptian, Jordanian, Gulf and Yemeni) was introduced. Several experiments have been conducted using conventional Machine Learning Classifiers (MLCs) and Deep Learning Models (DLMs) as baselines to measure the performance and effectiveness of the proposed models. In addition, binary classification is performed between two dialects to determine which are closest to each other. The performance of the model is measured using common metrics such as precision, recall, F-score and F-measure. Experiment results demonstrate the superior efficiency of the proposed hybrid models in ADI, CAMeLBERT with BiLSTM and ALBERT with BiLSTM, which both recorded an accuracy of 87.67 % and 86.51 %, respectively.