Single-channel EOG sleep staging on a heterogeneous cohort of subjects with sleep disorders.

Objective.Sleep staging based on full polysomnography is the gold standard in the diagnosis of many sleep disorders. It is however costly, complex, and obtrusive due to the use of multiple electrodes. Automatic sleep staging based on single-channel electro-oculography (EOG) is a promising alternative, requiring fewer electrodes which could be self-applied below the hairline. EOG sleep staging algorithms are however yet to be validated in clinical populations with sleep disorders.Approach.We utilized the SOMNIA dataset, comprising 774 recordings from subjects with various sleep disorders, including insomnia, sleep-disordered breathing, hypersomnolence, circadian rhythm disorders, parasomnias, and movement disorders. The recordings were divided into train (574), validation (100), and test (100) groups. We trained a neural network that integrated transformers within a U-Net backbone. This design facilitated learning of arbitrary-distance temporal relationships within and between the EOG and hypnogram.Main results.For 5-class sleep staging, we achieved median accuracies of 85.0% and 85.2% and Cohen's kappas of 0.781 and 0.796 for left and right EOG, respectively. The performance using the right EOG was significantly better than using the left EOG, possibly because in the recommended AASM setup, this electrode is located closer to the scalp. The proposed model is robust to the presence of a variety of sleep disorders, displaying no significant difference in performance for subjects with a certain sleep disorder compared to those without.Significance.The results show that accurate sleep staging using single-channel EOG can be done reliably for subjects with a variety of sleep disorders.

Results of the 2023 ISBI challenge to reduce GABA-edited MRS acquisition time.

PURPOSE: Use a conference challenge format to compare machine learning-based gamma-aminobutyric acid (GABA)-edited magnetic resonance spectroscopy (MRS) reconstruction models using one-quarter of the transients typically acquired during a complete scan. METHODS: There were three tracks: Track 1: simulated data, Track 2: identical acquisition parameters with in vivo data, and Track 3: different acquisition parameters with in vivo data. The mean squared error, signal-to-noise ratio, linewidth, and a proposed shape score metric were used to quantify model performance. Challenge organizers provided open access to a baseline model, simulated noise-free data, guides for adding synthetic noise, and in vivo data. RESULTS: Three submissions were compared. A covariance matrix convolutional neural network model was most successful for Track 1. A vision transformer model operating on a spectrogram data representation was most successful for Tracks 2 and 3. Deep learning (DL) reconstructions with 80 transients achieved equivalent or better SNR, linewidth and fit error compared to conventional 320 transient reconstructions. However, some DL models optimized linewidth and SNR without actually improving overall spectral quality, indicating a need for more robust metrics. CONCLUSION: DL-based reconstruction pipelines have the promise to reduce the number of transients required for GABA-edited MRS.

Ultrasound Viscoelastography by Acoustic Radiation Force: A State-of-the-Art Review.

Ultrasound elastography (USE) is a promising tool for tissue characterization as several diseases result in alterations of tissue structure and composition, which manifest as changes in tissue mechanical properties. By imaging the tissue response to an applied mechanical excitation, USE mimics the manual palpation performed by clinicians to sense the tissue elasticity for diagnostic purposes. Next to elasticity, viscosity has recently been investigated as an additional, relevant, diagnostic biomarker. Moreover, since biological tissues are inherently viscoelastic, accounting for viscosity in the tissue characterization process enhances the accuracy of the elasticity estimation. Recently, methods exploiting different acquisition and processing techniques have been proposed to perform ultrasound viscoelastography. After introducing the physics describing viscoelasticity, a comprehensive overview of the currently available USE acquisition techniques is provided, followed by a structured review of the existing viscoelasticity estimators classified according to the employed processing technique. These estimators are further reviewed from a clinical usage perspective, and current outstanding challenges are discussed.

Maximum a posteriori detection of heartbeats from a chest-worn accelerometer.

Objective. Unobtrusive long-term monitoring of cardiac parameters is important in a wide variety of clinical applications, such as the assesment of acute illness severity and unobtrusive sleep monitoring. Here we determined the accuracy and robustness of heartbeat detection by an accelerometer worn on the chest.Approach. We performed overnight recordings in 147 individuals (69 female, 78 male) referred to two sleep centers. Two methods for heartbeat detection in the acceleration signal were compared: one previously described approach, based on local periodicity, and a novel extended method incorporating maximumaposterioriestimation and a Markov decision process to approach an optimal solution.Main results. The maximumaposterioriestimation significantly improved performance, with a mean absolute error for the estimation of inter-beat intervals of only 3.5 ms, and 95% limits of agreement of -1.7 to +1.0 beats per minute for heartrate measurement. Performance held during posture changes and was only weakly affected by the presence of sleep disorders and demographic factors.Significance. The new method may enable the use of a chest-worn accelerometer in a variety of applications such as ambulatory sleep staging and in-patient monitoring.

Dehazing Ultrasound using Diffusion Models.

Echocardiography has been a prominent tool for the diagnosis of cardiac disease. However, these diagnoses can be heavily impeded by poor image quality. Acoustic clutter emerges due to multipath reflections imposed by layers of skin, subcutaneous fat, and intercostal muscle between the transducer and heart. As a result, haze and other noise artifacts pose a real challenge to cardiac ultrasound imaging. In many cases, especially with difficult-to-image patients such as patients with obesity, a diagnosis from B-Mode ultrasound imaging is effectively rendered unusable, forcing sonographers to resort to contrast-enhanced ultrasound examinations or refer patients to other imaging modalities. Tissue harmonic imaging has been a popular approach to combat haze, but in severe cases is still heavily impacted by haze. Alternatively, denoising algorithms are typically unable to remove highly structured and correlated noise, such as haze. It remains a challenge to accurately describe the statistical properties of structured haze, and develop an inference method to subsequently remove it. Diffusion models have emerged as powerful generative models and have shown their effectiveness in a variety of inverse problems. In this work, we present a joint posterior sampling framework that combines two separate diffusion models to model the distribution of both clean ultrasound and haze in an unsupervised manner. Furthermore, we demonstrate techniques for effectively training diffusion models on radio-frequency ultrasound data and highlight the advantages over image data. Experiments on both in-vitro and in-vivo cardiac datasets show that the proposed dehazing method effectively removes haze while preserving signals from weakly reflected tissue.

Artificial intelligence based cardiotocogram assessment during labor.

OBJECTIVE: To assess whether artificial intelligence, inspired by clinical decision-making procedures in delivery rooms, can correctly interpret cardiotocographic tracings and distinguish between normal and pathological events. STUDY DESIGN: A method based on artificial intelligence was developed to determine whether a cardiotocogram shows a normal response of the fetal heart rate to uterine activity (UA). For a given fetus and given the UA and previous FHR, the method predicts a fetal heart rate response, under the assumption that the fetus is still in good condition and based on how that specific fetus has responded so far. We hypothesize that this method, when having only learned from fetuses born in good condition, is incapable of predicting the response of a compromised fetus or an episode of transient fetal distress. The (in)capability of the method to predict the fetal heart rate response would then yield a method that can help to assess fetal condition when the obstetrician is in doubt. Cardiotocographic data of 678 deliveries during labor were selected based on a healthy outcome just after birth. The method was trained on the cardiotocographic data of 548 fetuses of this group to learn their heart rate response. Subsequently it was evaluated on 87 fetuses, by assessing whether the method was able to predict their heart rate responses. The remaining 43 cardiotocograms were segment-by-segment annotated by three experienced gynecologists, indicating normal, suspicious, and pathological segments, while having access to the full recording and neonatal outcome. This future knowledge makes the expert annotations of a quality that is unachievable during live interpretation. RESULTS: The comparison between abnormalities detected by the method (only using past and present input) and the annotated CTG segments by gynecologists (also looking at future input) yields an area under the curve of 0.96 for the distinction between normal and pathological events in majority-voted annotations. CONCLUSION: The developed method can distinguish between normal and pathological events in near real-time, with a performance close to the agreement between three gynecologists with access to the entire CTG tracing and fetal outcome. The method has a strong potential to support clinicians in assessing fetal condition in clinical practice.

Temporal dynamics of awakenings from slow-wave sleep in non-rapid eye movement parasomnia.

Non-rapid eye movement parasomnia disorders, also called disorders of arousal, are characterized by abnormal nocturnal behaviours, such as confusional arousals or sleep walking. Their pathophysiology is not yet fully understood, and objective diagnostic criteria are lacking. It is known, however, that behavioural episodes occur mostly in the beginning of the night, after an increase in slow-wave activity during slow-wave sleep. A better understanding of the prospect of such episodes may lead to new insights in the underlying mechanisms and eventually facilitate objective diagnosis. We investigated temporal dynamics of transitions from slow-wave sleep of 52 patients and 79 controls. Within the patient group, behavioural and non-behavioural N3 awakenings were distinguished. Patients showed a higher probability to wake up after an N3 bout ended than controls, and this probability increased with N3 bout duration. Bouts longer than 15 min resulted in an awakening in 73% and 34% of the time in patients and controls, respectively. Behavioural episodes reduced over sleep cycles due to a reduction in N3 sleep and a reducing ratio between behavioural and non-behavioural awakenings. In the first two cycles, N3 bouts prior to non-behavioural awakenings were significantly shorter than N3 bouts advancing behavioural awakenings in patients, and N3 awakenings in controls. Our findings provide insights in the timing and prospect of both behavioural and non-behavioural awakenings from N3, which may result in prediction and potentially prevention of behavioural episodes. This work, moreover, leads to a more complete characterization of a prototypical hypnogram of parasomnias, which could facilitate diagnosis.

Modeling the Impact of Inter-Rater Disagreement on Sleep Statistics Using Deep Generative Learning.

Sleep staging is the process by which an overnight polysomnographic measurement is segmented into epochs of 30 seconds, each of which is annotated as belonging to one of five discrete sleep stages. The resulting scoring is graphically depicted as a hypnogram, and several overnight sleep statistics are derived, such as total sleep time and sleep onset latency. Gold standard sleep staging as performed by human technicians is time-consuming, costly, and comes with imperfect inter-scorer agreement, which also results in inter-scorer disagreement about the overnight statistics. Deep learning algorithms have shown promise in automating sleep scoring, but struggle to model inter-scorer disagreement in sleep statistics. To that end, we introduce a novel technique using conditional generative models based on Normalizing Flows that permits the modeling of the inter-rater disagreement of overnight sleep statistics, termed U-Flow. We compare U-Flow to other automatic scoring methods on a hold-out test set of 70 subjects, each scored by six independent scorers. The proposed method achieves similar sleep staging performance in terms of accuracy and Cohen's kappa on the majority-voted hypnograms. At the same time, U-Flow outperforms the other methods in terms of modeling the inter-rater disagreement of overnight sleep statistics. The consequences of inter-rater disagreement about overnight sleep statistics may be great, and the disagreement potentially carries diagnostic and scientifically relevant information about sleep structure. U-Flow is able to model this disagreement efficiently and can support further investigations into the impact inter-rater disagreement has on sleep medicine and basic sleep research.

The impact of angles of insonation on left and right ventricular global longitudinal strain estimation in fetal speckle tracking echocardiography.

OBJECTIVES: Two-dimensional speckle tracking echocardiography has been considered an angle-independent modality. However, current literature is limited and inconclusive on the actual impact of angle of insonation on strain values. Therefore, the primary objective of this study was to assess the impact of angles of insonation on the estimation of fetal left ventricular and right ventricular global longitudinal strain. Secondarily, the impact of different definitions for angles of insonation was investigated in a sensitivity analysis. METHODS: This is a retrospective analysis of a prospective longitudinal cohort study with 124 healthy subjects. The analyses were based on the four-chamber view ultrasound clips taken between 18+0 and 21+6 weeks of gestation. Angles of insonation were categorized into three groups: up/down, oblique and perpendicular. The mean fetal left and right ventricular and global longitudinal strain values corresponding to these three groups were compared by an ANOVA test corrected for heteroscedasticity. RESULTS: Fetal left and right ventricular global longitudinal strain values were not statistically different between the three angles of insonation (p-value >0.062 and >0.149, respectively). When applying another definition for angles of insonation in the sensitivity analysis, the mean left ventricular global longitudinal strain value was significantly decreased for the oblique compared to the up/down angle of insonation (p-value 0.041). CONCLUSIONS: There is no evidence of a difference in fetal left and right ventricular global longitudinal strain between the different angles of insonation in fetal two-dimensional speckle tracking echocardiography.

A review of machine learning applications for the proton MR spectroscopy workflow.

This literature review presents a comprehensive overview of machine learning (ML) applications in proton MR spectroscopy (MRS). As the use of ML techniques in MRS continues to grow, this review aims to provide the MRS community with a structured overview of the state-of-the-art methods. Specifically, we examine and summarize studies published between 2017 and 2023 from major journals in the MR field. We categorize these studies based on a typical MRS workflow, including data acquisition, processing, analysis, and artificial data generation. Our review reveals that ML in MRS is still in its early stages, with a primary focus on processing and analysis techniques, and less attention given to data acquisition. We also found that many studies use similar model architectures, with little comparison to alternative architectures. Additionally, the generation of artificial data is a crucial topic, with no consistent method for its generation. Furthermore, many studies demonstrate that artificial data suffers from generalization issues when tested on in vivo data. We also conclude that risks related to ML models should be addressed, particularly for clinical applications. Therefore, output uncertainty measures and model biases are critical to investigate. Nonetheless, the rapid development of ML in MRS and the promising results from the reviewed studies justify further research in this field.

Shear-Wave Particle-Velocity Estimation and Enhancement Using a Multi-Resolution Convolutional Neural Network.

OBJECTIVE: Tissue mechanical properties are valuable markers for tissue characterization, aiding in the detection and staging of pathologies. Shear wave elastography (SWE) offers a quantitative assessment of tissue mechanical characteristics based on the SW propagation profile, which is derived from the SW particle motion. Improving the signal-to-noise ratio (SNR) of the SW particle motion would directly enhance the accuracy of the material property estimates such as elasticity or viscosity. METHODS: In this paper, we present a 3-D multi-resolution convolutional neural network (MRCNN) to perform improved estimation of the SW particle velocity Vz. Additionally, we propose a novel approach to generate training data from real acquisitions, providing high SNR ground truth target data, one-to-one paired to inputs that are corrupted with real-world noise and disturbances. DISCUSSION: By testing the network on in vitro data acquired from a commercial breast elastography phantom, we show that the MRCNN outperforms Loupas' autocorrelation algorithm with an improved SNR of 4.47 dB for the Vz signals, a two-fold decrease in the standard deviation of the downstream elasticity estimates, and a two-fold increase in the contrast-to-noise ratio of the elasticity maps. The generalizability of the network was further demonstrated with a set of ex vivo porcine liver data. CONCLUSION: The proposed MRCNN outperforms the standard autocorrelation method, in particular in low SNR regimes.

Ultrasound Signal Processing: From Models to Deep Learning.

Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions. Conventionally, reconstruction algorithms have been derived from physical principles. These algorithms rely on assumptions and approximations of the underlying measurement model, limiting image quality in settings where these assumptions break down. Conversely, more sophisticated solutions based on statistical modeling or careful parameter tuning or derived from increased model complexity can be sensitive to different environments. Recently, deep learning-based methods, which are optimized in a data-driven fashion, have gained popularity. These model-agnostic techniques often rely on generic model structures and require vast training data to converge to a robust solution. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning and exploiting domain knowledge. These model-based solutions yield high robustness and require fewer parameters and training data than conventional neural networks. In this work we provide an overview of these techniques from the recent literature and discuss a wide variety of ultrasound applications. We aim to inspire the reader to perform further research in this area and to address the opportunities within the field of ultrasound signal processing. We conclude with a future perspective on model-based deep learning techniques for medical ultrasound.

Interpretation and further development of the hypnodensity representation of sleep structure.

Objective.The recently-introduced hypnodensity graph provides a probability distribution over sleep stages per data window (i.e. an epoch). This work explored whether this representation reveals continuities that can only be attributed to intra- and inter-rater disagreement of expert scorings, or also to co-occurrence of sleep stage-dependent features within one epoch.Approach.We proposed a simplified model for time series like the ones measured during sleep, and a second model to describe the annotation process by an expert. Generating data according to these models, enabled controlled experiments to investigate the interpretation of the hypnodensity graph. Moreover, the influence of both the supervised training strategy, and the used softmax non-linearity were investigated. Polysomnography recordings of 96 healthy sleepers (of which 11 were used as independent test set), were subsequently used to transfer conclusions to real data.Main results.A hypnodensity graph, predicted by a supervised neural classifier, represents the probability with which the sleep expert(s) assigned a label to an epoch. It thus reflects annotator behavior, and is thereby only indirectly linked to the ratio of sleep stage-dependent features in the epoch. Unsupervised training was shown to result in hypnodensity graph that were slightly less dependent on this annotation process, resulting in, on average, higher-entropy distributions over sleep stages (Hunsupervised= 0.41 versusHsupervised= 0.29). Moreover, pre-softmax predictions were, for both training strategies, found to better reflect the ratio of sleep stage-dependent characteristics in an epoch, as compared to the post-softmax counterparts (i.e. the hypnodensity graph). In real data, this was observed from the linear relation between pre-softmax N3 predictions and the amount of delta power.Significance.This study provides insights in, and proposes new, representations of sleep that may enhance our comprehension about sleep and sleep disorders.

A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning.

The Gumbel-max trick is a method to draw a sample from a categorical distribution, given by its unnormalized (log-)probabilities. Over the past years, the machine learning community has proposed several extensions of this trick to facilitate, e.g., drawing multiple samples, sampling from structured domains, or gradient estimation for error backpropagation in neural network optimization. The goal of this survey article is to present background about the Gumbel-max trick, and to provide a structured overview of its extensions to ease algorithm selection. Moreover, it presents a comprehensive outline of (machine learning) literature in which Gumbel-based algorithms have been leveraged, reviews commonly-made design choices, and sketches a future perspective.

Improving Lateral Resolution in 3-D Imaging With Micro-beamforming Through Adaptive Beamforming by Deep Learning.

There is an increased desire for miniature ultrasound probes with small apertures to provide volumetric images at high frame rates for in-body applications. Satisfying these increased requirements makes simultaneous achievement of a good lateral resolution a challenge. As micro-beamforming is often employed to reduce data rate and cable count to acceptable levels, receive processing methods that try to improve spatial resolution will have to compensate the introduced reduction in focusing. Existing beamformers do not realize sufficient improvement and/or have a computational cost that prohibits their use. Here we propose the use of adaptive beamforming by deep learning (ABLE) in combination with training targets generated by a large aperture array, which inherently has better lateral resolution. In addition, we modify ABLE to extend its receptive field across multiple voxels. We illustrate that this method improves lateral resolution both quantitatively and qualitatively, such that image quality is improved compared with that achieved by existing delay-and-sum, coherence factor, filtered-delay-multiplication-and-sum and Eigen-based minimum variance beamformers. We found that only in silica data are required to train the network, making the method easily implementable in practice.

The Transform-and-Perform framework: Explainable deep learning beyond classification.

In recent years, visual analytics (VA) has shown promise in alleviating the challenges of interpreting black-box deep learning (DL) models. While the focus of VA for explainable DL has been mainly on classification problems, DL is gaining popularity in high-dimensional-to-high-dimensional (H-H) problems such as image-to-image translation. In contrast to classification, H-H problems have no explicit instance groups or classes to study. Each output is continuous, high-dimensional, and changes in an unknown non-linear manner with changes in the input. These unknown relations between the input, model and output necessitate the user to analyze them in conjunction, leveraging symmetries between them. Since classification tasks do not exhibit some of these challenges, most existing VA systems and frameworks allow limited control of the components required to analyze models beyond classification. Hence, we identify the need for and present a unified conceptual framework, the Transform-and-Perform framework (T&P), to facilitate the design of VA systems for DL model analysis focusing on H-H problems. T&P provides a checklist to structure and identify workflows and analysis strategies to design new VA systems, and understand existing ones to uncover potential gaps for improvements. The goal is to aid the creation of effective VA systems that support the structuring of model understanding and identifying actionable insights for model improvements. We highlight the growing need for new frameworks like T&P with a real-world image-to-image translation application. We illustrate how T&P effectively supports the understanding and identification of potential gaps in existing VA systems.

Automatic deep learning-based consolidation/collapse classification in lung ultrasound images for COVID-19 induced pneumonia.

Our automated deep learning-based approach identifies consolidation/collapse in LUS images to aid in the identification of late stages of COVID-19 induced pneumonia, where consolidation/collapse is one of the possible associated pathologies. A common challenge in training such models is that annotating each frame of an ultrasound video requires high labelling effort. This effort in practice becomes prohibitive for large ultrasound datasets. To understand the impact of various degrees of labelling precision, we compare labelling strategies to train fully supervised models (frame-based method, higher labelling effort) and inaccurately supervised models (video-based methods, lower labelling effort), both of which yield binary predictions for LUS videos on a frame-by-frame level. We moreover introduce a novel sampled quaternary method which randomly samples only 10% of the LUS video frames and subsequently assigns (ordinal) categorical labels to all frames in the video based on the fraction of positively annotated samples. This method outperformed the inaccurately supervised video-based method and more surprisingly, the supervised frame-based approach with respect to metrics such as precision-recall area under curve (PR-AUC) and F1 score, despite being a form of inaccurate learning. We argue that our video-based method is more robust with respect to label noise and mitigates overfitting in a manner similar to label smoothing. The algorithm was trained using a ten-fold cross validation, which resulted in a PR-AUC score of 73% and an accuracy of 89%. While the efficacy of our classifier using the sampled quaternary method significantly lowers the labelling effort, it must be verified on a larger consolidation/collapse dataset, our proposed classifier using the sampled quaternary video-based method is clinically comparable with trained experts' performance.

Self-Organizing Maps for Contrastive Embeddings of Sleep Recordings.

Nowadays, high amounts of data can be acquired in various applications, spurring the need for interpretable data representations that provide actionable insights. Algorithms that yield such representations ideally require as little a priori knowledge about the data or corresponding annotations as possible. To this end, we here investigate the use of Kohonen's Self-Organizing Map (SOM) in combination with data-driven low-dimensional embeddings obtained through self-supervised Contrastive Predictive Coding. We compare our approach to embeddings found with an auto-encoder and, moreover, investigate three ways to deal with node selection during SOM optimization. As a challenging experiment we analyze nocturnal sleep recordings of healthy subjects, and conclude that - for this noisy real-life data - contrastive learning yields a better low-dimensional embedding for the purpose of SOM training, compared to an auto-encoder. In addition, we show that a stochastic temperature-annealed SOM-training outperforms both a deterministic and a non-temperature-annealed stochastic approach. Clinical relevance - The hypnogram has for decades been the clinical standard in sleep medicine despite the fact that it is a highly simplified representation of a polysomnography recording. We propose a sensor-agnostic algorithm that is able to reveal more intricate patterns in sleep recordings which might teach us about sleep structure and sleep disorders.

Unsupervised domain adaptation method for segmenting cross-sectional CCA images.

BACKGROUND AND OBJECTIVES: Automatic vessel segmentation in ultrasound is challenging due to the quality of the ultrasound images, which is affected by attenuation, high level of speckle noise and acoustic shadowing. Recently, deep convolutional neural networks are increasing in popularity due to their great performance on image segmentation problems, including vessel segmentation. Traditionally, large labeled datasets are required to train a network that achieves high performance, and is able to generalize well to different orientations, transducers and ultrasound scanners. However, these large datasets are rare, given that it is challenging and time-consuming to acquire and manually annotate in-vivo data. METHODS: In this work, we present a model-based, unsupervised domain adaptation method that consists of two stages. In the first stage, the network is trained on simulated ultrasound images, which have an accurate ground truth. In the second stage, the network continues training on in-vivo data in an unsupervised way, therefore not requiring the data to be labelled. Rather than using an adversarial neural network, prior knowledge on the elliptical shape of the segmentation mask is used to detect unexpected outputs. RESULTS: The segmentation performance was quantified using manually segmented images as ground truth. Due to the proposed domain adaptation method, the median Dice similarity coefficient increased from 0 to 0.951, outperforming a domain adversarial neural network (median Dice 0.922) and a state-of-the-art Star-Kalman algorithm that was specifically designed for this dataset (median Dice 0.942). CONCLUSIONS: The results show that it is feasible to first train a neural network on simulated data, and then apply model-based domain adaptation to further improve segmentation performance by training on unlabeled in-vivo data. This overcomes the limitation of conventional deep learning approaches to require large amounts of manually labeled in-vivo data. Since the proposed domain adaptation method only requires prior knowledge on the shape of the segmentation mask, performance can be explored in various domains and applications in future research.

Dynamic Probabilistic Pruning: A General Framework for Hardware-Constrained Pruning at Different Granularities.

Unstructured neural network pruning algorithms have achieved impressive compression ratios. However, the resulting-typically irregular-sparse matrices hamper efficient hardware implementations, leading to additional memory usage and complex control logic that diminishes the benefits of unstructured pruning. This has spurred structured coarse-grained pruning solutions that prune entire feature maps or even layers, enabling efficient implementation at the expense of reduced flexibility. Here, we propose a flexible new pruning mechanism that facilitates pruning at different granularities (weights, kernels, and feature maps) while retaining efficient memory organization (e.g., pruning exactly k -out-of- n weights for every output neuron or pruning exactly k -out-of- n kernels for every feature map). We refer to this algorithm as dynamic probabilistic pruning (DPP). DPP leverages the Gumbel-softmax relaxation for differentiable k -out-of- n sampling, facilitating end-to-end optimization. We show that DPP achieves competitive compression ratios and classification accuracy when pruning common deep learning models trained on different benchmark datasets for image classification. Relevantly, the dynamic masking of DPP facilitates for joint optimization of pruning and weight quantization in order to even further compress the network, which we show as well. Finally, we propose novel information-theoretic metrics that show the confidence and pruning diversity of pruning masks within a layer.