TraHGR: Transformer for Hand Gesture Recognition via Electromyography.

Deep learning-based Hand Gesture Recognition (HGR) via surface Electromyogram (sEMG) signals have recently shown considerable potential for development of advanced myoelectric-controlled prosthesis. Although deep learning techniques can improve HGR accuracy compared to their classical counterparts, classifying hand movements based on sparse multichannel sEMG signals is still a challenging task. Furthermore, existing deep learning approaches, typically, include only one model as such can hardly extract representative features. In this paper, we aim to address this challenge by capitalizing on the recent advances in hybrid models and transformers. In other words, we propose a hybrid framework based on the transformer architecture, which is a relatively new and revolutionizing deep learning model. The proposed hybrid architecture, referred to as the Transformer for Hand Gesture Recognition (TraHGR), consists of two parallel paths followed by a linear layer that acts as a fusion center to integrate the advantage of each module. We evaluated the proposed architecture TraHGR based on the commonly used second Ninapro dataset, referred to as the DB2. The sEMG signals in the DB2 dataset are measured in real-life conditions from 40 healthy users, each performing 49 gestures. We have conducted an extensive set of experiments to test and validate the proposed TraHGR architecture, and compare its achievable accuracy with several recently proposed HGR classification algorithms over the same dataset. We have also compared the results of the proposed TraHGR architecture with each individual path and demonstrated the distinguishing power of the proposed hybrid architecture. The recognition accuracies of the proposed TraHGR architecture for the window of size 200ms and step size of 100ms are 86.00%, 88.72%, 81.27%, and 93.74%, which are 2.30%, 4.93%, 8.65%, and 4.20% higher than the state-of-the-art performance for DB2 (49 gestures), DB2-B (17 gestures), DB2-C (23 gestures), and DB2-D (9 gestures), respectively.

DF-SSmVEP: Dual Frequency Aggregated Steady-State Motion Visual Evoked Potential Design with Bifold Canonical Correlation Analysis.

Recent advancements in Electroencephalographic (EEG) sensor technologies and signal processing algorithms have paved the way for further evolution of Brain Computer Interfaces (BCI) in several practical applications, ranging from rehabilitation systems to smart consumer technologies. When it comes to Signal Processing (SP) for BCI, there has been a surge of interest on Steady-State motion Visual Evoked Potentials (SSmVEP), where motion stimulation is used to address key issues associated with conventional light flashing/flickering. Such benefits, however, come with the price of being less accurate and having a lower Information Transfer Rate (ITR). From this perspective, this paper focuses on the design of a novel SSmVEP paradigm without using resources such as trial time, phase, and/or number of targets to enhance the ITR. The proposed design is based on the intuitively pleasing idea of integrating more than one motion within a single SSmVEP target stimuli, simultaneously. To elicit SSmVEP, we designed a novel and innovative dual frequency aggregated modulation paradigm, called the Dual Frequency Aggregated Steady-State motion Visual Evoked Potential (DF-SSmVEP), by concurrently integrating "Radial Zoom" and "Rotation" motions in a single target without increasing the trial length. Compared to conventional SSmVEPs, the proposed DF-SSmVEP framework consists of two motion modes integrated and shown simultaneously each modulated by a specific target frequency. The paper also develops a specific unsupervised classification model, referred to as the Bifold Canonical Correlation Analysis (BCCA), based on two motion frequencies per target. The corresponding covariance coefficients are used as extra features improving the classification accuracy. The proposed DF-SSmVEP is evaluated based on a real EEG dataset and the results corroborate its superiority. The proposed DF-SSmVEP outperforms its counterparts and achieved an average ITR of 30.7 ± 1.97 and an average accuracy of 92.5 ± 2.04, while the Radial Zoom and Rotation result in average ITRs of 18.35 ± 1 and 20.52 ± 2.5, and average accuracies of 68.12 ± 3.5 and 77.5 ± 3.5, respectively.

A Unified Optimization for Resilient Dynamic Event-Triggering Consensus Under Denial of Service.

This article proposes a resilient framework for optimized consensus using a dynamic event-triggering (DET) scheme, where the multiagent system (MAS) is subject to denial-of-service (DoS) attacks. When initiated by an adversary, DoS blocks the local and neighboring communication channels in the network. A distributed DET scheme is utilized to limit transmissions between the neighboring agents. A novel convex optimization approach is proposed that simultaneously co-designs all unknown control and DET parameters. The optimization is based on the weighted sum approach and increases the interevent interval for a predefined consensus convergence rate. In the presence of DoS, the proposed co-design framework is beneficial in two ways: 1) the desired level of resilience to DoS is included as a given (desired) input and 2) the upper bound for guaranteed resilience associated with the proposed co-design approach is less conservative (larger) compared to those obtained from other analytical solutions. A structured tradeoff between relevant features of the MAS, namely, the consensus convergence rate, frequency of event triggerings, and level of resilience to DoS attacks, is established. Simulations based on nonholonomic mobile robots quantify the effectiveness of the proposed implementation.

Deep Learning for Ultrasound Image Formation: CUBDL Evaluation Framework and Open Datasets.

Deep learning for ultrasound image formation is rapidly garnering research support and attention, quickly rising as the latest frontier in ultrasound image formation, with much promise to balance both image quality and display speed. Despite this promise, one challenge with identifying optimal solutions is the absence of unified evaluation methods and datasets that are not specific to a single research group. This article introduces the largest known international database of ultrasound channel data and describes the associated evaluation methods that were initially developed for the challenge on ultrasound beamforming with deep learning (CUBDL), which was offered as a component of the 2020 IEEE International Ultrasonics Symposium. We summarize the challenge results and present qualitative and quantitative assessments using both the initially closed CUBDL evaluation test dataset (which was crowd-sourced from multiple groups around the world) and additional in vivo breast ultrasound data contributed after the challenge was completed. As an example quantitative assessment, single plane wave images from the CUBDL Task 1 dataset produced a mean generalized contrast-to-noise ratio (gCNR) of 0.67 and a mean lateral resolution of 0.42 mm when formed with delay-and-sum beamforming, compared with a mean gCNR as high as 0.81 and a mean lateral resolution as low as 0.32 mm when formed with networks submitted by the challenge winners. We also describe contributed CUBDL data that may be used for training of future networks. The compiled database includes a total of 576 image acquisition sequences. We additionally introduce a neural-network-based global sound speed estimator implementation that was necessary to fairly evaluate the results obtained with this international database. The integration of CUBDL evaluation methods, evaluation code, network weights from the challenge winners, and all datasets described herein are publicly available (visit https://cubdl.jhu.edu for details).

FS-HGR: Few-Shot Learning for Hand Gesture Recognition via Electromyography.

This work is motivated by the recent advances in Deep Neural Networks (DNNs) and their widespread applications in human-machine interfaces. DNNs have been recently used for detecting the intended hand gesture through the processing of surface electromyogram (sEMG) signals. Objective: Although DNNs have shown superior accuracy compared to conventional methods when large amounts of data are available for training, their performance substantially decreases when data are limited. Collecting large datasets for training may be feasible in research laboratories, but it is not a practical approach for real-life applications. The main objective of this work is to design a modern DNN-based gesture detection model that relies on minimal training data while providing high accuracy. Methods: We propose the novel Few-Shot learning- Hand Gesture Recognition (FS-HGR) architecture. Few-shot learning is a variant of domain adaptation with the goal of inferring the required output based on just one or a few training observations. The proposed FS-HGR generalizes after seeing very few observations from each class by combining temporal convolutions with attention mechanisms. This allows the meta-learner to aggregate contextual information from experience and to pinpoint specific pieces of information within its available set of inputs. Data Source & Summary of Results: The performance of FS-HGR was tested on the second and fifth Ninapro databases, referred to as the DB2 and DB5, respectively. The DB2 consists of 50 gestures (rest included) from 40 healthy subjects. The Ninapro DB5 contains data from 10 healthy participants performing a total of 53 different gestures (rest included). The proposed approach for the Ninapro DB2 led to 85.94% classification accuracy on new repetitions with few-shot observation (5-way 5-shot), 81.29% accuracy on new subjects with few-shot observation (5-way 5-shot), and 73.36% accuracy on new gestures with few-shot observation (5-way 5-shot). Moreover, the proposed approach for the Ninapro DB5 led to 64.65% classification accuracy on new subjects with few-shot observation (5-way 5-shot).

Plane-Wave Ultrasound Beamforming Through Independent Component Analysis.

BACKGROUND AND OBJECTIVE: Beamforming in coherent plane-wave compounding (CPWC) is an essential step in maintaining high resolution, contrast and framerate. Adaptive methods have been designed to achieve this goal by estimating the apodization weights from echo traces acquired by several transducer elements. METHODS: Herein, we formulate plane-wave beamforming as a blind source separation problem, where the output of each transducer element is considered as a non-independent observation of the field. As such, beamforming can be formulated as the estimation of an independent component out of the observations. We then adapt the independent component analysis (ICA) algorithm to solve this problem and reconstruct the final image. RESULTS: The proposed method is evaluated on a set of simulations, real phantom, and in vivo data available from the plane-wave imaging challenge in medical ultrasound. Moreover, the results are compared with other well-known adaptive methods. CONCLUSIONS: Results demonstrate that the proposed method simultaneously improves the resolution and contrast.

Virtual Source Synthetic Aperture for Accurate Lateral Displacement Estimation in Ultrasound Elastography.

Ultrasound elastography (USE) is an emerging noninvasive imaging technique in which pathological alterations can be visualized by revealing the mechanical properties of the tissue. Estimating tissue displacement in all directions is required to accurately estimate the mechanical properties. Despite capabilities of elastography techniques in estimating displacement in both axial and lateral directions, estimation of axial displacement is more accurate than lateral direction due to higher sampling frequency, higher resolution, and having a carrier signal propagating in the axial direction. Among different ultrasound imaging techniques, synthetic aperture (SA) has better lateral resolution than others, but it is not commonly used for USE due to its limitation in imaging depth of field. Virtual source synthetic aperture (VSSA) imaging is a technique to implement SA beamforming on the focused transmitted data to overcome the limitation of SA in depth of field while maintaining the same lateral resolution as SA. Besides lateral resolution, VSSA has the capability of increasing sampling frequency in the lateral direction without interpolation. In this article, we utilize VSSA to perform beamforming to enable higher resolution and sampling frequency in the lateral direction. The beamformed data are then processed using our recently published elastography technique, OVERWIND. Simulation and experimental results show substantial improvement in the estimation of lateral displacements.

Siamese Neural Networks for EEG-based Brain-computer Interfaces.

Motivated by the inconceivable capability of human brain in simultaneously processing multi-modal signals and its real-time feedback to the outer world events, there has been a surge of interest in establishing a communication bridge between the human brain and a computer, which are referred to as Brain-computer Interfaces (BCI). To this aim, monitoring the electrical activity of brain through Electroencephalogram (EEG) has emerged as the prime choice for BCI systems. To discover the underlying and specific features of brain signals for different mental tasks, a considerable number of research works are developed based on statistical and data-driven techniques. However, a major bottleneck in development of practical and commercial BCI systems is their limited performance when the number of mental tasks for classification is increased. In this work, we propose a new EEG processing and feature extraction paradigm based on Siamese neural networks, which can be conveniently merged and scaled up for multi-class problems. The idea of Siamese networks is to train a double-input neural network based on a contrastive loss-function, which provides the capability of verifying if two input EEG trials are from the same class or not. In this work, a Siamese architecture, which is developed based on Convolutional Neural Networks (CNN) and provides a binary output on the similarity of two inputs, is combined with One vs. Rest (OVR) and One vs. One (OVO) techniques to scale up for multi-class problems. The efficacy of this architecture is evaluated on a 4-class Motor Imagery (MI) dataset from BCI Competition IV2a and the results suggest a promising performance compared to its counterparts.

High Frequency Ultrasound Image Recovery Using Tight Frame Generative Adversarial Networks.

In ultrasound imaging, there is a trade-off between imaging depth and axial resolution because of physical limitations. Increasing the center frequency of the transmitted ultrasound wave improves the axial resolution of resulting image. However, High Frequency (HF) ultrasound has a shallower depth of penetration. Herein, we propose a novel method based on Generative Adversarial Network (GAN) for achieving a high axial resolution without a reduction in imaging depth. Results on simulated phantoms show that a mapping function between Low Frequency (LF) and HF ultrasound images can be constructed.

Synthetic aperture with high lateral sampling frequency for ultrasound elastography.

Ultrasound elastography is a non-invasive technique for detecting pathological alterations in tissue. It is known that pathological alteration of tissue often has a direct impact on its elastic modulus, which can be revealed using elastography. For estimating elastic modulus, we need to estimate both axial and lateral displacement accurately. Current state of the art elastography techniques provide a substantially less accurate lateral displacement field as compared to the axial displacement field. One of the most important factors in poor lateral estimation is a low sampling frequency in the lateral direction. In this paper, we use synthetic aperture beamforming to benefit from its capability of high sampling frequency in the lateral direction. We compare highly sampled data and focused line per line beam formed data by feeding them to our recently published elastography method, OVERWIND [1]. According to simulation and phantom experiments, not only the lateral displacement estimation is substantially improved, but also the axial displacement estimation is improved.

Accurate and Precise Time-Delay Estimation for Ultrasound Elastography With Prebeamformed Channel Data.

Free-hand palpation ultrasound elastography is a noninvasive approach for detecting pathological alteration in tissue. In this method, the tissue is compressed by a handheld probe and displacement of each sample is estimated, a process which is also known as time-delay estimation (TDE). Even with the simplifying assumption that ignores out of plane motion, TDE is an ill-posed problem requiring estimation of axial and lateral displacements for each sample from its intensity. A well-known class of methods for making elastography a well-posed problem is regularized optimization-based methods, which imposes smoothness regularization in the associated cost function. In this article, we propose to utilize channel data that have been compensated for time gain and time delay (introduced by transmission) instead of postbeamformed radio frequency (RF) data in the optimization problem. We name our proposed method Channel data for GLobal Ultrasound Elastography (CGLUE). We analytically derive bias and variances of TDE as functions of data noise for CGLUE and Global Ultrasound Elastography (GLUE) and use the Cauchy-Schwarz inequality to prove that CGLUE provides a TDE with lower bias and variance error. To further illustrate the improved performance of CGLUE, the results of simulation, experimental phantom, and ex-vivo experiments are presented.

3D normalized cross-correlation for estimation of the displacement field in ultrasound elastography.

This paper introduces a novel technique to estimate tissue displacement in quasi-static elastography. A major challenge in elastography is estimation of displacement (also referred to time-delay estimation) between pre-compressed and post-compressed ultrasound data. Maximizing normalized cross correlation (NCC) of ultrasound radio-frequency (RF) data of the pre- and post-compressed images is a popular technique for strain estimation due to its simplicity and computational efficiency. Several papers have been published to increase the accuracy and quality of displacement estimation based on NCC. All of these methods use 2D spatial windows in RF data to estimate NCC, wherein displacement is assumed to be constant within each window. In this work, we extend this assumption along the third dimension. Two approaches are proposed to get third dimension. In the first approach, we use temporal domain to exploit neighboring samples in both spatial and temporal directions. Considering temporal information is important since traditional and ultrafast ultrasound machines are, respectively, capable of imaging at more than 30 frame per second (fps) and 1000 fps. Another approach is to use time-delayed pre-beam formed data (channel data) instead of RF data. In this method information of all channels that are recorded as pre-beam formed data of each RF line will be considered as 3rd dimension. We call these methods as spatial temporal normalized cross correlation (STNCC) and channel data normalized cross correlation (CNCC) and show that they substantially outperforms NCC using simulation, phantom and in-vivo experiments. Given substantial improvements of results in addition to the relative simplicity of implementing STNCC and CNCC, the proposed approaches can potentially have a large impact in both academic and commercial work on ultrasound elastography.

Combining Total Variation Regularization with Window-Based Time Delay Estimation in Ultrasound Elastography.

A major challenge of free-hand palpation ultrasound elastography (USE) is estimating the displacement of RF samples between pre- and post-compressed RF data. The problem of displacement estimation is ill-posed since the displacement of one sample by itself cannot be uniquely calculated. To resolve this problem, two categories of methods have emerged. The first category assumes that the displacement of samples within a small window surrounding the reference sample is constant. The second class imposes smoothness regularization and optimizes an energy function. Herein, we propose a novel method that combines both approaches, and as such, is more robust to noise. The second contribution of this work is the introduction of the L1 norm as the regularization term in our cost function, which is often referred to as the total variation (TV) regularization. Compared to previous work that used the L2 norm regularization, optimization of the new cost function is more challenging. However, the advantages of using the L1 norm are twofold. First, it leads to substantial improvement in the sharpness of displacement estimates. Second, to optimize the cost function with the L1 norm regularization, we use an iterative method that further increases the robustness. We name our proposed method tOtal Variation Regularization and WINDow-based time delay estimation (OVERWIND) and show that it is robust to signal decorrelation and generates sharp displacement and strain maps for simulated, experimental phantom and in-vivo data. In particular, OVERWIND improves strain contrast-to-noise ratio (CNR) by 27.26%, 144.05%, and 49.90% on average in simulation, phantom, and in-vivo data, respectively, compared to our recent Global Ultrasound Elastography (GLUE) method.

Analytical Minimization-Based Regularized Subpixel Shear-Wave Tracking for Ultrasound Elastography.

Ultrasound elastography is a convenient and affordable method for imaging mechanical properties of tissue, which are often correlated with pathologies. An emerging novel elastography technique applies an external acoustic radiation force to generate a shear wave in the tissue and uses ultrasound imaging to track the shear wave. Accurate tracking of the small tissue motion is a critical step in shear-wave elastography (SWE), but it is challenging due to various sources of noise in the ultrasound data. We formulate tissue displacement estimation as an optimization problem and propose two computationally efficient approaches to estimate the displacement field. The first algorithm is referred to as dynamic programming analytic minimization (DPAM), which utilizes first-order Taylor series expansion of the highly nonlinear cost function to allow for its efficient optimization, and was previously proposed for quasistatic elastography. The second algorithm is a novel technique that utilizes second-order derivatives of the nonlinear cost function. We call the new algorithm second-order analytic minimization elastography (SESAME). We compare DPAM and SESAME to the standard normalized cross correlation (NCC) approach in the context of displacement and speed estimation of wave propagation in SWE. The results of micrometer-order displacement estimation in a uniform simulation phantom illustrate that SESAME outperforms DPAM, which in turn outperforms NCC in terms of signal-to-noise ratio (SNR) and jitter. In addition, the relative difference between true and reconstructed shear modulus (averaged over excitations at different focal depths and several scatterer realizations at each depth) is approximately 3.41%, 1.12%, and 1.01%, respectively, for NCC, DPAM, and SESAME. The performance of the proposed methods is also assessed with real data acquired using a tissue-mimicking phantom, wherein, in comparison to NCC, DPAM and SESAME improve the SNR of displacement estimates by 7.6 and 9.5 dB, respectively. Experimental results on a tissue-mimicking phantom also show that shear modulus reconstruction substantially improved with the proposed DPAM technique over NCC and with some further improvement achieved by utilizing the second-order Taylor series approximation in SESAME instead of the first-order DPAM.