DeGCN: Deformable Graph Convolutional Networks for Skeleton-Based Action Recognition.

Graph convolutional networks (GCN) have recently been studied to exploit the graph topology of the human body for skeleton-based action recognition. However, most of these methods unfortunately aggregate messages via an inflexible pattern for various action samples, lacking the awareness of intra-class variety and the suitableness for skeleton sequences, which often contain redundant or even detrimental connections. In this paper, we propose a novel Deformable Graph Convolutional Network (DeGCN) to adaptively capture the most informative joints. The proposed DeGCN learns the deformable sampling locations on both spatial and temporal graphs, enabling the model to perceive discriminative receptive fields. Notably, considering human action is inherently continuous, the corresponding temporal features are defined in a continuous latent space. Furthermore, we design an innovative multi-branch framework, which not only strikes a better trade-off between accuracy and model size, but also elevates the effect of ensemble between the joint and bone modalities remarkably. Extensive experiments show that our proposed method achieves state-of-the-art performances on three widely used datasets, NTU RGB+D, NTU RGB+D 120, and NW-UCLA.

A Dual-Scale Lead-Separated Transformer for ECG Classification.

This work proposes a novel dual-scale lead-separated transformer for the auxiliary diagnosis of 12-lead electrocardiograms (ECGs). We added a new structure design on the basis of traditional ECG signal processing, which led to our model with only 2.6M parameters. The output of the system is the classification results. The fixed 0.5 second ECG segments of each lead are interpreted as independent patches. Together with the reduced dimension signal, patches form a dual-scale representation. As a method to reduce interference from segments with low correlation, a lead-orthogonal attention module is proposed. Experimental results show the effectiveness and scalability of our model.Clinical relevance- Our method improves the scores of clinical 12-lead ECG classification and shows generalization ability. Our model is suitable for single-label and multi-label classification tasks on clinical 12-lead ECG and is compatible with single lead classification. The integration of clinical information can further improve the effectiveness of the model.

MVKT-ECG: Efficient single-lead ECG classification for multi-label arrhythmia by multi-view knowledge transferring.

Electrocardiogram (ECG) is a widely used technique for diagnosing cardiovascular disease. The widespread emergence of smart ECG devices has sparked the demand for intelligent single-lead ECG-based diagnostic systems. However, it is challenging to develop a single-lead-based ECG interpretation model for multiple disease diagnosis due to the lack of some key disease information. We aim to improve the diagnostic capabilities of single-lead ECG for multi-label disease classification in a new teacher-student manner, where the teacher trained by multi-lead ECG educates a student who observes only single-lead ECG We present a new disease-aware Contrastive Lead-information Transferring (CLT) to improve the mutual disease information between the single-lead-based ECG interpretation model and multi-lead-based ECG interpretation model. Moreover, We modify the traditional Knowledge Distillation into Multi-label disease Knowledge Distillation (MKD) to make it applicable for multi-label disease diagnosis. The whole knowledge transferring process is inter-lead Multi-View Knowledge Transferring of ECG (MVKT-ECG). By employing the training strategy, we can effectively transfer comprehensive disease knowledge from various views of ECG, such as the 12-lead ECG, to a single-lead-based ECG interpretation model. This enables the model to extract intricate details from single-lead ECG signals and enhances the model's capability of diagnosing and identifying single-lead signals. Extensive experiments on two commonly used public multi-label datasets, ICBEB2018 and PTB-XL demonstrate that our MVKT-ECG yields exceptional diagnostic performance improvements for single-lead ECG. The student outperforms its baseline observably on the PTB-XL dataset (1.3 % on PTB.super, and 1.4 % on PTB.sub), and on ICBEB2018 dataset (3.2 %).

Hierarchical Vision Navigation System for Quadruped Robots with Foothold Adaptation Learning.

Legged robots can travel through complex scenes via dynamic foothold adaptation. However, it remains a challenging task to efficiently utilize the dynamics of robots in cluttered environments and to achieve efficient navigation. We present a novel hierarchical vision navigation system combining foothold adaptation policy with locomotion control of the quadruped robots. The high-level policy trains an end-to-end navigation policy, generating an optimal path to approach the target with obstacle avoidance. Meanwhile, the low-level policy trains the foothold adaptation network through auto-annotated supervised learning to adjust the locomotion controller and to provide more feasible foot placement. Extensive experiments in both simulation and the real world show that the system achieves efficient navigation against challenges in dynamic and cluttered environments without prior information.

SEpi-3D: soft epipolar 3D shape measurement with an event camera for multipath elimination.

Multipath in 3D imaging happens when one pixel receives light from multiple reflections, which causes errors in the measured point cloud. In this paper, we propose the soft epipolar 3D(SEpi-3D) method to eliminate multipath in temporal space with an event camera and a laser projector. Specifically, we align the projector and event camera row onto the same epipolar plane with stereo rectification; we capture event flow synchronized with the projector frame to construct a mapping relationship between event timestamp and projector pixel; we develop a multipath eliminating method that utilizes the temporal information from the event data together with the epipolar geometry. Experiments show that the RMSE decreases by 6.55mm on average in the tested multipath scenes, and the percentage of error points decreases by 7.04%.

A Feature-Trajectory-Smoothed High-Speed Model for Video Anomaly Detection.

High-speed detection of abnormal frames in surveillance videos is essential for security. This paper proposes a new video anomaly-detection model, namely, feature trajectory-smoothed long short-term memory (FTS-LSTM). This model trains an LSTM autoencoder network to generate future frames on normal video streams, and uses the FTS detector and generation error (GE) detector to detect anomalies on testing video streams. FTS loss is a new indicator in the anomaly-detection area. In the training stage, the model applies a feature trajectory smoothness (FTS) loss to constrain the LSTM layer. This loss enables the LSTM layer to learn the temporal regularity of video streams more precisely. In the detection stage, the model utilizes the FTS loss and the GE loss as two detectors to detect anomalies. By cascading the FTS detector and the GE detector to detect anomalies, the model achieves a high speed and competitive anomaly-detection performance on multiple datasets.

A community effort to assess and improve computerized interpretation of 12-lead resting electrocardiogram.

Computerized interpretation of electrocardiogram plays an important role in daily cardiovascular healthcare. However, inaccurate interpretations lead to misdiagnoses and delay proper treatments. In this work, we built a high-quality Chinese 12-lead resting electrocardiogram dataset with 15,357 records, and called for a community effort to improve the performances of CIE through the China ECG AI Contest 2019. This dataset covers most types of ECG interpretations, including the normal type, 8 common abnormal types, and the other type which includes both uncommon abnormal and noise signals. Based on the Contest, we systematically assessed and analyzed a set of top-performing methods, most of which are deep neural networks, with both their commonalities and characteristics. This study establishes the benchmarks for computerized interpretation of 12-lead resting electrocardiogram and provides insights for the development of new methods. Graphical Abstract A community effort to assess and improve computerized interpretation of 12-lead resting electrocardiogram.

OMC-SLIO: Online Multiple Calibrations Spinning LiDAR Inertial Odometry.

Light detection and ranging (LiDAR) is often combined with an inertial measurement unit (IMU) to get the LiDAR inertial odometry (LIO) for robot localization and mapping. In order to apply LIO efficiently and non-specialistically, self-calibration LIO is a hot research topic in the related community. Spinning LiDAR (SLiDAR), which uses an additional rotating mechanism to spin a common LiDAR and scan the surrounding environment, achieves a large field of view (FoV) with low cost. Unlike common LiDAR, in addition to the calibration between the IMU and the LiDAR, the self-calibration odometer for SLiDAR must also consider the mechanism calibration between the rotating mechanism and the LiDAR. However, existing self-calibration LIO methods require the LiDAR to be rigidly attached to the IMU and do not take the mechanism calibration into account, which cannot be applied to the SLiDAR. In this paper, we propose firstly a novel self-calibration odometry scheme for SLiDAR, named the online multiple calibration inertial odometer (OMC-SLIO) method, which allows online estimation of multiple extrinsic parameters among the LiDAR, rotating mechanism and IMU, as well as the odometer state. Specially, considering that the rotating and static parts of the motor encoder inside the SLiDAR are rigidly connected to the LiDAR and IMU respectively, we formulate the calibration within the SLiDAR as two separate sets of calibrations: the mechanism calibration between the LiDAR and the rotating part of the motor encoder and the sensor calibration between the static part of the motor encoder and the IMU. Based on such a SLiDAR calibration formulation, we can construct a well-defined kinematic model from the LiDAR to the IMU with the angular information from the motor encoder. Based on the kinematic model, a two-stage motion compensation method is presented to eliminate the point cloud distortion resulting from LiDAR spinning and platform motion. Furthermore, the mechanism and sensor calibration as well as the odometer state are wrapped in a measurement model and estimated via an error-state iterative extended Kalman filter (ESIEKF). Experimental results show that our OMC-SLIO is effective and attains excellent performance.

Improving Automatic Detection of ECG Abnormality with Less Manual Annotations using Siamese Network.

Electrocardiography is a very common, non-invasive diagnostic procedure and its interpretation is increasingly supported by automatic interpretation algorithms. Recently many works also focused on the design of automatic ECG abnormality detection algorithms. However, clinical electrocardiogram datasets often suffer from their heavy needs for expert annotations, which are often expensive and hard to obtain. In this work, we proposed a weakly supervised pretraining method based on the Siamese neural network, which utilizes the original diagnostic information written by physicians to produce useful feature representations of the ECG signal which improves performance of ECG abnormality detection algorithms with fewer expert annotations. The experiment showed that with the proposed weekly supervised pretraining, the performance of ECG abnormality detection algorithms that was trained with only 1/8 annotated ECG data outperforms classical models that was trained with fully annotated ECG data, which implies a large proportion of annotation resource could be saved. The proposed technique could be easily extended to other tasks beside abnormality detection provided that the text similarity metric is specifically designed for the given task.Clinical Relevance-This work proposes a novel framework for the automatic detection of cardiovascular disease based on electrocardiogram.

Segment Origin Prediction: A Self-supervised Learning Method for Electrocardiogram Arrhythmia Classification.

The automatic arrhythmia classification system has made a significant contribution to reducing the mortality rate of cardiovascular diseases. Although the current deep-learning-based models have achieved ideal effects in arrhythmia classification, their performance still needs to be further improved due to the small scale of the dataset. In this paper, we propose a novel self-supervised pre-training method called Segment Origin Prediction (SOP) to improve the model's arrhythmia classification performance. We design a data reorganization module, which allows the model to learn ECG features by predicting whether two segments are from the same original signal without using annotations. Further, by adding a feed-forward layer to the pre-training stage, the model can achieve better performance when using labeled data for arrhythmia classification in the downstream stage. We apply the proposed SOP method to six representative models and evaluate the performances on the PhysioNet Challenge 2017 dataset. After using the SOP pre-training method, all baseline models gain significant improvement. The experimental results verify the effectiveness of the proposed SOP method.

The association between ECG criteria and Echo criteria for left ventricular hypertrophy in a general Chinese population.

BACKGROUND: Several ECG criteria have been widely used for diagnosis of left ventricular hypertrophy (LVH) in clinical practice. However, their performance in a general Chinese population is limited. METHODS AND RESULTS: A multi-stage, stratified cluster sampling across China was performed and 7415 representative Chinese adults aged 18-85 years were analyzed. ECG was collected by using GE MAC 5500 machine. The association between five ECG-LVH criteria (i.e., Peguero-Lo Presti, Cornell, Cornell product, Sokolow-Lyon and Sokolow-Lyon product) and echocardiographic LVH (Echo-LVH) was assessed by Pearson's correlation, diagnostic statistics like predictive values, and receiver operating characteristics (ROC) curve. We found that the prevalence of the Echo-LVH was 11% while ECG-LVH ranged from 3% to 27%. All ECG-LVH criteria had high negative predictive value (NPV) (89%) and specificity (73-96%) but low positive predictive value (PPV) (12-24%) and sensitivity (4-29%). The newly Peguero-Lo Presti criteria had higher sensitivity (29%) but lower specificity (73%) and accuracy (68%) compared with other criteria. Cornell product had the best diagnostic performance (AUC: 0.59), as well as the highest specificity (96%) and accuracy (86%) but lowest sensitivity (4%). Among single-lead components of ECG criteria, RaVL voltage and QRS duration performed relatively better than others. Hypertensive and older individuals had higher sensitivity but lower specificity and accuracy than their counterparts. CONCLUSION: ECG-LVH criteria had high NPV to detect Echo-LVH. Though with higher sensitivity, Peguero-Lo Presti criteria did not have better diagnostic performance to detect Echo-LVH. RaVL and QRS duration had stronger association with Echo-LVH among all single-lead components.

Class-Variant Margin Normalized Softmax Loss for Deep Face Recognition.

In deep face recognition, the commonly used softmax loss and its newly proposed variations are not yet sufficiently effective to handle the class imbalance and softmax saturation issues during the training process while extracting discriminative features. In this brief, to address both issues, we propose a class-variant margin (CVM) normalized softmax loss, by introducing a true-class margin and a false-class margin into the cosine space of the angle between the feature vector and the class-weight vector. The true-class margin alleviates the class imbalance problem, and the false-class margin postpones the early individual saturation of softmax. With negligible computational complexity increment during training, the new loss function is easy to implement in the common deep learning frameworks. Comprehensive experiments on the LFW, YTF, and MegaFace protocols demonstrate the effectiveness of the proposed CVM loss function.

Unsupervised Domain Adaptation for ECG Arrhythmia Classification.

Electrocardiograph (ECG) is one of the most critical physiological signals for arrhythmia diagnosis in clinical practice. In recent years, various algorithms based on deep learning have been proposed to solve the heartbeat classification problem and achieved saturated accuracy in intrapatient paradigm, but encountered performance degradation in inter-patient paradigm due to the drastic variation of ECG signals among different individuals. In this paper, we propose a novel unsupervised domain adaptation scheme to address this problem. Specifically, we first propose a robust baseline model called Multi-path Atrous Convolutional Network (MACN) to tackle ECG heartbeat classification. Further, we introduce Cluster-aligning loss and Cluster-separating loss to align the distributions of training and test data and increase the discriminability, respectively. The proposed method requires no expert annotations but a short period of unlabelled data in new records. Experimental results on the MIT-BIH database demonstrate that our scheme effectively intensifies the baseline model and achieves competitive performance with other state-of-the-arts.

Large depth-of-field three-dimensional shape measurement with the focal sweep technique.

Three-dimensional (3D) shape measurement based on the fringe projection technique has been extensively used for scientific discoveries and industrial practices. Yet, one of the most challenging issues is its limited depth of field (DOF). This paper presents a method to drastically increase DOF of 3D shape measurement technique by employing the focal sweep method. The proposed method employs an electrically tunable lens (ETL) to rapidly sweep the focal plane during image integration and the post deconvolution algorithm to reconstruct focused images for 3D reconstruction. Experimental results demonstrated that our proposed method can achieve high-resolution and high-accuracy 3D shape measurement with greatly improved DOF in real time.

Bi-Stream Pose-Guided Region Ensemble Network for Fingertip Localization From Stereo Images.

In human-computer interaction, it is important to accurately estimate the hand pose, especially fingertips. However, traditional approaches to fingertip localization mainly rely on depth images and thus suffer considerably from noise and missing values. Instead of depth images, stereo images can also provide 3-D information of hands. There are nevertheless limitations on the dataset size, global viewpoints, hand articulations, and hand shapes in publicly available stereo-based hand pose datasets. To mitigate these limitations and promote further research on hand pose estimation from stereo images, we build a new large-scale binocular hand pose dataset called THU-Bi-Hand, offering a new perspective for fingertip localization. In the THU-Bi-Hand dataset, there are 447k pairs of stereo images of different hand shapes from ten subjects with accurate 3-D location annotations of the wrist and five fingertips. Captured with minimal restriction on the range of hand motion, the dataset covers a large global viewpoint space and hand articulation space. To better present the performance of fingertip localization on THU-Bi-Hand, we propose a novel scheme termed bi-stream pose-guided region ensemble network (Bi-Pose-REN). It extracts more representative feature regions around joints in the feature maps under the guidance of the previously estimated pose. The feature regions are integrated hierarchically according to the topology of hand joints to regress a refined hand pose. Bi-Pose-REN and several existing methods are evaluated on THU-Bi-Hand so that benchmarks are provided for further research. Experimental results show that our Bi-Pose-REN has achieved the best performance on THU-Bi-Hand.

Large depth-of-field 3D shape measurement using an electrically tunable lens.

The state-of-the-art 3D shape measurement system has rather shallow working volume due to the limited depth-of-field (DOF) of conventional lens. In this paper, we propose to use the electrically tunable lens to substantially enlarge the DOF. Specifically, we capture always in-focus phase-shifted fringe patterns by precisely synchronizing the tunable lens attached to the camera with the image acquisition and the pattern projection; we develop a phase unwrapping framework that fully utilizes the geometric constraint from the camera focal length setting; and we pre-calibrate the system under different focal distance to reconstruct 3D shape from unwrapped phase map. To validate the proposed idea, we developed a prototype system that can perform high-quality measurement for the depth range of approximately 1,000 mm (400 mm - 1400 mm) with the measurement error of 0.05%. Furthermore, we demonstrated that such a technique can be used for real-time 3D shape measurement by experimentally measuring moving objects.

Guided, Fusion-Based, Large Depth-of-field 3D Imaging Using a Focal Stack.

Three dimensional (3D) imaging technology has been widely used for many applications, such as human-computer interactions, making industrial measurements, and dealing with cultural relics. However, existing active methods often require both large apertures of projector and camera to maximize light throughput, resulting in a shallow working volume in which projector and camera are simultaneously in focus. In this paper, we propose a novel method to extend the working range of the structured light 3D imaging system based on the focal stack. Specifically in the case of large depth variation scenes, we first adopted the gray code method for local, 3D shape measurement with multiple focal distance settings. Then we extracted the texture map of each focus position into a focal stack to generate a global coarse depth map. Under the guidance of the global coarse depth map, the high-quality 3D shape measurement of the overall scene was obtained by local, 3D shape-measurement fusion. To validate the method, we developed a prototype system that can perform high-quality measurements in the depth range of 400 mm with a measurement error of 0.08%.

MFA-Net: Motion Feature Augmented Network for Dynamic Hand Gesture Recognition from Skeletal Data.

Dynamic hand gesture recognition has attracted increasing attention because of its importance for human⁻computer interaction. In this paper, we propose a novel motion feature augmented network (MFA-Net) for dynamic hand gesture recognition from skeletal data. MFA-Net exploits motion features of finger and global movements to augment features of deep network for gesture recognition. To describe finger articulated movements, finger motion features are extracted from the hand skeleton sequence via a variational autoencoder. Global motion features are utilized to represent the global movements of hand skeleton. These motion features along with the skeleton sequence are then fed into three branches of a recurrent neural network (RNN), which augment the motion features for RNN and improve the classification performance. The proposed MFA-Net is evaluated on two challenging skeleton-based dynamic hand gesture datasets, including DHG-14/28 dataset and SHREC'17 dataset. Experimental results demonstrate that our proposed method achieves comparable performance on DHG-14/28 dataset and better performance on SHREC'17 dataset when compared with start-of-the-art methods.

Real-Time ECG Delineation with Randomly Selected Wavelet Transform Feature and Random Walk Estimation.

Detection of Electrocardiogram (ECG) characteristic points can provide critical diagnostic information about heart diseases. We propose a novel feature extraction and machine learning scheme for ECG delineation. A new feature, termed as randomly selected wavelet transform (RSWT), is proposed to effectively represent ECG morphology. With the RSWT feature pool, a regression tree is trained to estimate the probability distribution to the direction toward the target point, relative to the current position. The continual random walk through 1D space will eventually produce a reliable region from which the final position of the target point is derived. The evaluation results on QT database show better detection accuracy compared with other studies while providing real-time processing capability.

Bidirectional Recurrent Neural Network And Convolutional Neural Network (BiRCNN) For ECG Beat Classification.

We propose a novel electrocardiogram (ECG) beat classification algorithm using a combination of Bidirectional Recurrent Neural Network (BiRNN) and Convolutional Neural Network (CNN) named as BiRCNN. Our model is an end-to-end model. The morphological features of each ECG beat is extracted by CNN. Then the features of each beat are considered in the context via BiRNN. The assessment on MIT-BIH Arrhythmia Database (MITDB) resulted in a sensitivity of 98.7% and a positive predictivity of 96.4% on average for the VEB class. For the SVEB class, the sensitivity was 92.8%, which was an over 6% promotion compared with the state-of-the-art method, and the positive predictivity was 81.9% on average. The results demonstrate the superior classification performance of our method.