Electronic medical records imputation by temporal Generative Adversarial Network.

The loss of electronic medical records has seriously affected the practical application of biomedical data. Therefore, it is a meaningful research effort to effectively fill these lost data. Currently, state-of-the-art methods focus on using Generative Adversarial Networks (GANs) to fill the missing values of electronic medical records, achieving breakthrough progress. However, when facing datasets with high missing rates, the imputation accuracy of these methods sharply deceases. This motivates us to explore the uncertainty of GANs and improve the GAN-based imputation methods. In this paper, the GRUD (Gate Recurrent Unit Decay) network and the UGAN (Uncertainty Generative Adversarial Network) are proposed and organically combined, called UGAN-GRUD. In UGAN-GRUD, it highlights using GAN to generate imputation values and then leveraging GRUD to compensate them. We have designed the UGAN and the GRUD network. The former is employed to learn the distribution pattern and uncertainty of data through the Generator and Discriminator, iteratively. The latter is exploited to compensate the former by leveraging the GRUD based on time decay factor, which can learn the specific temporal relations in electronic medical records. Through experimental research on publicly available biomedical datasets, the results show that UGAN-GRUD outperforms the current state-of-the-art methods, with average 13% RMSE (Root Mean Squared Error) and 24.5% MAPE (Mean Absolute Percentage Error) improvements.

ContrAttNet: Contribution and attention approach to multivariate time-series data imputation.

The imputation of missing values in multivariate time-series data is a basic and popular data processing technology. Recently, some studies have exploited Recurrent Neural Networks (RNNs) and Generative Adversarial Networks (GANs) to impute/fill the missing values in multivariate time-series data. However, when faced with datasets with high missing rates, the imputation error of these methods increases dramatically. To this end, we propose a neural network model based on dynamic contribution and attention, denoted as ContrAttNet. ContrAttNet consists of three novel modules: feature attention module, iLSTM (imputation Long Short-Term Memory) module, and 1D-CNN (1-Dimensional Convolutional Neural Network) module. ContrAttNet exploits temporal information and spatial feature information to predict missing values, where iLSTM attenuates the memory of LSTM according to the characteristics of the missing values, to learn the contributions of different features. Moreover, the feature attention module introduces an attention mechanism based on contributions, to calculate supervised weights. Furthermore, under the influence of these supervised weights, 1D-CNN processes the time-series data by treating them as spatial features. Experimental results show that ContrAttNet outperforms other state-of-the-art models in the missing value imputation of multivariate time-series data, with average 6% MAPE and 9% MAE on the benchmark datasets.

PeNet: A feature excitation learning approach to advertisement click-through rate prediction.

Since the physical meaning of the fields of the dataset is unknown, we have to use the feature interaction method to select the correlated features and exclude uncorrelated features. The current state-of-the-art methods employ various methods based on feature interaction to predict advertisement Click-Through Rate (CTR); however, the feature interaction based on potential new feature mining is rarely considered, which can provide effective assistance for feature interaction. This motivates us to investigate methods that combine potential new features and feature interactions. Thus, we propose a potential feature excitation learning network (PeNet), which is a neural network model based on feature combination and feature interaction. In PeNet, we treat the row compression and column compression of the original feature matrix as potential new features, and proposed the excitation learning mechanism that is a weighted mechanism based on residual principle. Through this excitation learning mechanism, the original embedded features and potential new features are subjected to weighted interaction based on the residual principle. Moreover, a deep neural network is exploited to iteratively learn and iteratively combine features. The excitation learning structure of PeNet neural network is well demonstrated in this paper, that is, the control flow of embedding, compression, excitation and output, which further strengthens the correlated features and weakens the uncorrelated features by compressing and expanding the features. Experimental results on multiple benchmark datasets indicate the PeNet as a general-purpose plug-in has more superior performance and better efficiency than previous state-of-the-art methods.