Altered amygdala functional connectivity after real-time functional MRI emotion self-regulation training.

Real-time functional MRI neurofeedback (rtfMRI-NF) is a noninvasive technique that extracts concurrent brain states and provides feedback to subjects in an online method. Our study aims to investigate the effect of rtfMRI-NF on amygdala-based emotion self-regulation by analyzing resting-state functional connectivity. We conducted a task experiment to train subjects in self-regulating amygdala activity in response to emotional stimuli. Twenty subjects were divided into two groups. The up-regulate group (URG) viewed positive stimulus, while the down-regulate group (DRG) viewed negative stimulus. The rtfMRI-NF experiment paradigm consisted of three conditions. The URG's percent amplitude fluctuation (PerAF) scores are significant, indicating that positive emotions may be a partial side effect, with increased activity in the left hemisphere. Resting-state functional connectivity was analyzed via a paired-sample t-test before and after neurofeedback training. Brain network properties and functional connectivity analysis showed a significant difference between the default mode network (DMN) and the brain region associated with the limbic system. These results reveal to some extent the mechanism of neurofeedback training to improve individuals' emotional regulate regulation ability. Our study has shown that rtfMRI-neurofeedback training can effectively enhance the ability to voluntarily control brain responses. Furthermore, the results of the functional analysis have revealed distinct changes in the amygdala functional connectivity circuits following rtfMRI-neurofeedback training. These findings may suggest the potential clinical applications of rtfMRI-neurofeedback as a new therapy for emotionally related mental disorders.

Improving the Transferability of Adversarial Examples With a Noise Data Enhancement Framework and Random Erasing.

Deep neural networks (DNNs) are proven vulnerable to attack against adversarial examples. Black-box transfer attacks pose a massive threat to AI applications without accessing target models. At present, the most effective black-box attack methods mainly adopt data enhancement methods, such as input transformation. Previous data enhancement frameworks only work on input transformations that satisfy accuracy or loss invariance. However, it does not work for other transformations that do not meet the above conditions, such as the transformation which will lose information. To solve this problem, we propose a new noise data enhancement framework (NDEF), which only transforms adversarial perturbation to avoid the above issues effectively. In addition, we introduce random erasing under this framework to prevent the over-fitting of adversarial examples. Experimental results show that the black-box attack success rate of our method Random Erasing Iterative Fast Gradient Sign Method (REI-FGSM) is 4.2% higher than DI-FGSM in six models on average and 6.6% higher than DI-FGSM in three defense models. REI-FGSM can combine with other methods to achieve excellent performance. The attack performance of SI-FGSM can be improved by 22.9% on average when combined with REI-FGSM. Besides, our combined version with DI-TI-MI-FGSM, i.e., DI-TI-MI-REI-FGSM can achieve an average attack success rate of 97.0% against three ensemble adversarial training models, which is greater than the current gradient iterative attack method. We also introduce Gaussian blur to prove the compatibility of our framework.

Dissociable Neural Representations of Adversarially Perturbed Images in Convolutional Neural Networks and the Human Brain.

Despite the remarkable similarities between convolutional neural networks (CNN) and the human brain, CNNs still fall behind humans in many visual tasks, indicating that there still exist considerable differences between the two systems. Here, we leverage adversarial noise (AN) and adversarial interference (AI) images to quantify the consistency between neural representations and perceptual outcomes in the two systems. Humans can successfully recognize AI images as the same categories as their corresponding regular images but perceive AN images as meaningless noise. In contrast, CNNs can recognize AN images similar as corresponding regular images but classify AI images into wrong categories with surprisingly high confidence. We use functional magnetic resonance imaging to measure brain activity evoked by regular and adversarial images in the human brain, and compare it to the activity of artificial neurons in a prototypical CNN-AlexNet. In the human brain, we find that the representational similarity between regular and adversarial images largely echoes their perceptual similarity in all early visual areas. In AlexNet, however, the neural representations of adversarial images are inconsistent with network outputs in all intermediate processing layers, providing no neural foundations for the similarities at the perceptual level. Furthermore, we show that voxel-encoding models trained on regular images can successfully generalize to the neural responses to AI images but not AN images. These remarkable differences between the human brain and AlexNet in representation-perception association suggest that future CNNs should emulate both behavior and the internal neural presentations of the human brain.

ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping.

With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this field. However, the images generated by previous GAN-based methods often show instability. The fundamental reason is that the GAN in these frameworks is difficult to converge to the distribution of face space in training completely. To solve this problem, we propose a novel face-swapping method based on pretrained StyleGAN generator with a stronger ability of high-quality face image generation. The critical issue is how to control StyleGAN to generate swapped images accurately. We design the control strategy of the generator based on the idea of encoding and decoding and propose an encoder called ShapeEditor to complete this task. ShapeEditor is a two-step encoder used to generate a set of coding vectors that integrate the identity and attribute of the input faces. In the first step, we extract the identity vector of the source image and the attribute vector of the target image; in the second step, we map the concatenation of the identity vector and attribute vector onto the potential internal space of StyleGAN. Extensive experiments on the test dataset show that the results of the proposed method are not only superior in clarity and authenticity than other state-of-the-art methods but also sufficiently integrate identity and attribute.

Multi-energy CT reconstruction using tensor nonlocal similarity and spatial sparsity regularization.

BACKGROUND: Multi-energy computed tomography (MECT) based on a photon-counting detector is an emerging imaging modality that collects projections at several energy bins with a single scan. However, the limited number of photons collected into the divided, narrow energy bins results in high quantum noise levels in reconstructed images. This study aims to improve MECT image quality by minimizing noise levels while retaining image details. METHODS: A novel MECT reconstruction method was proposed by exploiting the nonlocal tensor similarity among interchannel images and spatial sparsity in single-channel images. Similar patches were initially extracted from the interchannel images in spectral and spatial domains, then stacked into a new three-order tensor. Intrinsic tensor sparsity regularization that combined the Tuker and canonical polyadic (CP) low-rank decomposition techniques were applied to exploit the nonlocal similarity of the formulated tensor. Spatial sparsity in single-channel images was modeled by total variation (TV) regularization that utilizes the compressibility of gradient image. A new MECT reconstruction model was established by simultaneously incorporating the intrinsic tensor sparsity and TV regularizations. The iterative alternating minimization method was utilized to solve the reconstruction model based on a flexible framework. RESULTS: The proposed method was applied to the digital phantom and real mouse data to assess its feasibility and reliability. The reconstruction and decomposition results in the mouse data were encouraging and demonstrated the ability of the proposed method in noise suppression while preserving image details, not observed with other methods. Imaging data from the digital phantom illustrated this method as achieving the best intuitive reconstruction and decomposition results among all compared methods. They reduced the root mean square error (RMSE) by 89.75%, 50.75%, and 36.54% on the reconstructed images compared with analytic, TV-based, and tensor-based methods, respectively. This phenomenon was also observed with decomposition results, where the RMSE was also reduced by 97.96%, 67.74%, 72.05%, respectively. CONCLUSIONS: In this study, we proposed a reconstruction method for photon counting detector-based MECT, using the intrinsic tensor sparsity and TV regularizations. Improvements in noise suppression and detail preservation in the digital phantom and real mouse data were validated by the qualitative and quantitative evaluations on the reconstruction and decomposition results, verifying the potential of the proposed method in MECT reconstruction.

Reconstruction method for DECT with one half-scan plus a second limited-angle scan using prior knowledge of complementary support set (Pri-CSS).

Dual-energy computed tomography (DECT) has capability to improve material differentiation, but most scanning schemes require two sets of full-scan measurements at different x-ray spectra, limiting its application to imaging system with incomplete scan. In this study, using one half-scan and a second limited-angle scan, we propose a DECT reconstruction method by exploiting the consistent information of gradient images at high- and low-energy spectra, which relaxes the requirement of data acquisition of DECT. Based on the theory of sampling condition analysis, the complementary support set of gradient images plays an important role in image reconstruction because it constitutes the sufficient and necessary condition for accurate CT reconstruction. For DECT, the gradient images of high- and low-energy CT images ideally share the same complementary support set for the same object. Inspired by this idea, we extract the prior knowledge of complementary support set (Pri-CSS) from the gradient image of the first half-scan CT image to promote the second limited-angle CT reconstruction. Pri-CSS will be incorporated into total variation regularization model in the form of constrains. Alternative direction method is applied to iteratively solve the modified optimization model, thereby deriving the proposed algorithm to recover low-energy CT image from limited-angle measurements. The qualitative and quantitative experiments on digital and real data are performed to validate the proposed method. The results show that the proposed method outperforms its counterparts and achieve high reconstruction quality for the designed scanning configuration.

A visual encoding model based on deep neural networks and transfer learning for brain activity measured by functional magnetic resonance imaging.

BACKGROUND: Building visual encoding models to accurately predict visual responses is a central challenge for current vision-based brain-machine interface techniques. To achieve high prediction accuracy on neural signals, visual encoding models should include precise visual features and appropriate prediction algorithms. Most existing visual encoding models employ hand-craft visual features (e.g., Gabor wavelets or semantic labels) or data-driven features (e.g., features extracted from deep neural networks (DNN)). They also assume a linear mapping between feature representations to brain activity. However, it remains unknown whether such linear mapping is sufficient for maximizing prediction accuracy. NEW METHOD: We construct a new visual encoding framework to predict cortical responses in a benchmark functional magnetic resonance imaging (fMRI) dataset. In this framework, we employ the transfer learning technique to incorporate a pre-trained DNN (i.e., AlexNet) and train a nonlinear mapping from visual features to brain activity. This nonlinear mapping replaces the conventional linear mapping and is supposed to improve prediction accuracy on measured activity in the human visual cortex. RESULTS: The proposed framework can significantly predict responses of over 20% voxels in early visual areas (i.e., V1-lateral occipital region, LO) and achieve unprecedented prediction accuracy. COMPARISON WITH EXISTING METHODS: Comparing to two conventional visual encoding models, we find that the proposed encoding model shows consistent higher prediction accuracy in all early visual areas, especially in relatively anterior visual areas (i.e., V4 and LO). CONCLUSIONS: Our work proposes a new framework to utilize pre-trained visual features and train non-linear mappings from visual features to brain activity.

Optimization-based region-of-interest reconstruction for X-ray computed tomography based on total variation and data derivative.

Region-of-interest (ROI) and interior reconstructions for computed tomography (CT) have drawn much attention and can be of practical value for potential applications in reducing radiation dose and hardware cost. The conventional wisdom is that the exact reconstruction of an interior ROI is very difficult to be obtained by only using data associated with lines through the ROI. In this study, we propose and investigate optimization-based methods for ROI and interior reconstructions based on total variation (TV) and data derivative. Objective functions are built by the image TV term plus the data finite difference term. Different data terms in the forms of L1-norm, L2-norm, and Kullback-Leibler divergence are incorporated and investigated in the optimizations. Efficient algorithms are developed using the proximal alternating direction method of multipliers (ADMM) for each program. All sub-problems of ADMM are solved by using closed-form solutions with high efficiency. The customized optimizations and algorithms based on the TV and derivative-based data terms can serve as a powerful tool for interior reconstructions. Simulations and real-data experiments indicate that the proposed methods can be of practical value for CT imaging applications.

Block matching sparsity regularization-based image reconstruction for incomplete projection data in computed tomography.

In medical imaging many conventional regularization methods, such as total variation or total generalized variation, impose strong prior assumptions which can only account for very limited classes of images. A more reasonable sparse representation frame for images is still badly needed. Visually understandable images contain meaningful patterns, and combinations or collections of these patterns can be utilized to form some sparse and redundant representations which promise to facilitate image reconstructions. In this work, we propose and study block matching sparsity regularization (BMSR) and devise an optimization program using BMSR for computed tomography (CT) image reconstruction for an incomplete projection set. The program is built as a constrained optimization, minimizing the L1-norm of the coefficients of the image in the transformed domain subject to data observation and positivity of the image itself. To solve the program efficiently, a practical method based on the proximal point algorithm is developed and analyzed. In order to accelerate the convergence rate, a practical strategy for tuning the BMSR parameter is proposed and applied. The experimental results for various settings, including real CT scanning, have verified the proposed reconstruction method showing promising capabilities over conventional regularization.

Sparse-view image reconstruction via total absolute curvature combining total variation for X-ray computed tomography.

Sparse-view imaging is a promising scanning approach which has fast scanning rate and low-radiation dose in X-ray computed tomography (CT). Conventional L1-norm based total variation (TV) has been widely used in image reconstruction since the advent of compressive sensing theory. However, with only the first order information of the image used, the TV often generates dissatisfactory image for some applications. As is widely known, image curvature is among the most important second order features of images and can potentially be applied in image reconstruction for quality improvement. This study incorporates the curvature in the optimization model and proposes a new total absolute curvature (TAC) based reconstruction method. The proposed model contains both total absolute curvature and total variation (TAC-TV), which are intended for better description of the featured complicated image. As for the practical algorithm development, the efficient alternating direction method of multipliers (ADMM) is utilized, which generates a practical and easy-coded algorithm. The TAC-TV iterations mainly contain FFTs, soft-thresholding and projection operations and can be launched on graphics processing unit, which leads to relatively high performance. To evaluate the presented algorithm, both qualitative and quantitative studies were performed using various few view datasets. The results illustrated that the proposed approach yielded better reconstruction quality and satisfied convergence property compared with TV-based methods.

Optimization-based image reconstruction in computed tomography by alternating direction method with ordered subsets.

Nowadays, diversities of task-specific applications for computed tomography (CT) have already proposed multiple challenges for algorithm design of image reconstructions. Consequently, efficient algorithm design tool is necessary to be established. A fast and efficient algorithm design framework for CT image reconstruction, which is based on alternating direction method (ADM) with ordered subsets (OS), is proposed, termed as OS-ADM. The general ideas of ADM and OS have been abstractly introduced and then they are combined for solving convex optimizations in CT image reconstruction. Standard procedures are concluded for algorithm design which contain 1) model mapping, 2) sub-problem dividing and 3) solving, 4) OS level setting and 5) algorithm evaluation. Typical reconstruction problems are modeled as convex optimizations, including (non-negative) least-square, constrained L1 minimization, constrained total variation (TV) minimization and TV minimizations with different data fidelity terms. Efficient working algorithms for these problems are derived with detailed derivations by the proposed framework. In addition, both simulations and real CT projections are tested to verify the performances of two TV-based algorithms. Experimental investigations indicate that these algorithms are of the state-of-the-art performances. The algorithm instances show that the proposed OS-ADM framework is promising for practical applications.

A two-step filtering-based iterative image reconstruction method for interior tomography.

The optimization-based method that utilizes the additional sparse prior of region-of-interest (ROI) image, such as total variation, has been the subject of considerable research in problems of interior tomography reconstruction. One challenge for optimization-based iterative ROI image reconstruction is to build the relationship between ROI image and truncated projection data. When the reconstruction support region is smaller than the original object, an unsuitable representation of data fidelity may lead to bright truncation artifacts in the boundary region of field of view. In this work, we aim to develop an iterative reconstruction method to suppress the truncation artifacts and improve the image quality for direct ROI image reconstruction. A novel reconstruction approach is proposed based on an optimization problem involving a two-step filtering-based data fidelity. Data filtering is achieved in two steps: the first takes the derivative of projection data; in the second step, Hilbert filtering is applied in the differentiated data. Numerical simulations and real data reconstructions have been conducted to validate the new reconstruction method. Both qualitative and quantitative results indicate that, as theoretically expected, the proposed method brings reasonable performance in suppressing truncation artifacts and preserving detailed features. The presented local reconstruction method based on the two-step filtering strategy provides a simple and efficient approach for the iterative reconstruction from truncated projections.

Iterative metal artifact reduction for x-ray computed tomography using unmatched projector/backprojector pairs.

PURPOSE: Metal artifact reduction (MAR) is a major problem and a challenging issue in x-ray computed tomography (CT) examinations. Iterative reconstruction from sinograms unaffected by metals shows promising potential in detail recovery. This reconstruction has been the subject of much research in recent years. However, conventional iterative reconstruction methods easily introduce new artifacts around metal implants because of incomplete data reconstruction and inconsistencies in practical data acquisition. Hence, this work aims at developing a method to suppress newly introduced artifacts and improve the image quality around metal implants for the iterative MAR scheme. METHODS: The proposed method consists of two steps based on the general iterative MAR framework. An uncorrected image is initially reconstructed, and the corresponding metal trace is obtained. The iterative reconstruction method is then used to reconstruct images from the unaffected sinogram. In the reconstruction step of this work, an iterative strategy utilizing unmatched projector/backprojector pairs is used. A ramp filter is introduced into the back-projection procedure to restrain the inconsistency components in low frequencies and generate more reliable images of the regions around metals. Furthermore, a constrained total variation (TV) minimization model is also incorporated to enhance efficiency. The proposed strategy is implemented based on an iterative FBP and an alternating direction minimization (ADM) scheme, respectively. The developed algorithms are referred to as "iFBP-TV" and "TV-FADM," respectively. Two projection-completion-based MAR methods and three iterative MAR methods are performed simultaneously for comparison. RESULTS: The proposed method performs reasonably on both simulation and real CT-scanned datasets. This approach could reduce streak metal artifacts effectively and avoid the mentioned effects in the vicinity of the metals. The improvements are evaluated by inspecting regions of interest and by comparing the root-mean-square errors, normalized mean absolute distance, and universal quality index metrics of the images. Both iFBP-TV and TV-FADM methods outperform other counterparts in all cases. Unlike the conventional iterative methods, the proposed strategy utilizing unmatched projector/backprojector pairs shows excellent performance in detail preservation and prevention of the introduction of new artifacts. CONCLUSIONS: Qualitative and quantitative evaluations of experimental results indicate that the developed method outperforms classical MAR algorithms in suppressing streak artifacts and preserving the edge structural information of the object. In particular, structures lying close to metals can be gradually recovered because of the reduction of artifacts caused by inconsistency effects.

Constrained Total Generalized p-Variation Minimization for Few-View X-Ray Computed Tomography Image Reconstruction.

Total generalized variation (TGV)-based computed tomography (CT) image reconstruction, which utilizes high-order image derivatives, is superior to total variation-based methods in terms of the preservation of edge information and the suppression of unfavorable staircase effects. However, conventional TGV regularization employs l1-based form, which is not the most direct method for maximizing sparsity prior. In this study, we propose a total generalized p-variation (TGpV) regularization model to improve the sparsity exploitation of TGV and offer efficient solutions to few-view CT image reconstruction problems. To solve the nonconvex optimization problem of the TGpV minimization model, we then present an efficient iterative algorithm based on the alternating minimization of augmented Lagrangian function. All of the resulting subproblems decoupled by variable splitting admit explicit solutions by applying alternating minimization method and generalized p-shrinkage mapping. In addition, approximate solutions that can be easily performed and quickly calculated through fast Fourier transform are derived using the proximal point method to reduce the cost of inner subproblems. The accuracy and efficiency of the simulated and real data are qualitatively and quantitatively evaluated to validate the efficiency and feasibility of the proposed method. Overall, the proposed method exhibits reasonable performance and outperforms the original TGV-based method when applied to few-view problems.

Efficient TpV minimization for circular, cone-beam computed tomography reconstruction via non-convex optimization.

An efficient iterative algorithm, based on recent work in non-convex optimization and generalized p-shrinkage mappings, is proposed for volume image reconstruction from circular cone-beam scans. Conventional total variation regularization makes use of L1 norm of gradient magnitude images (GMI). However, this paper utilizes a generalized penalty function, induced by p-shrinkage, of GMI which is proven to be a better measurement of its sparsity. The reconstruction model is formed using generalized total p-variation (TpV) minimization, which differs with the state of the art methods, with the constraint that the estimated projection data is within a specified tolerance of the available data and that the values of the volume image are non-negative. Theoretically, the proximal mapping for penalty functions induced by p-shrinkage has an exact and closed-form expression; thus, the constrained optimization can be stably and efficiently solved by the alternating direction minimization (ADM) scheme. Each sub-problem decoupled by variable splitting is minimized by explicit and easy-to-implement formulas developed by ADM. The proposed algorithm is efficiently implemented using a graphics processing unit and is referred to as "TpV-ADM." This method is robust and accurate even for very few view reconstruction datasets. Verifications and comparisons performed using various datasets (including ideal, noisy, and real projections) illustrate that the proposed method is effective and promising.

System matrix analysis for sparse-view iterative image reconstruction in X-ray CT.

Iterative image reconstruction (IIR) with sparsity-exploiting methods, such as total variation (TV) minimization, used for investigations in compressive sensing (CS) claim potentially large reductions in sampling requirements. Quantifying this claim for computed tomography (CT) is non-trivial, as both the singularity of undersampled reconstruction and the sufficient view number for sparse-view reconstruction are ill-defined. In this paper, the singular value decomposition method is used to study the condition number and singularity of the system matrix and the regularized matrix. An estimation method of the empirical lower bound is proposed, which is helpful for estimating the number of projection views required for exact reconstruction. Simulation studies show that the singularity of the system matrices for different projection views is effectively reduced by regularization. Computing the condition number of a regularized matrix is necessary to provide a reference for evaluating the singularity and recovery potential of reconstruction algorithms using regularization. The empirical lower bound is helpful for estimating the projections view number with a sparse reconstruction algorithm.

Distributed CT image reconstruction algorithm based on the alternating direction method.

With the development of compressive sensing theory, image reconstruction from few-view projections has been paid considerable research attention in the field of computed tomography (CT). Total variation (TV)-based CT image reconstruction has been shown experimentally to be capable of producing accurate reconstructions from sparse-view data. Motivated by the need of solving few-view reconstruction problem with large scale data, a general block distribution reconstruction algorithm based on TV minimization and the alternating direction method (ADM) has been developed in this study. By utilizing the inexact ADM, which involves linearization and proximal point techniques, the algorithm is relatively simple and hence convenient for the derivation and distributed implementation. And because the data as well as the computation are distributed to individual nodes, an outstanding acceleration factor is achieved. Experimental results demonstrate that the proposed method can accelerate the alternating direction total variation minimization (ADTVM) algorithm with nearly no loss of accuracy, which means compared with ADTVM, the proposed algorithm has a better accuracy with same running time.

Sparse models for visual image reconstruction from fMRI activity.

Statistical model is essential for constraint-free visual image reconstruction, as it may overfit training data and have poor generalization. In this study, we investigate the sparsity of the distributed patterns of visual representation and introduce a suitable sparse model for the visual image reconstruction experiment. We use elastic net regularization to model the sparsity of the distributed patterns for local decoder training. We also investigate the relationship between the sparsity of the visual representation and sparse models with different parameters. Our experimental results demonstrate that the sparsity needed by visual reconstruction models differs from the sparsest one, and the l2-norm regularization introduced in the EN model improves not only the robustness of the model but also the generalization performance of the learning results. We therefore conclude that the sparse learning model for visual image reconstruction should reflect the spasity of visual perceptual experience, and have a solution with high but not the highest sparsity, and some robustness as well.

Distributed reconstruction via alternating direction method.

With the development of compressive sensing theory, image reconstruction from few-view projections has received considerable research attentions in the field of computed tomography (CT). Total-variation- (TV-) based CT image reconstruction has been shown to be experimentally capable of producing accurate reconstructions from sparse-view data. In this study, a distributed reconstruction algorithm based on TV minimization has been developed. This algorithm is very simple as it uses the alternating direction method. The proposed method can accelerate the alternating direction total variation minimization (ADTVM) algorithm without losing accuracy.

Semi-orthogonal Frame Wavelets and Parseval Frame Wavelets Associated with GMRA.

In this paper, we study semi-orthogonal frame wavelets and Parseval frame wavelets(PFWs) in L(2)(R(d)) with matrix dilations of form (Df)(x)=2f(Ax), where A is an arbitrary expanding d x d matrix with integer coefficients, such that |detA| = 2. Firstly, we obtain a necessary and sufficient condition for a frame wavelet to be a semi-orthogonal frame wavelet. Secondly, we present a necessary condition for the semi-orthogonal frame wavelets. When the frame wavelets are the PFWs, we prove that all PFWs associated with generalized multiresolution analysis (GMRA) are equivalent to a closed subspace W(0) for which {T(k) psi : k in Z(d)} is a PFW. Finally, by showing the relation between principal shift invariant spaces and their bracket function, we discover a property of the PFWs associated with GMRA by the PFWs' minimal vector-filter. In each section, we construct concrete examples.