Quantum recurrent neural networks for sequential learning.

Quantum neural network (QNN) is one of the promising directions where the near-term noisy intermediate-scale quantum (NISQ) devices could find advantageous applications against classical resources. Recurrent neural networks are the most fundamental networks for sequential learning, but up to now there is still a lack of canonical model of quantum recurrent neural network (QRNN), which certainly restricts the research in the field of quantum deep learning. In the present work, we propose a new kind of QRNN which would be a good candidate as the canonical QRNN model, where, the quantum recurrent blocks (QRBs) are constructed in the hardware-efficient way, and the QRNN is built by stacking the QRBs in a staggered way that can greatly reduce the algorithm's requirement with regard to the coherent time of quantum devices. That is, our QRNN is much more accessible on NISQ devices. Furthermore, the performance of the present QRNN model is verified concretely using three different kinds of classical sequential data, i.e., meteorological indicators, stock price, and text categorization. The numerical experiments show that our QRNN achieves much better performance in prediction (classification) accuracy against the classical RNN and state-of-the-art QNN models for sequential learning, and can predict the changing details of temporal sequence data. The practical circuit structure and superior performance indicate that the present QRNN is a promising learning model to find quantum advantageous applications in the near term.

Transformer for Image Harmonization and Beyond.

Image harmonization, aiming to make composite images look more realistic, is an important and challenging task. The composite, synthesized by combining foreground from one image with background from another image, inevitably suffers from the issue of inharmonious appearance caused by distinct imaging conditions, i.e., lights. Current solutions mainly adopt an encoder-decoder architecture with convolutional neural network (CNN) to capture the context of composite images, trying to understand what it should look like in the foreground referring to surrounding background. In this work, we seek to solve image harmonization with Transformer, by leveraging its powerful ability of modeling long-range context dependencies, for adjusting foreground light to make it compatible with background light while keeping structure and semantics unchanged. We present the design of our two vision Transformer frameworks and corresponding methods, as well as comprehensive experiments and empirical study, demonstrating the power of Transformer and investigating the Transformer for vision. Our methods achieve state-of-the-art performance on the image harmonization as well as four additional vision and graphics tasks, i.e., image enhancement, image inpainting, white-balance editing, and portrait relighting, indicating the superiority of our work. Code, models, more results and details can be found at the project website http://ouc.ai/project/HarmonyTransformer.

SGUIE-Net: Semantic Attention Guided Underwater Image Enhancement with Multi-Scale Perception.

Due to the wavelength-dependent light attenuation, refraction and scattering, underwater images usually suffer from color distortion and blurred details. However, due to the limited number of paired underwater images with undistorted images as reference, training deep enhancement models for diverse degradation types is quite difficult. To boost the performance of data-driven approaches, it is essential to establish more effective learning mechanisms that mine richer supervised information from limited training sample resources. In this paper, we propose a novel underwater image enhancement network, called SGUIE-Net, in which we introduce semantic information as high-level guidance via region-wise enhancement feature learning. Accordingly, we propose semantic region-wise enhancement module to better learn local enhancement features for semantic regions with multi-scale perception. After using them as complementary features and feeding them to the main branch, which extracts the global enhancement features on the original image scale, the fused features bring semantically consistent and visually superior enhancements. Extensive experiments on the publicly available datasets and our proposed dataset demonstrate the impressive performance of SGUIE-Net. The code and proposed dataset are available at https://trentqq.github.io/SGUIE-Net.html.

Generative Adversarial Network with Multi-branch Discriminator for imbalanced cross-species image-to-image translation.

There has been an increased interest in high-level image-to-image translation to achieve semantic matching. Through a powerful translation model, we can efficiently synthesize high-quality images with diverse appearances while retaining semantic matching. In this paper, we address an imbalanced learning problem using a cross-species image-to-image translation. We aim to perform the data augmentation through the image translation to boost the recognition performance of imbalanced learning. It requires a strong ability of the model to perform a biomorphic transformation on a semantic level. To tackle this problem, we propose a novel, simple, and effective structure of Multi-Branch Discriminator (termed as MBD) based on Generative Adversarial Networks (GANs). We demonstrate the effectiveness of the proposed MBD through theoretical analysis as well as empirical evaluation. We provide theoretical proof of why the proposed MBD is an effective and optimal case to achieve remarkable performance. Comprehensive experiments on various cross-species image translation tasks illustrate that our MBD can dramatically promote the performance of popular GANs with state-of-the-art results in terms of both objective and subjective assessments. Extensive downstream image recognition evaluations at a few-shot setting have also been conducted to demonstrate that the proposed method can effectively boost the performance of imbalanced learning.

TumorGAN: A Multi-Modal Data Augmentation Framework for Brain Tumor Segmentation.

The high human labor demand involved in collecting paired medical imaging data severely impedes the application of deep learning methods to medical image processing tasks such as tumor segmentation. The situation is further worsened when collecting multi-modal image pairs. However, this issue can be resolved through the help of generative adversarial networks, which can be used to generate realistic images. In this work, we propose a novel framework, named TumorGAN, to generate image segmentation pairs based on unpaired adversarial training. To improve the quality of the generated images, we introduce a regional perceptual loss to enhance the performance of the discriminator. We also develop a regional L1 loss to constrain the color of the imaged brain tissue. Finally, we verify the performance of TumorGAN on a public brain tumor data set, BraTS 2017. The experimental results demonstrate that the synthetic data pairs generated by our proposed method can practically improve tumor segmentation performance when applied to segmentation network training.

Underwater Inherent Optical Properties Estimation Using a Depth Aided Deep Neural Network.

Underwater inherent optical properties (IOPs) are the fundamental clues to many research fields such as marine optics, marine biology, and underwater vision. Currently, beam transmissometers and optical sensors are considered as the ideal IOPs measuring methods. But these methods are inflexible and expensive to be deployed. To overcome this problem, we aim to develop a novel measuring method using only a single underwater image with the help of deep artificial neural network. The power of artificial neural network has been proved in image processing and computer vision fields with deep learning technology. However, image-based IOPs estimation is a quite different and challenging task. Unlike the traditional applications such as image classification or localization, IOP estimation looks at the transparency of the water between the camera and the target objects to estimate multiple optical properties simultaneously. In this paper, we propose a novel Depth Aided (DA) deep neural network structure for IOPs estimation based on a single RGB image that is even noisy. The imaging depth information is considered as an aided input to help our model make better decision.

Feeble object detection of underwater images through LSR with delay loop.

Feeble object detection is a long-standing problem in vision based underwater exploration work. However, because of the complicated light propagation situation and high background noise, underwater images are highly degraded. Noise is not always detrimental. Logical stochastic resonance (LSR) can be a useful tool for amplifying feeble signals by utilizing the constructive interplay of noise and a nonlinear system. In the present study, an appropriate LSR structure with a delay loop is proposed to process a low-quality underwater image for enhancing the vision detection accuracy of underwater feeble objects. Ocean experiments are conducted to demonstrate the effectiveness of the proposed structure. We also give explicit numerical results to illustrate the relationship between the structure of LSR and the correct detection probability. Methods presented in this paper are quite general and can thus be potentially extended to other applications for obtaining better performance.

Automatic plankton image classification combining multiple view features via multiple kernel learning.

BACKGROUND: Plankton, including phytoplankton and zooplankton, are the main source of food for organisms in the ocean and form the base of marine food chain. As the fundamental components of marine ecosystems, plankton is very sensitive to environment changes, and the study of plankton abundance and distribution is crucial, in order to understand environment changes and protect marine ecosystems. This study was carried out to develop an extensive applicable plankton classification system with high accuracy for the increasing number of various imaging devices. Literature shows that most plankton image classification systems were limited to only one specific imaging device and a relatively narrow taxonomic scope. The real practical system for automatic plankton classification is even non-existent and this study is partly to fill this gap. RESULTS: Inspired by the analysis of literature and development of technology, we focused on the requirements of practical application and proposed an automatic system for plankton image classification combining multiple view features via multiple kernel learning (MKL). For one thing, in order to describe the biomorphic characteristics of plankton more completely and comprehensively, we combined general features with robust features, especially by adding features like Inner-Distance Shape Context for morphological representation. For another, we divided all the features into different types from multiple views and feed them to multiple classifiers instead of only one by combining different kernel matrices computed from different types of features optimally via multiple kernel learning. Moreover, we also applied feature selection method to choose the optimal feature subsets from redundant features for satisfying different datasets from different imaging devices. We implemented our proposed classification system on three different datasets across more than 20 categories from phytoplankton to zooplankton. The experimental results validated that our system outperforms state-of-the-art plankton image classification systems in terms of accuracy and robustness. CONCLUSIONS: This study demonstrated automatic plankton image classification system combining multiple view features using multiple kernel learning. The results indicated that multiple view features combined by NLMKL using three kernel functions (linear, polynomial and Gaussian kernel functions) can describe and use information of features better so that achieve a higher classification accuracy.

Object extraction from underwater images through logical stochastic resonance.

Logical stochastic resonance (LSR), the phenomenon in which the interplay of noise and nonlinearity can raise the accurate probability of response to feeble input signals, is studied in this Lettter to extract objects from highly degraded underwater images. Images captured under water, especially in the turbid areas, always suffer from interference through heavy noise caused by the suspended particles. Inherent noise and nonlinearity cause difficulty in processing these images through conventional image processing methods. However, LSR can optimally address such issues. A heavily degraded image is first extended to a 1D form in the direction determined by the illumination condition, and then normalized to be placed in the LSR system as an input signal. Additional Gaussian noise is added to the system as the auxiliary power to help separate the object and the background. Results in the natural offshore area demonstrate the effect of LSR on image processing, and the proposed method creates an interesting direction in the processing of heavily degraded images.

Automatic setae segmentation from Chaetoceros microscopic images.

A novel image processing model Grayscale Surface Direction Angle Model (GSDAM) is presented and the algorithm based on GSDAM is developed to segment setae from Chaetoceros microscopic images. The proposed model combines the setae characteristics of the microscopic images with the spatial analysis of image grayscale surface to detect and segment the direction thin and long setae from the low contrast background as well as noise which may make the commonly used segmentation methods invalid. The experimental results show that our algorithm based on GSDAM outperforms the boundary-based and region-based segmentation methods Canny edge detector, iterative threshold selection, Otsu's thresholding, minimum error thresholding, K-means clustering, and marker-controlled watershed on the setae segmentation more accurately and completely.