Aggregating Different Scales of Attention on Feature Variants for Tomato Leaf Disease Diagnosis from Image Data: A Transformer Driven Study.

Tomato leaf diseases can incur significant financial damage by having adverse impacts on crops and, consequently, they are a major concern for tomato growers all over the world. The diseases may come in a variety of forms, caused by environmental stress and various pathogens. An automated approach to detect leaf disease from images would assist farmers to take effective control measures quickly and affordably. Therefore, the proposed study aims to analyze the effects of transformer-based approaches that aggregate different scales of attention on variants of features for the classification of tomato leaf diseases from image data. Four state-of-the-art transformer-based models, namely, External Attention Transformer (EANet), Multi-Axis Vision Transformer (MaxViT), Compact Convolutional Transformers (CCT), and Pyramid Vision Transformer (PVT), are trained and tested on a multiclass tomato disease dataset. The result analysis showcases that MaxViT comfortably outperforms the other three transformer models with 97% overall accuracy, as opposed to the 89% accuracy achieved by EANet, 91% by CCT, and 93% by PVT. MaxViT also achieves a smoother learning curve compared to the other transformers. Afterwards, we further verified the legitimacy of the results on another relatively smaller dataset. Overall, the exhaustive empirical analysis presented in the paper proves that the MaxViT architecture is the most effective transformer model to classify tomato leaf disease, providing the availability of powerful hardware to incorporate the model.

Multi-Stage Network for Event-Based Video Deblurring with Residual Hint Attention.

Video deblurring aims at removing the motion blur caused by the movement of objects or camera shake. Traditional video deblurring methods have mainly focused on frame-based deblurring, which takes only blurry frames as the input to produce sharp frames. However, frame-based deblurring has shown poor picture quality in challenging cases of video restoration where severely blurred frames are provided as the input. To overcome this issue, recent studies have begun to explore the event-based approach, which uses the event sequence captured by an event camera for motion deblurring. Event cameras have several advantages compared to conventional frame cameras. Among these advantages, event cameras have a low latency in imaging data acquisition (0.001 ms for event cameras vs. 10 ms for frame cameras). Hence, event data can be acquired at a high acquisition rate (up to one microsecond). This means that the event sequence contains more accurate motion information than video frames. Additionally, event data can be acquired with less motion blur. Due to these advantages, the use of event data is highly beneficial for achieving improvements in the quality of deblurred frames. Accordingly, the results of event-based video deblurring are superior to those of frame-based deblurring methods, even for severely blurred video frames. However, the direct use of event data can often generate visual artifacts in the final output frame (e.g., image noise and incorrect textures), because event data intrinsically contain insufficient textures and event noise. To tackle this issue in event-based deblurring, we propose a two-stage coarse-refinement network by adding a frame-based refinement stage that utilizes all the available frames with more abundant textures to further improve the picture quality of the first-stage coarse output. Specifically, a coarse intermediate frame is estimated by performing event-based video deblurring in the first-stage network. A residual hint attention (RHA) module is also proposed to extract useful attention information from the coarse output and all the available frames. This module connects the first and second stages and effectively guides the frame-based refinement of the coarse output. The final deblurred frame is then obtained by refining the coarse output using the residual hint attention and all the available frame information in the second-stage network. We validated the deblurring performance of the proposed network on the GoPro synthetic dataset (33 videos and 4702 frames) and the HQF real dataset (11 videos and 2212 frames). Compared to the state-of-the-art method (D2Net), we achieved a performance improvement of 1 dB in PSNR and 0.05 in SSIM on the GoPro dataset, and an improvement of 1.7 dB in PSNR and 0.03 in SSIM on the HQF dataset.

Hint-Based Image Colorization Based on Hierarchical Vision Transformer.

Hint-based image colorization is an image-to-image translation task that aims at creating a full-color image from an input luminance image when a small set of color values for some pixels are given as hints. Though traditional deep-learning-based methods have been proposed in the literature, they are based on convolution neural networks (CNNs) that have strong spatial locality due to the convolution operations. This often causes non-trivial visual artifacts in the colorization results, such as false color and color bleeding artifacts. To overcome this limitation, this study proposes a vision transformer-based colorization network. The proposed hint-based colorization network has a hierarchical vision transformer architecture in the form of an encoder-decoder structure based on transformer blocks. As the proposed method uses the transformer blocks that can learn rich long-range dependency, it can achieve visually plausible colorization results, even with a small number of color hints. Through the verification experiments, the results reveal that the proposed transformer model outperforms the conventional CNN-based models. In addition, we qualitatively analyze the effect of the long-range dependency of the transformer model on hint-based image colorization.

Multi-Scale Attention-Guided Non-Local Network for HDR Image Reconstruction.

High-dynamic-range (HDR) image reconstruction methods are designed to fuse multiple Low-dynamic-range (LDR) images captured with different exposure values into a single HDR image. Recent CNN-based methods mostly perform local attention- or alignment-based fusion of multiple LDR images to create HDR contents. Depending on a single attention mechanism or alignment causes failure in compensating ghosting artifacts, which can arise in the synthesized HDR images due to the motion of objects or camera movement across different LDR image inputs. In this study, we propose a multi-scale attention-guided non-local network called MSANLnet for efficient HDR image reconstruction. To mitigate the ghosting artifacts, the proposed MSANLnet performs implicit alignment of LDR image features with multi-scale spatial attention modules and then reconstructs pixel intensity values using long-range dependencies through non-local means-based fusion. These modules adaptively select useful information that is not damaged by an object's movement or unfavorable lighting conditions for image pixel fusion. Quantitative evaluations against several current state-of-the-art methods show that the proposed approach achieves higher performance than the existing methods. Moreover, comparative visual results show the effectiveness of the proposed method in restoring saturated information from original input images and mitigating ghosting artifacts caused by large movement of objects. Ablation studies show the effectiveness of the proposed method, architectural choices, and modules for efficient HDR reconstruction.

Global and Local Attention-Based Free-Form Image Inpainting.

Deep-learning-based image inpainting methods have shown significant promise in both rectangular and irregular holes. However, the inpainting of irregular holes presents numerous challenges owing to uncertainties in their shapes and locations. When depending solely on convolutional neural network (CNN) or adversarial supervision, plausible inpainting results cannot be guaranteed because irregular holes need attention-based guidance for retrieving information for content generation. In this paper, we propose two new attention mechanisms, namely a mask pruning-based global attention module and a global and local attention module to obtain global dependency information and the local similarity information among the features for refined results. The proposed method is evaluated using state-of-the-art methods, and the experimental results show that our method outperforms the existing methods in both quantitative and qualitative measures.

Deep Color Transfer for Color-Plus-Mono Dual Cameras.

A few approaches have studied image fusion using color-plus-mono dual cameras to improve the image quality in low-light shooting. Among them, the color transfer approach, which transfers the color information of a color image to a mono image, is considered to be promising for obtaining improved images with less noise and more detail. However, the color transfer algorithms rely heavily on appropriate color hints from a given color image. Unreliable color hints caused by errors in stereo matching of a color-plus-mono image pair can generate various visual artifacts in the final fused image. This study proposes a novel color transfer method that seeks reliable color hints from a color image and colorizes a corresponding mono image with reliable color hints that are based on a deep learning model. Specifically, a color-hint-based mask generation algorithm is developed to obtain reliable color hints. It removes unreliable color pixels using a reliability map computed by the binocular just-noticeable-difference model. In addition, a deep colorization network that utilizes structural information is proposed for solving the color bleeding artifact problem. The experimental results demonstrate that the proposed method provides better results than the existing image fusion algorithms for dual cameras.

Deep color reconstruction for a sparse color sensor.

Despite the advances in image sensors, mainstream RGB sensors are still struggling from low quantum efficiency due to the low sensitivity of the Bayer color filter array. To address this issue, a sparse color sensor uses mostly panchromatic white pixels and a smaller percentage of sparse color pixels to provide better low-light photography performance than a conventional Bayer RGB sensor. However, due to the lack of a proper color reconstruction method, sparse color sensors have not been developed thus far. This study proposes a deep-learning-based method for sparse color reconstruction that can realize such a sparse color sensor. The proposed color reconstruction method consists of a novel two-stage deep model followed by an adversarial training technique to reduce visual artifacts in the reconstructed color image. In simulations and experiments, visual results and quantitative comparisons demonstrate that the proposed color reconstruction method can outperform existing methods. In addition, a prototype system was developed using a hybrid color-plus-mono camera system. Experiments using the prototype system reveal the feasibility of a very sparse color sensor in different lighting conditions.

Enhancement of low light level images using color-plus-mono dual camera.

In digital photography, the improvement of imaging quality in low light shooting is one of the users' needs. Unfortunately, conventional smartphone cameras that use a single, small image sensor cannot provide satisfactory quality in low light level images. A color-plus-mono dual camera that consists of two horizontally separate image sensors, which simultaneously captures both a color and mono image pair of the same scene, could be useful for improving the quality of low light level images. However, an incorrect image fusion between the color and mono image pair could also have negative effects, such as the introduction of severe visual artifacts in the fused images. This paper proposes a selective image fusion technique that applies an adaptive guided filter-based denoising and selective detail transfer to only those pixels deemed reliable with respect to binocular image fusion. We employ a dissimilarity measure and binocular just-noticeable-difference (BJND) analysis to identify unreliable pixels that are likely to cause visual artifacts during image fusion via joint color image denoising and detail transfer from the mono image. By constructing an experimental system of color-plus-mono camera, we demonstrate that the BJND-aware denoising and selective detail transfer is helpful in improving the image quality during low light shooting.

Crosstalk reduction in stereoscopic 3D displays: disparity adjustment using crosstalk visibility index for crosstalk cancellation.

Stereoscopic displays provide viewers with a truly fascinating viewing experience. However, current stereoscopic displays suffer from crosstalk that is detrimental to image quality, depth quality, and visual comfort. In order to reduce the perceived crosstalk in stereoscopic displays, this paper proposes a crosstalk reduction method that combines disparity adjustment and crosstalk cancellation. The main idea of the proposed method is to displace the visible crosstalk using the disparity adjustment in a way that less amounts of intensity leakage occur on perceptually important regions in a scene. To this purpose, we estimate a crosstalk visibility index map for the scene that represents pixel-by-pixel importance values associated with the amount of perceived crosstalk and negative-after-effects of the crosstalk cancellation. Based on the crosstalk visibility index, we introduce a new disparity adjustment method that reduces the annoying crosstalk in processed images, which is followed by the crosstalk cancellation. The effectiveness of the proposed method has been successfully evaluated by subjective assessments of image quality and viewing preference. Experimental results demonstrate that the proposed method effectively improves the image quality and overall viewing quality of stereoscopic videos.

Volumetric analysis of cerebellum in short-track speed skating players.

The cerebellum is associated with balance control and coordination, which might be important for gliding on smooth ice at high speeds. A number of case studies have shown that cerebellar damage induces impaired balance and coordination. As a positive model, therefore, we investigated whether plastic changes in the volumes of cerebellar subregions occur in short-track speed skating players who must have extraordinary abilities of balance and coordination, using three-dimensional magnetic resonance imaging volumetry. The manual tracing was performed and the volumes of cerebellar hemisphere and vermian lobules were compared between short-track speed skating players (n=16) and matched healthy controls (n=18). We found larger right cerebellar hemisphere volume and vermian lobules VI-VII (declive, folium, and tuber) in short-track speed skating players in comparison with the matched controls. The finding suggests that the specialized abilities of balance and coordination are associated with structural plasticity of the right hemisphere of cerebellum and vermian VI-VII and these regions play an essential role in balance and coordination.

Quantitative measurement of binocular color fusion limit for non-spectral colors.

Human perception becomes difficult in the event of binocular color fusion when the color difference presented for the left and right eyes exceeds a certain threshold value, known as the binocular color fusion limit. This paper discusses the binocular color fusion limit for non-spectral colors within the color gamut of a conventional LCD 3DTV. We performed experiments to measure the color fusion limit for eight chromaticity points sampled from the CIE 1976 chromaticity diagram. A total of 2480 trials were recorded for a single observer. By analyzing the results, the color fusion limit was quantified by ellipses in the chromaticity diagram. The semi-minor axis of the ellipses ranges from 0.0415 to 0.0923 in terms of the Euclidean distance in the u'v´ chromaticity diagram and the semi-major axis ranges from 0.0640 to 0.1560. These eight ellipses are drawn on the chromaticity diagram.