Explicit Margin Equilibrium for Few-Shot Object Detection.

Under low data regimes, few-shot object detection (FSOD) transfers related knowledge from base classes with sufficient annotations to novel classes with limited samples in a two-step paradigm, including base training and balanced fine-tuning. In base training, the learned embedding space needs to be dispersed with large class margins to facilitate novel class accommodation and avoid feature aliasing while in balanced fine-tuning properly concentrating with small margins to represent novel classes precisely. Although obsession with the discrimination and representation dilemma has stimulated substantial progress, explorations for the equilibrium of class margins within the embedding space are still in full swing. In this study, we propose a class margin optimization scheme, termed explicit margin equilibrium (EME), by explicitly leveraging the quantified relationship between base and novel classes. EME first maximizes base-class margins to reserve adequate space to prepare for novel class adaptation. During fine-tuning, it quantifies the interclass semantic relationships by calculating the equilibrium coefficients based on the assumption that novel instances can be represented by linear combinations of base-class prototypes. EME finally reweights margin loss using equilibrium coefficients to adapt base knowledge for novel instance learning with the help of instance disturbance (ID) augmentation. As a plug-and-play module, EME can also be applied to few-shot classification. Consistent performance gains upon various baseline methods and benchmarks validate the generality and efficacy of EME. The code is available at github.com/Bohao-Lee/EME.

Prediction of coronary artery disease based on facial temperature information captured by non-contact infrared thermography.

BACKGROUND: Current approaches for initial coronary artery disease (CAD) assessment rely on pretest probability (PTP) based on risk factors and presentations, with limited performance. Infrared thermography (IRT), a non-contact technology that detects surface temperature, has shown potential in assessing atherosclerosis-related conditions, particularly when measured from body regions such as faces. We aim to assess the feasibility of using facial IRT temperature information with machine learning for the prediction of CAD. METHODS: Individuals referred for invasive coronary angiography or coronary CT angiography (CCTA) were enrolled. Facial IRT images captured before confirmatory CAD examinations were used to develop and validate a deep-learning IRT image model for detecting CAD. We compared the performance of the IRT image model with the guideline-recommended PTP model on the area under the curve (AUC). In addition, interpretable IRT tabular features were extracted from IRT images to further validate the predictive value of IRT information. RESULTS: A total of 460 eligible participants (mean (SD) age, 58.4 (10.4) years; 126 (27.4%) female) were included. The IRT image model demonstrated outstanding performance (AUC 0.804, 95% CI 0.785 to 0.823) compared with the PTP models (AUC 0.713, 95% CI 0.691 to 0.734). A consistent level of superior performance (AUC 0.796, 95% CI 0.782 to 0.811), achieved with comprehensive interpretable IRT features, further validated the predictive value of IRT information. Notably, even with only traditional temperature features, a satisfactory performance (AUC 0.786, 95% CI 0.769 to 0.803) was still upheld. CONCLUSION: In this prospective study, we demonstrated the feasibility of using non-contact facial IRT information for CAD prediction.

Cell-free biosynthesis and engineering of ribosomally synthesized lanthipeptides.

Ribosomally synthesized and post-translationally modified peptides (RiPPs) are a major class of natural products with diverse chemical structures and potent biological activities. A vast majority of RiPP gene clusters remain unexplored in microbial genomes, which is partially due to the lack of rapid and efficient heterologous expression systems for RiPP characterization and biosynthesis. Here, we report a unified biocatalysis (UniBioCat) system based on cell-free gene expression for rapid biosynthesis and engineering of RiPPs. We demonstrate UniBioCat by reconstituting a full biosynthetic pathway for de novo biosynthesis of salivaricin B, a lanthipeptide RiPP. Next, we delete several protease/peptidase genes from the source strain to enhance the performance of UniBioCat, which then can synthesize and screen salivaricin B variants with enhanced antimicrobial activity. Finally, we show that UniBioCat is generalizable by synthesizing and evaluating the bioactivity of ten uncharacterized lanthipeptides. We expect UniBioCat to accelerate the discovery, characterization, and synthesis of RiPPs.

Deep Lossy Plus Residual Coding for Lossless and Near-Lossless Image Compression.

Lossless and near-lossless image compression is of paramount importance to professional users in many technical fields, such as medicine, remote sensing, precision engineering and scientific research. But despite rapidly growing research interests in learning-based image compression, no published method offers both lossless and near-lossless modes. In this paper, we propose a unified and powerful deep lossy plus residual (DLPR) coding framework for both lossless and near-lossless image compression. In the lossless mode, the DLPR coding system first performs lossy compression and then lossless coding of residuals. We solve the joint lossy and residual compression problem in the approach of VAEs, and add autoregressive context modeling of the residuals to enhance lossless compression performance. In the near-lossless mode, we quantize the original residuals to satisfy a given ℓ∞ error bound, and propose a scalable near-lossless compression scheme that works for variable ℓ∞ bounds instead of training multiple networks. To expedite the DLPR coding, we increase the degree of algorithm parallelization by a novel design of coding context, and accelerate the entropy coding with adaptive residual interval. Experimental results demonstrate that the DLPR coding system achieves both the state-of-the-art lossless and near-lossless image compression performance with competitive coding speed.

Multispectral Depolarization Based Living-Skin Detection: A New Measurement Principle.

Camera-based photoplethysmographic imaging enabled the segmentation of living-skin tissues in a video, but it has inherent limitations to be used in real-life applications such as video health monitoring and face anti-spoofing. Inspired by the use of polarization for improving vital signs monitoring (i.e. specular reflection removal), we observed that skin tissues have an attractive property of wavelength-dependent depolarization due to its multi-layer structure containing different absorbing chromophores, i.e. polarized light photons with longer wavelengths (R) have deeper skin penetrability and thus experience thorougher depolarization than those with shorter wavelengths (G and B). Thus we proposed a novel dual-polarization setup and an elegant algorithm (named "MSD") that exploits the nature of multispectral depolarization of skin tissues to detect living-skin pixels, which only requires two images sampled at the parallel and cross polarizations to estimate the characteristic chromaticity changes (R/G) caused by tissue depolarization. Our proposal was verified in both the laboratory and hospital settings (ICU and NICU) focused on anti-spoofing and patient skin segmentation. The clinical experiments in ICU also indicate the potential of MSD for skin perfusion analysis, which may lead to a new diagnostic imaging approach in the future.

CarsiDock: a deep learning paradigm for accurate protein-ligand docking and screening based on large-scale pre-training.

The expertise accumulated in deep neural network-based structure prediction has been widely transferred to the field of protein-ligand binding pose prediction, thus leading to the emergence of a variety of deep learning-guided docking models for predicting protein-ligand binding poses without relying on heavy sampling. However, their prediction accuracy and applicability are still far from satisfactory, partially due to the lack of protein-ligand binding complex data. To this end, we create a large-scale complex dataset containing ∼9 M protein-ligand docking complexes for pre-training, and propose CarsiDock, the first deep learning-guided docking approach that leverages pre-training of millions of predicted protein-ligand complexes. CarsiDock contains two main stages, i.e., a deep learning model for the prediction of protein-ligand atomic distance matrices, and a translation, rotation and torsion-guided geometry optimization procedure to reconstruct the matrices into a credible binding pose. The pre-training and multiple innovative architectural designs facilitate the dramatically improved docking accuracy of our approach over the baselines in terms of multiple docking scenarios, thereby contributing to its outstanding early recognition performance in several retrospective virtual screening campaigns. Further explorations demonstrate that CarsiDock can not only guarantee the topological reliability of the binding poses but also successfully reproduce the crucial interactions in crystalized structures, highlighting its superior applicability.

Expanding the toolbox of probiotic Escherichia coli Nissle 1917 for synthetic biology.

Escherichia coli Nissle 1917 (EcN) is a probiotic microbe that has the potential to be developed as a promising chassis for synthetic biology applications. However, the molecular tools and techniques for utilizing EcN remain to be further explored. To address this opportunity, the EcN-based toolbox was systematically expanded, enabling EcN as a powerful platform for more applications. First, two EcN cryptic plasmids and other compatible plasmids were genetically engineered to enrich the manipulable plasmid toolbox for multiple gene coexpression. Next, two EcN-based technologies were developed, including the conjugation strategy for DNA transfer, and quantification of protein expression capability. Finally, the EcN-based applications were further expanded by developing EcN native integrase-mediated genetic engineering and establishing an in vitro cell-free protein synthesis (CFPS) system. Overall, this study expanded the toolbox for manipulating and making full use of EcN as a commonly used probiotic chassis, providing several simplified, dependable, and predictable strategies for researchers working in synthetic biology fields.

Proposal Distribution Calibration for Few-Shot Object Detection.

Adapting object detectors learned with sufficient supervision to novel classes under low data regimes is charming yet challenging. In few-shot object detection (FSOD), the two-step training paradigm is widely adopted to mitigate the severe sample imbalance, i.e., holistic pre-training on base classes, then partial fine-tuning in a balanced setting with all classes. Since unlabeled instances are suppressed as backgrounds in the base training phase, the learned region proposal network (RPN) is prone to produce biased proposals for novel instances, resulting in dramatic performance degradation. Unfortunately, the extreme data scarcity aggravates the proposal distribution bias, hindering the region of interest (RoI) head from evolving toward novel classes. In this brief, we introduce a simple yet effective proposal distribution calibration (PDC) approach to neatly enhance the localization and classification abilities of the RoI head by recycling its localization ability endowed in base training and enriching high-quality positive samples for semantic fine-tuning. Specifically, we sample proposals based on the base proposal statistics to calibrate the distribution bias and impose additional localization and classification losses upon the sampled proposals for fast expanding the base detector to novel classes. Experiments on the commonly used Pascal VOC and MS COCO datasets with explicit state-of-the-art performances justify the efficacy of our PDC for FSOD. Code is available at github.com/Bohao-Lee/PDC.

MagConv: Mask-Guided Convolution for Image Inpainting.

Standard convolution applied to image inpainting would lead to color discrepancy and blurriness for treating valid and invalid/hole regions without difference, which was partially amended by partial convolution (PConv). In PConv, a binary/hard mask was maintained as an indicator of valid and invalid pixels, where valid pixels and invalid pixels were treated differently. However, it can not describe validity degree of an impaired pixel. In addition, mask and image paths were separated, without sharing convolution kernel and exchanging information mutually, reducing data utilization efficiency. In this paper, a mask-guided convolution (MagConv) is proposed for image inpainting. In MagConv, mask and image paths share a convolution kernel to interact with each other and form a joint optimization scheme. In addition, a learnable piecewise activation function is raised to replace the reciprocal function of PConv, providing more flexible and adaptable compensation to convolution contaminated by invalid pixels. It also results in a soft mask of floating-point coefficients from 0 to 1 capable of indicating the validity degree of each pixel. Last but not least, MagConv splits the convolution kernel into positive and negative weights so that they can evaluate the validity of each pixel faithfully. Qualitative and quantitative experiments on the CelebA, Paris StreetView and Places2 datasets demonstrate that our method achieves favorable visual quality against state-of-the-art approaches.

Self-Supervised Arbitrary-Scale Implicit Point Clouds Upsampling.

Point clouds upsampling (PCU), which aims to generate dense and uniform point clouds from the captured sparse input of 3D sensor such as LiDAR, is a practical yet challenging task. It has potential applications in many real-world scenarios, such as autonomous driving, robotics, AR/VR, etc. Deep neural network based methods achieve remarkable success in PCU. However, most existing deep PCU methods either take the end-to-end supervised training, where large amounts of pairs of sparse input and dense ground-truth are required to serve as the supervision; or treat up-scaling of different factors as independent tasks, where multiple networks are required for different scaling factors, leading to significantly increased model complexity and training time. In this article, we propose a novel method that achieves self-supervised and magnification-flexible PCU simultaneously. No longer explicitly learning the mapping between sparse and dense point clouds, we formulate PCU as the task of seeking nearest projection points on the implicit surface for seed points. We then define two implicit neural functions to estimate projection direction and distance respectively, which can be trained by the pretext learning tasks. Moreover, the projection rectification strategy is tailored to remove outliers so as to keep the shape of object clear and sharp. Experimental results demonstrate that our self-supervised learning based scheme achieves competitive or even better performance than state-of-the-art supervised methods.

Multiple Instance Differentiation Learning for Active Object Detection.

Despite the substantial progress of active learning for image recognition, there lacks a systematic investigation of instance-level active learning for object detection. In this paper, we propose to unify instance uncertainty calculation with image uncertainty estimation for informative image selection, creating a multiple instance differentiation learning (MIDL) method for instance-level active learning. MIDL consists of a classifier prediction differentiation module and a multiple instance differentiation module. The former leverages two adversarial instance classifiers trained on the labeled and unlabeled sets to estimate instance uncertainty of the unlabeled set. The latter treats unlabeled images as instance bags and re-estimates image-instance uncertainty using the instance classification model in a multiple instance learning fashion. Through weighting the instance uncertainty using instance class probability and instance objectness probability under the total probability formula, MIDL unifies the image uncertainty with instance uncertainty in the Bayesian theory framework. Extensive experiments validate that MIDL sets a solid baseline for instance-level active learning. On commonly used object detection datasets, it outperforms other state-of-the-art methods by significant margins, particularly when the labeled sets are small.

Efficient framework of solving time-gated reflection matrix for imaging through turbid medium.

Imaging through turbid medium is a long pursuit in many research fields, such as biomedicine, astronomy and automatic vehicle, in which the reflection matrix-based method is a promising solution. However, the epi-detection geometry suffers from round-trip distortion and it is challenging to isolate the input and output aberrations in non-ideal cases due to system imperfections and measurement noises. Here, we present an efficient framework based on single scattering accumulation together with phase unwrapping that can accurately separate input and output aberrations from the noise-affected reflection matrix. We propose to only correct the output aberration while suppressing the input aberration by incoherent averaging. The proposed method is faster in convergence and more robust against noise, avoiding precise and tedious system adjustments. In both simulations and experiments, we demonstrate the diffraction-limited resolution capability under optical thickness beyond 10 scattering mean free paths, showing the potential of applications in neuroscience and dermatology.

Perspectively Equivariant Keypoint Learning for Omnidirectional Images.

Robust keypoint detection on omnidirectional images against large perspective variations, is a key problem in many computer vision tasks. In this paper, we propose a perspectively equivariant keypoint learning framework named OmniKL for addressing this problem. Specifically, the framework is composed of a perspective module and a spherical module, each one including a keypoint detector specific to the type of the input image and a shared descriptor providing uniform description for omnidirectional and perspective images. In these detectors, we propose a differentiable candidate position sorting operation for localizing keypoints, which directly sorts the scores of the candidate positions in a differentiable manner and returns the globally top-K keypoints on the image. This approach does not break the differentiability of the two modules, thus they are end-to-end trainable. Moreover, we design a novel training strategy combining the self-supervised and co-supervised methods to train the framework without any labeled data. Extensive experiments on synthetic and real-world 360° image datasets demonstrate the effectiveness of OmniKL in detecting perspectively equivariant keypoints on omnidirectional images. Our source code are available online at https://github.com/vandeppce/sphkpt.

Deep Attentional Guided Image Filtering.

Guided filter is a fundamental tool in computer vision and computer graphics, which aims to transfer structure information from the guide image to the target image. Most existing methods construct filter kernels from the guidance itself without considering the mutual dependency between the guidance and the target. However, since there typically exist significantly different edges in two images, simply transferring all structural information from the guide to the target would result in various artifacts. To cope with this problem, we propose an effective framework named deep attentional guided image filtering, the filtering process of which can fully integrate the complementary information contained in both images. Specifically, we propose an attentional kernel learning module to generate dual sets of filter kernels from the guidance and the target and then adaptively combine them by modeling the pixelwise dependency between the two images. Meanwhile, we propose a multiscale guided image filtering module to progressively generate the filtering result with the constructed kernels in a coarse-to-fine manner. Correspondingly, a multiscale fusion strategy is introduced to reuse the intermediate results in the coarse-to-fine process. Extensive experiments show that the proposed framework compares favorably with the state-of-the-art methods in a wide range of guided image filtering applications, such as guided super-resolution (SR), cross-modality restoration, and semantic segmentation. Moreover, our scheme achieved the first place in the real depth map SR challenge held in ACM ICMR 2021. The codes can be found at https://github.com/zhwzhong/DAGF.

Asymmetric Loss Functions for Noise-Tolerant Learning: Theory and Applications.

Supervised deep learning has achieved tremendous success in many computer vision tasks, which however is prone to overfit noisy labels. To mitigate the undesirable influence of noisy labels, robust loss functions offer a feasible approach to achieve noise-tolerant learning. In this work, we systematically study the problem of noise-tolerant learning with respect to both classification and regression. Specifically, we propose a new class of loss function, namely asymmetric loss functions (ALFs), which are tailored to satisfy the Bayes-optimal condition and thus are robust to noisy labels. For classification, we investigate general theoretical properties of ALFs on categorical noisy labels, and introduce the asymmetry ratio to measure the asymmetry of a loss function. We extend several commonly-used loss functions, and establish the necessary and sufficient conditions to make them asymmetric and thus noise-tolerant. For regression, we extend the concept of noise-tolerant learning for image restoration with continuous noisy labels. We theoretically prove that lp loss ( ) is noise-tolerant for targets with the additive white Gaussian noise. For targets with general noise, we introduce two losses as surrogates of l0 loss that seeks the mode when clean pixels keep dominant. Experimental results demonstrate that ALFs can achieve better or comparative performance compared with the state-of-the-arts. The source code of our method is available at: https://github.com/hitcszx/ALFs.

Engineering Escherichia coli to Utilize Erythritol as Sole Carbon Source.

Erythritol, one of the natural sugar alcohols, is widely used as a sugar substitute sweetener in food industries. Humans themselves are not able to catabolize erythritol and their gut microbes lack related catabolic pathways either to metabolize erythritol. Here, Escherichia coli (E. coli) is engineered to utilize erythritol as sole carbon source aiming for defined applications. First, the erythritol metabolic gene cluster is isolated and the erythritol-binding transcriptional repressor and its DNA-binding site are experimentally characterized. Transcriptome analysis suggests that carbohydrate metabolism-related genes in the engineered E. coli are overall upregulated. In particular, the enzymes of transaldolase (talA and talB) and transketolase (tktA and tktB) are notably overexpressed (e.g., the expression of tktB is improved by nearly sixfold). By overexpression of the four genes, cell growth can be increased as high as three times compared to the cell cultivation without overexpression. Finally, engineered E. coli strains can be used as a living detector to distinguish erythritol-containing soda soft drinks and can grow in the simulated intestinal fluid supplemented with erythritol. This work is expected to inspire the engineering of more hosts to respond and utilize erythritol for broad applications in metabolic engineering, synthetic biology, and biomedical engineering.

Conditional Molecular Generation Net Enables Automated Structure Elucidation Based on 13C NMR Spectra and Prior Knowledge.

Structure elucidation of unknown compounds based on nuclear magnetic resonance (NMR) remains a challenging problem in both synthetic organic and natural product chemistry. Library matching has been an efficient method to assist structure elucidation. However, it is limited by the coverage of libraries. In addition, prior knowledge such as molecular fragments is neglected. To solve the problem, we propose a conditional molecular generation net (CMGNet) to allow input of multiple sources of information. CMGNet not only uses 13C NMR spectrum data as input but molecular formulas and fragments of molecules are also employed as input conditions. Our model applies large-scale pretraining for molecular understanding and fine-tuning on two NMR spectral data sets of different granularity levels to accommodate structure elucidation tasks. CMGNet generates structures based on 13C NMR data, molecular formula, and fragment information, with a recovery rate of 94.17% in the top 10 recommendations. In addition, the generative model performed well in the generation of various classes of compounds and in the structural revision task. CMGNet has a deep understanding of molecular connectivities from 13C NMR, molecular formula, and fragments, paving the way for a new paradigm of deep learning-assisted inverse problem-solving.

Permeability-Engineered Compartmentalization Enables In Vitro Reconstitution of Sustained Synthetic Biology Systems.

In nature, biological compartments such as cells rely on dynamically controlled permeability for matter exchange and complex cellular activities. Likewise, the ability to engineer compartment permeability is crucial for in vitro systems to gain sustainability, robustness, and complexity. However, rendering in vitro compartments such a capability is challenging. Here, a facile strategy is presented to build permeability-configurable compartments, and marked advantages of such compartmentalization are shown in reconstituting sustained synthetic biology systems in vitro. Through microfluidics, the strategy produces micrometer-sized layered microgels whose shell layer serves as a sieving structure for biomolecules and particles. In this configuration, the transport of DNAs, proteins, and bacteriophages across the compartments can be controlled an guided by a physical model. Through permeability engineering, a compartmentalized cell-free protein synthesis system sustains multicycle protein production; ≈100 000 compartments are repeatedly used in a five-cycle synthesis, featuring a yield of 2.2 mg mL-1 . Further, the engineered bacteria-enclosing compartments possess near-perfect phage resistance and enhanced environmental fitness. In a complex river silt environment, compartmentalized whole-cell biosensors show maintained activity throughout the 32 h pollutant monitoring. It is anticipated that permeability-engineered compartmentalization should pave the way for practical synthetic biology applications such as green bioproduction, environmental sensing, and bacteria-based therapeutics.

Full-duplex visible light communication system using a single channel.

Multiple quantum well (MQW) III-nitride diodes can emit light and detect light at the same time. In particular, given the overlapping region between the emission spectrum and the detection spectrum, the III-nitride diode can absorb photons of shorter wavelengths generated from another III-nitride diode with the same MQW structure. In this study, a wireless visible light communication system was established using two pairs of identical III-nitride diodes with different wavelengths. In this system, two green light diode chips were used to transmit and receive green light signals on both sides. We have integrated two blue light chips with optical filtering in the middle of the optical link to carry out blue light communication, with one end transmitting and one end receiving. Simultaneously, green light was allowed to pass through two blue light chips for optical communication. Combined with a distributed Bragg reflection (DBR) coating, we proposed using four chips in one optical path to carry out optical communication between chips with the same wavelength and used the coating principle to gate the optical wavelength to filter the clutter of green light chips on both sides to make the channel purer and the symbols easier to demodulate. Based on this multifunctional equipment, advanced single-optical path, III-nitride, full-duplex optical communication links can be developed for the deployment of the Internet of Things.

Optimizing the dynamic treatment regime of in-hospital warfarin anticoagulation in patients after surgical valve replacement using reinforcement learning.

OBJECTIVE: Warfarin anticoagulation management requires sequential decision-making to adjust dosages based on patients' evolving states continuously. We aimed to leverage reinforcement learning (RL) to optimize the dynamic in-hospital warfarin dosing in patients after surgical valve replacement (SVR). MATERIALS AND METHODS: 10 408 SVR cases with warfarin dosage-response data were retrospectively collected to develop and test an RL algorithm that can continuously recommend daily warfarin doses based on patients' evolving multidimensional states. The RL algorithm was compared with clinicians' actual practice and other machine learning and clinical decision rule-based algorithms. The primary outcome was the ratio of patients without in-hospital INRs >3.0 and the INR at discharge within the target range (1.8-2.5) (excellent responders). The secondary outcomes were the safety responder ratio (no INRs >3.0) and the target responder ratio (the discharge INR within 1.8-2.5). RESULTS: In the test set (n = 1260), the excellent responder ratio under clinicians' guidance was significantly lower than the RL algorithm: 41.6% versus 80.8% (relative risk [RR], 0.51; 95% confidence interval [CI], 0.48-0.55), also the safety responder ratio: 83.1% versus 99.5% (RR, 0.83; 95% CI, 0.81-0.86), and the target responder ratio: 49.7% versus 81.1% (RR, 0.61; 95% CI, 0.58-0.65). The RL algorithms performed significantly better than all the other algorithms. Compared with clinicians' actual practice, the RL-optimized INR trajectory reached and maintained within the target range significantly faster and longer. DISCUSSION: RL could offer interactive, practical clinical decision support for sequential decision-making tasks and is potentially adaptable for varied clinical scenarios. Prospective validation is needed. CONCLUSION: An RL algorithm significantly optimized the post-operation warfarin anticoagulation quality compared with clinicians' actual practice, suggesting its potential for challenging sequential decision-making tasks.