Searching by parts: Towards fine-grained image retrieval respecting species correlation.

Most of the existing works on fine-grained image categorization and retrieval focus on finding similar images from the same species and often give little importance to inter-species similarities. However, these similarities may carry species correlations such as the same ancestors or similar habits, which are helpful in taxonomy and understanding biological traits. In this paper, we devise a new fine-grained retrieval task that searches for similar instances from different species based on body parts. To this end, we propose a two-step strategy. In the first step, we search for visually similar parts to a query image using a deep convolutional neural network (CNN). To improve the quality of the retrieved candidates, structural cues are introduced into the CNN using a novel part-pooling layer, in which the receptive field of each part is adjusted automatically. In the second step, we re-rank the retrieved candidates to improve the species diversity. We achieve this by formulating a novel ranking function that balances between the similarity of the candidates to the queried parts, while decreasing the similarity to the query species. We provide experiments on the benchmark CUB200 dataset and Columbia Dogs dataset, and demonstrate clear benefits of our schemes.

Breakthroughs and Applications of Organ-on-a-Chip Technology.

Organ-on-a-chip (OOAC) is an emerging technology based on microfluid platforms and in vitro cell culture that has a promising future in the healthcare industry. The numerous advantages of OOAC over conventional systems make it highly popular. The chip is an innovative combination of novel technologies, including lab-on-a-chip, microfluidics, biomaterials, and tissue engineering. This paper begins by analyzing the need for the development of OOAC followed by a brief introduction to the technology. Later sections discuss and review the various types of OOACs and the fabrication materials used. The implementation of artificial intelligence in the system makes it more advanced, thereby helping to provide a more accurate diagnosis as well as convenient data management. We introduce selected OOAC projects, including applications to organ/disease modelling, pharmacology, personalized medicine, and dentistry. Finally, we point out certain challenges that need to be surmounted in order to further develop and upgrade the current systems.

Epigenetic reactivation of transcriptional programs orchestrating fetal lung development in human pulmonary hypertension.

Phenotypic alterations in resident vascular cells contribute to the vascular remodeling process in diseases such as pulmonary (arterial) hypertension [P(A)H]. How the molecular interplay between transcriptional coactivators, transcription factors (TFs), and chromatin state alterations facilitate the maintenance of persistently activated cellular phenotypes that consequently aggravate vascular remodeling processes in PAH remains poorly explored. RNA sequencing (RNA-seq) in pulmonary artery fibroblasts (FBs) from adult human PAH and control lungs revealed 2460 differentially transcribed genes. Chromatin immunoprecipitation sequencing (ChIP-seq) revealed extensive differential distribution of transcriptionally accessible chromatin signatures, with 4152 active enhancers altered in PAH-FBs. Integrative analysis of RNA-seq and ChIP-seq data revealed that the transcriptional signatures for lung morphogenesis were epigenetically derepressed in PAH-FBs, including coexpression of T-box TF 4 (TBX4), TBX5, and SRY-box TF 9 (SOX9), which are involved in the early stages of lung development. These TFs were expressed in mouse fetuses and then repressed postnatally but were maintained in persistent PH of the newborn and reexpressed in adult PAH. Silencing of TBX4, TBX5, SOX9, or E1A-associated protein P300 (EP300) by RNA interference or small-molecule compounds regressed PAH phenotypes and mesenchymal signatures in arterial FBs and smooth muscle cells. Pharmacological inhibition of the P300/CREB-binding protein complex reduced the remodeling of distal pulmonary vessels, improved hemodynamics, and reversed established PAH in three rodent models in vivo, as well as reduced vascular remodeling in precision-cut tissue slices from human PAH lungs ex vivo. Epigenetic reactivation of TFs associated with lung development therefore underlies PAH pathogenesis, offering therapeutic opportunities.

Tensor Representations for Action Recognition.

Human actions in video sequences are characterized by the complex interplay between spatial features and their temporal dynamics. In this paper, we propose novel tensor representations for compactly capturing such higher-order relationships between visual features for the task of action recognition. We propose two tensor-based feature representations, viz. (i) sequence compatibility kernel (SCK) and (ii) dynamics compatibility kernel (DCK). SCK builds on the spatio-temporal correlations between features, whereas DCK explicitly models the action dynamics of a sequence. We also explore generalization of SCK, coined SCK ⊕, that operates on subsequences to capture the local-global interplay of correlations, which can incorporate multi-modal inputs e.g., skeleton 3D body-joints and per-frame classifier scores obtained from deep learning models trained on videos. We introduce linearization of these kernels that lead to compact and fast descriptors. We provide experiments on (i) 3D skeleton action sequences, (ii) fine-grained video sequences, and (iii) standard non-fine-grained videos. As our final representations are tensors that capture higher-order relationships of features, they relate to co-occurrences for robust fine-grained recognition (Lin, 2017), (Koniusz, 2018). We use higher-order tensors and so-called Eigenvalue Power Normalization (EPN) which have been long speculated to perform spectral detection of higher-order occurrences (Koniusz, 2013), (Koniusz, 2017), thus detecting fine-grained relationships of features rather than merely count features in action sequences. We prove that a tensor of order r, built from Z* dimensional features, coupled with EPN indeed detects if at least one higher-order occurrence is 'projected' into one of its [Formula: see text] subspaces of dim. r represented by the tensor, thus forming a Tensor Power Normalization metric endowed with [Formula: see text] such 'detectors'.

Learning Log-Determinant Divergences for Positive Definite Matrices.

Representations in the form of Symmetric Positive Definite (SPD) matrices have been popularized in a variety of visual learning applications due to their demonstrated ability to capture rich second-order statistics of visual data. There exist several similarity measures for comparing SPD matrices with documented benefits. However, selecting an appropriate measure for a given problem remains a challenge and in most cases, is the result of a trial-and-error process. In this paper, we propose to learn similarity measures in a data-driven manner. To this end, we capitalize on the αß-log-det divergence, which is a meta-divergence parametrized by scalars α and ß, subsuming a wide family of popular information divergences on SPD matrices for distinct and discrete values of these parameters. Our key idea is to cast these parameters in a continuum and learn them from data. We systematically extend this idea to learn vector-valued parameters, thereby increasing the expressiveness of the underlying non-linear measure. We conjoin the divergence learning problem with several standard tasks in machine learning, including supervised discriminative dictionary learning and unsupervised SPD matrix clustering. We present Riemannian gradient descent schemes for optimizing our formulations efficiently, and show the usefulness of our method on eight standard computer vision tasks.

Generalized One-Class Learning Using Pairs of Complementary Classifiers.

One-class learning is the classic problem of fitting a model to the data for which annotations are available only for a single class. In this paper, we explore novel objectives for one-class learning, which we collectively refer to as Generalized One-class Discriminative Subspaces (GODS). Our key idea is to learn a pair of complementary classifiers to flexibly bound the one-class data distribution, where the data belongs to the positive half-space of one of the classifiers in the complementary pair and to the negative half-space of the other. To avoid redundancy while allowing non-linearity in the classifier decision surfaces, we propose to design each classifier as an orthonormal frame and seek to learn these frames via jointly optimizing for two conflicting objectives, namely: i) to minimize the distance between the two frames, and ii) to maximize the margin between the frames and the data. The learned orthonormal frames will thus characterize a piecewise linear decision surface that allows for efficient inference, while our objectives seek to bound the data within a minimal volume that maximizes the decision margin, thereby robustly capturing the data distribution. We explore several variants of our formulation under different constraints on the constituent classifiers, including kernelized feature maps. We demonstrate the empirical benefits of our approach via experiments on data from several applications in computer vision, such as anomaly detection in video sequences, human poses, and human activities. We also explore the generality and effectiveness of GODS for non-vision tasks via experiments on several UCI datasets, demonstrating state-of-the-art results.

Targeted mutagenesis in Nicotiana tabacum ADF gene using shockwave-mediated ribonucleoprotein delivery increases osmotic stress tolerance.

DNA-free genome editing involves the direct introduction of ribonucleoprotein (RNP) complexes into cells, but this strategy has rarely been successful in plants. In the present study, we describe a new technique for the introduction of RNPs into plant cells involving the generation of cavitation bubbles using a pulsed laser. The resulting shockwave achieves the efficient transfection of walled cells in tissue explants by creating transient membrane pores. RNP-containing cells were rapidly identified by fluorescence microscopy, followed by regeneration and the screening of mutant plants by high-resolution melt analysis. We used this technique in Nicotiana tabacum to target the endogenous phytoene desaturase (PDS) and actin depolymerizing factor (ADF) genes. Genome-edited plants were produced with an efficiency of 35.2% for PDS and 16.5% for ADF. Further we evaluated the physiological, cellular and molecular effects of ADF mutations in T2 mutant plants under drought and salinity stress. The results suggest that ADF acts as a key regulator of osmotic stress tolerance in plants.

Discriminative Video Representation Learning Using Support Vector Classifiers.

Most popular deep models for action recognition in videos generate independent predictions for short clips, which are then pooled heuristically to assign an action label to the full video segment. As not all frames may characterize the underlying action-indeed, many are common across multiple actions-pooling schemes that impose equal importance on all frames might be unfavorable. In an attempt to tackle this problem, we propose discriminative pooling, based on the notion that among the deep features generated on all short clips, there is at least one that characterizes the action. To identify these useful features, we resort to a negative bag consisting of features that are known to be irrelevant, for example, they are sampled either from datasets that are unrelated to our actions of interest or are CNN features produced via random noise as input. With the features from the video as a positive bag and the irrelevant features as the negative bag, we cast an objective to learn a (nonlinear) hyperplane that separates the unknown useful features from the rest in a multiple instance learning formulation within a support vector machine setup. We use the parameters of this separating hyperplane as a descriptor for the full video segment. Since these parameters are directly related to the support vectors in a max-margin framework, they can be treated as a weighted average pooling of the features from the bags, with zero weights given to non-support vectors. Our pooling scheme is end-to-end trainable within a deep learning framework. We report results from experiments on eight computer vision benchmark datasets spanning a variety of video-related tasks and demonstrate state-of-the-art performance across these tasks.

Image Descriptors for Weakly Annotated Histopathological Breast Cancer Data.

INTRODUCTION: Cancerous Tissue Recognition (CTR) methodologies are continuously integrating advancements at the forefront of machine learning and computer vision, providing a variety of inference schemes for histopathological data. Histopathological data, in most cases, come in the form of high-resolution images, and thus methodologies operating at the patch level are more computationally attractive. Such methodologies capitalize on pixel level annotations (tissue delineations) from expert pathologists, which are then used to derive labels at the patch level. In this work, we envision a digital connected health system that augments the capabilities of the clinicians by providing powerful feature descriptors that may describe malignant regions. MATERIAL AND METHODS: We start with a patch level descriptor, termed Covariance-Kernel Descriptor (CKD), capable of compactly describing tissue architectures associated with carcinomas. To leverage the recognition capability of the CKDs to larger slide regions, we resort to a multiple instance learning framework. In that direction, we derive the Weakly Annotated Image Descriptor (WAID) as the parameters of classifier decision boundaries in a Multiple Instance Learning framework. The WAID is computed on bags of patches corresponding to larger image regions for which binary labels (malignant vs. benign) are provided, thus obviating the necessity for tissue delineations. RESULTS: The CKD was seen to outperform all the considered descriptors, reaching classification accuracy (ACC) of 92.83%. and area under the curve (AUC) of 0.98. The CKD captures higher order correlations between features and was shown to achieve superior performance against a large collection of computer vision features on a private breast cancer dataset. The WAID outperform all other descriptors on the Breast Cancer Histopathological database (BreakHis) where correctly classified malignant (CCM) instances reached 91.27 and 92.00% at the patient and image level, respectively, without resorting to a deep learning scheme achieves state-of-the-art performance. DISCUSSION: Our proposed derivation of the CKD and WAID can help medical experts accomplish their work accurately and faster than the current state-of-the-art.

Visual Permutation Learning.

We present a principled approach to uncover the structure of visual data by solving a deep learning task coined visual permutation learning. The goal of this task is to find the permutation that recovers the structure of data from shuffled versions of it. In the case of natural images, this task boils down to recovering the original image from patches shuffled by an unknown permutation matrix. Permutation matrices are discrete, thereby posing difficulties for gradient-based optimization methods. To this end, we resort to a continuous approximation using doubly-stochastic matrices and formulate a novel bi-level optimization problem on such matrices that learns to recover the permutation. Unfortunately, such a scheme leads to expensive gradient computations. We circumvent this issue by further proposing a computationally cheap scheme for generating doubly stochastic matrices based on Sinkhorn iterations. To implement our approach we propose DeepPermNet, an end-to-end CNN model for this task. The utility of DeepPermNet is demonstrated on three challenging computer vision problems, namely, relative attributes learning, supervised learning-to-rank, and self-supervised representation learning. Our results show state-of-the-art performance on the Public Figures and OSR benchmarks for relative attributes learning, chronological and interestingness image ranking for supervised learning-to-rank, and competitive results in the classification and segmentation tasks of the PASCAL VOC dataset for self-supervised representation learning.

Riemannian Dictionary Learning and Sparse Coding for Positive Definite Matrices.

Data encoded as symmetric positive definite (SPD) matrices frequently arise in many areas of computer vision and machine learning. While these matrices form an open subset of the Euclidean space of symmetric matrices, viewing them through the lens of non-Euclidean Riemannian (Riem) geometry often turns out to be better suited in capturing several desirable data properties. Inspired by the great success of dictionary learning and sparse coding (DLSC) for vector-valued data, our goal in this paper is to represent data in the form of SPD matrices as sparse conic combinations of SPD atoms from a learned dictionary via a Riem geometric approach. To that end, we formulate a novel Riem optimization objective for DLSC, in which the representation loss is characterized via the affine-invariant Riem metric. We also present a computationally simple algorithm for optimizing our model. Experiments on several computer vision data sets demonstrate superior classification and retrieval performance using our approach when compared with SC via alternative non-Riem formulations.

N-cadherin relocalization during cardiac trabeculation.

During cardiac trabeculation, cardiomyocytes delaminate from the outermost (compact) layer to form complex muscular structures known as trabeculae. As these cardiomyocytes delaminate, the remodeling of adhesion junctions must be tightly coordinated so cells can extrude from the compact layer while remaining in tight contact with their neighbors. In this study, we examined the distribution of N-cadherin (Cdh2) during cardiac trabeculation in zebrafish. By analyzing the localization of a Cdh2-EGFP fusion protein expressed under the control of the zebrafish cdh2 promoter, we initially observed Cdh2-EGFP expression along the lateral sides of embryonic cardiomyocytes, in an evenly distributed pattern, and with the occasional appearance of punctae. Within a few hours, Cdh2-EGFP distribution on the lateral sides of cardiomyocytes evolves into a clear punctate pattern as Cdh2-EGFP molecules outside the punctae cluster to increase the size of these aggregates. In addition, Cdh2-EGFP molecules also appear on the basal side of cardiomyocytes that remain in the compact layer. Delaminating cardiomyocytes accumulate Cdh2-EGFP on the surface facing the basal side of compact layer cardiomyocytes, thereby allowing tight adhesion between these layers. Importantly, we find that blood flow/cardiac contractility is required for the transition from an even distribution of Cdh2-EGFP to the formation of punctae. Furthermore, using time-lapse imaging of beating hearts in conjunction with a Cdh2 tandem fluorescent protein timer transgenic line, we observed that Cdh2-EGFP molecules appear to move from the lateral to the basal side of cardiomyocytes along the cell membrane, and that Erb-b2 receptor tyrosine kinase 2 (Erbb2) function is required for this relocalization.

Bayesian Nonparametric Clustering for Positive Definite Matrices.

Symmetric Positive Definite (SPD) matrices emerge as data descriptors in several applications of computer vision such as object tracking, texture recognition, and diffusion tensor imaging. Clustering these data matrices forms an integral part of these applications, for which soft-clustering algorithms (K-Means, expectation maximization, etc.) are generally used. As is well-known, these algorithms need the number of clusters to be specified, which is difficult when the dataset scales. To address this issue, we resort to the classical nonparametric Bayesian framework by modeling the data as a mixture model using the Dirichlet process (DP) prior. Since these matrices do not conform to the Euclidean geometry, rather belongs to a curved Riemannian manifold,existing DP models cannot be directly applied. Thus, in this paper, we propose a novel DP mixture model framework for SPD matrices. Using the log-determinant divergence as the underlying dissimilarity measure to compare these matrices, and further using the connection between this measure and the Wishart distribution, we derive a novel DPM model based on the Wishart-Inverse-Wishart conjugate pair. We apply this model to several applications in computer vision. Our experiments demonstrate that our model is scalable to the dataset size and at the same time achieves superior accuracy compared to several state-of-the-art parametric and nonparametric clustering algorithms.

Erianthus arundinaceus HSP70 (EaHSP70) Acts as a Key Regulator in the Formation of Anisotropic Interdigitation in Sugarcane (Saccharum spp. hybrid) in Response to Drought Stress.

Plant growth during abiotic stress is a long sought-after trait especially in crop plants in the context of global warming and climate change. Previous studies on leaf epidermal cells have revealed that during normal growth and development, adjacent cells interdigitate anisotropically to form cell morphological patterns known as interlocking marginal lobes (IMLs), involving the cell wall-cell membrane-cortical actin continuum. IMLs are growth-associated cell morphological changes in which auxin-binding protein (ABP), Rho GTPases and actin are known to play important roles. In the present study, we investigated the formation of IMLs under drought stress and found that Erianthus arundinaceus, a drought-tolerant wild relative of sugarcane, develops such growth-related cell morphological patterns under drought stress. Using confocal microscopy, we showed an increasing trend in cortical F-actin intensity in drought-tolerant plants with increasing soil moisture stress. In order to check the role of drought tolerance-related genes in IML formation under soil moisture stress, we adopted a structural data mining strategy and identified indirect connections between the ABPs and heat shock proteins (HSPs). Initial experimental evidence for this connection comes from the high transcript levels of HSP70 observed in drought-stressed Erianthus, which developed anisotropic interdigitation, i.e. IMLs. Subsequently, by overexpressing the E. arundinaceus HSP70 gene (EaHSP70) in sugarcane (Saccharum spp. hybrid), we confirm the role of HSP70 in the formation of anisotropic interdigitation under drought stress. Taken together, our results suggest that EaHSP70 acts as a key regulator in the formation of anisotropic interdigitation in drought-tolerant plants (Erianthus and HSP70 transgenic sugarcane) under moisture stress in an actin-mediated pathway. The possible biological significance of the formation of drought-associated interlocking marginal lobes (DaIMLs) in sugarcane plants upon drought stress is discussed.

Isotropic actomyosin dynamics promote organization of the apical cell cortex in epithelial cells.

Although cortical actin plays an important role in cellular mechanics and morphogenesis, there is surprisingly little information on cortex organization at the apical surface of cells. In this paper, we characterize organization and dynamics of microvilli (MV) and a previously unappreciated actomyosin network at the apical surface of Madin-Darby canine kidney cells. In contrast to short and static MV in confluent cells, the apical surfaces of nonconfluent epithelial cells (ECs) form highly dynamic protrusions, which are often oriented along the plane of the membrane. These dynamic MV exhibit complex and spatially correlated reorganization, which is dependent on myosin II activity. Surprisingly, myosin II is organized into an extensive network of filaments spanning the entire apical membrane in nonconfluent ECs. Dynamic MV, myosin filaments, and their associated actin filaments form an interconnected, prestressed network. Interestingly, this network regulates lateral mobility of apical membrane probes such as integrins or epidermal growth factor receptors, suggesting that coordinated actomyosin dynamics contributes to apical cell membrane organization.

Efficient nearest neighbors via robust sparse hashing.

This paper presents a new nearest neighbor (NN) retrieval framework: robust sparse hashing (RSH). Our approach is inspired by the success of dictionary learning for sparse coding. Our key idea is to sparse code the data using a learned dictionary, and then to generate hash codes out of these sparse codes for accurate and fast NN retrieval. But, direct application of sparse coding to NN retrieval poses a technical difficulty: when data are noisy or uncertain (which is the case with most real-world data sets), for a query point, an exact match of the hash code generated from the sparse code seldom happens, thereby breaking the NN retrieval. Borrowing ideas from robust optimization theory, we circumvent this difficulty via our novel robust dictionary learning and sparse coding framework called RSH, by learning dictionaries on the robustified counterparts of the perturbed data points. The algorithm is applied to NN retrieval on both simulated and real-world data. Our results demonstrate that RSH holds significant promise for efficient NN retrieval against the state of the art.

Jensen-Bregman LogDet divergence with application to efficient similarity search for covariance matrices.

Covariance matrices have found success in several computer vision applications, including activity recognition, visual surveillance, and diffusion tensor imaging. This is because they provide an easy platform for fusing multiple features compactly. An important task in all of these applications is to compare two covariance matrices using a (dis)similarity function, for which the common choice is the Riemannian metric on the manifold inhabited by these matrices. As this Riemannian manifold is not flat, the dissimilarities should take into account the curvature of the manifold. As a result, such distance computations tend to slow down, especially when the matrix dimensions are large or gradients are required. Further, suitability of the metric to enable efficient nearest neighbor retrieval is an important requirement in the contemporary times of big data analytics. To alleviate these difficulties, this paper proposes a novel dissimilarity measure for covariances, the Jensen-Bregman LogDet Divergence (JBLD). This divergence enjoys several desirable theoretical properties and at the same time is computationally less demanding (compared to standard measures). Utilizing the fact that the square root of JBLD is a metric, we address the problem of efficient nearest neighbor retrieval on large covariance datasets via a metric tree data structure. To this end, we propose a K-Means clustering algorithm on JBLD. We demonstrate the superior performance of JBLD on covariance datasets from several computer vision applications.

Pulsed-laser-induced damage in rat corneas: time-resolved imaging of physical effects and acute biological response.

Laser-induced damage is studied in the rat corneal epithelium and stroma using a combination of time-resolved imaging and biological assays. Cavitation bubble interactions with cells are visualized at a higher spatial resolution than previously reported. The shock wave is observed to propagate through the epithelium without cell displacement or deformation. Cavitation bubble expansion is damped in tissue with a reduction in maximum size in the range of 54 to 59%, as compared to 2-D and 3-D cultures. Bubble expansion on nanosecond timescales results in rupture of the epithelial sheet and severe compression of cell layers beyond the bubble rim. In the stroma, the dense collagen lamellae strongly damped bubble expansion, thus resulting in reduced damage. The acute biological response of this tissue to laser pulses is characterized by confocal fluorescence microscopy. A viability assay of the epithelium reveals that only cells around the immediate site of laser focus are killed, while cells seen to undergo large deformations remain alive. Actin morphology in cells facing this mechanical stress is unchanged. Collagen microstructure in the stroma as revealed by second-harmonic imaging around the ablation site shows minimal disruption. These cellular responses are also compared to in vitro 2-D and 3-D cell cultures.