OW-Adapter: Human-Assisted Open-World Object Detection with a Few Examples.

Open-world object detection (OWOD) is an emerging computer vision problem that involves not only the identification of predefined object classes, like what general object detectors do, but also detects new unknown objects simultaneously. Recently, several end-to-end deep learning models have been proposed to address the OWOD problem. However, these approaches face several challenges: a) significant changes in both network architecture and training procedure are required; b) they are trained from scratch, which can not leverage existing pre-trained general detectors; c) costly annotations for all unknown classes are needed. To overcome these challenges, we present a visual analytic framework called OW-Adapter. It acts as an adaptor to enable pre-trained general object detectors to handle the OWOD problem. Specifically, OW-Adapter is designed to identify, summarize, and annotate unknown examples with minimal human effort. Moreover, we introduce a lightweight classifier to learn newly annotated unknown classes and plug the classifier into pre-trained general detectors to detect unknown objects. We demonstrate the effectiveness of our framework through two case studies of different domains, including common object recognition and autonomous driving. The studies show that a simple yet powerful adaptor can extend the capability of pre-trained general detectors to detect unknown objects and improve the performance on known classes simultaneously.

Improved Spread Spectrum Aloha Protocol and Beam-Hopping Approach for Return Channel in Satellite Internet of Things.

This paper examines potential performances of the Spread Spectrum-based random access technique and proposes an Improved Spread Spectrum Aloha (ISSA) protocol for the return channel in satellite Internet of Things (IoT) based on the beam-hopping technique. The key design driver and detailed solution of ISSA protocol are presented in this work and it is shown that the proposed protocol achieves high throughput and low collision probability. To match user/traffic distribution, delay requirement and channel condition with beam allocation better, a low-complexity heuristic beam scheduling algorithm and a more effective Maximum-Weighted Clique (MWC) algorithm have been proposed. The heuristic algorithm considers the user/traffic distribution, inter-beam interference, and fairness primarily. However, the MWC algorithm gives considerations not only on above factors, but also on delay requirement and channel condition (path loss and rain attenuation) to maximize system capacity. The beam angle and interference avoidance threshold are proposed to measure the inter-beam interference, and the link propagation loss and rain attenuation are considered meanwhile in the channel condition. In the MWC algorithm, we construct an auxiliary graph to find the maximum-weighted clique and derive the weighting approach to be applied in different application scenarios. The performance evaluation of our ISSA protocol compared with the SSA protocol is presented, which achieves a gain of 16.7%. The simulation of the ISSA protocol combined with round robin, heuristic, and MWC beam scheduling for the return link in beam-hopping satellite IoTs is also provided. The results indicate that the throughput in nonuniform user distribution is much lower than in the uniform case without the beam scheduling algorithm. Through the application of the scheduling algorithm, the throughput performance can approach the uniform distribution. Finally, the degree of user satisfaction with different scheduling approaches is presented, which validates the effectiveness of heuristic and MWC algorithms.

SliceTeller: A Data Slice-Driven Approach for Machine Learning Model Validation.

Real-world machine learning applications need to be thoroughly evaluated to meet critical product requirements for model release, to ensure fairness for different groups or individuals, and to achieve a consistent performance in various scenarios. For example, in autonomous driving, an object classification model should achieve high detection rates under different conditions of weather, distance, etc. Similarly, in the financial setting, credit-scoring models must not discriminate against minority groups. These conditions or groups are called as "Data Slices". In product MLOps cycles, product developers must identify such critical data slices and adapt models to mitigate data slice problems. Discovering where models fail, understanding why they fail, and mitigating these problems, are therefore essential tasks in the MLOps life-cycle. In this paper, we present SliceTeller, a novel tool that allows users to debug, compare and improve machine learning models driven by critical data slices. SliceTeller automatically discovers problematic slices in the data, helps the user understand why models fail. More importantly, we present an efficient algorithm, SliceBoosting, to estimate trade-offs when prioritizing the optimization over certain slices. Furthermore, our system empowers model developers to compare and analyze different model versions during model iterations, allowing them to choose the model version best suitable for their applications. We evaluate our system with three use cases, including two real-world use cases of product development, to demonstrate the power of SliceTeller in the debugging and improvement of product-quality ML models.

Visual Concept Programming: A Visual Analytics Approach to Injecting Human Intelligence at Scale.

Data-centric AI has emerged as a new research area to systematically engineer the data to land AI models for real-world applications. As a core method for data-centric AI, data programming helps experts inject domain knowledge into data and label data at scale using carefully designed labeling functions (e.g., heuristic rules, logistics). Though data programming has shown great success in the NLP domain, it is challenging to program image data because of a) the challenge to describe images using visual vocabulary without human annotations and b) lacking efficient tools for data programming of images. We present Visual Concept Programming, a first-of-its-kind visual analytics approach of using visual concepts to program image data at scale while requiring a few human efforts. Our approach is built upon three unique components. It first uses a self-supervised learning approach to learn visual representation at the pixel level and extract a dictionary of visual concepts from images without using any human annotations. The visual concepts serve as building blocks of labeling functions for experts to inject their domain knowledge. We then design interactive visualizations to explore and understand visual concepts and compose labeling functions with concepts without writing code. Finally, with the composed labeling functions, users can label the image data at scale and use the labeled data to refine the pixel-wise visual representation and concept quality. We evaluate the learned pixel-wise visual representation for the downstream task of semantic segmentation to show the effectiveness and usefulness of our approach. In addition, we demonstrate how our approach tackles real-world problems of image retrieval for autonomous driving.

Where Can We Help? A Visual Analytics Approach to Diagnosing and Improving Semantic Segmentation of Movable Objects.

Semantic segmentation is a critical component in autonomous driving and has to be thoroughly evaluated due to safety concerns. Deep neural network (DNN) based semantic segmentation models are widely used in autonomous driving. However, it is challenging to evaluate DNN-based models due to their black-box-like nature, and it is even more difficult to assess model performance for crucial objects, such as lost cargos and pedestrians, in autonomous driving applications. In this work, we propose VASS, a Visual Analytics approach to diagnosing and improving the accuracy and robustness of Semantic Segmentation models, especially for critical objects moving in various driving scenes. The key component of our approach is a context-aware spatial representation learning that extracts important spatial information of objects, such as position, size, and aspect ratio, with respect to given scene contexts. Based on this spatial representation, we first use it to create visual summarization to analyze models' performance. We then use it to guide the generation of adversarial examples to evaluate models' spatial robustness and obtain actionable insights. We demonstrate the effectiveness of VASS via two case studies of lost cargo detection and pedestrian detection in autonomous driving. For both cases, we show quantitative evaluation on the improvement of models' performance with actionable insights obtained from VASS.

VATLD: A Visual Analytics System to Assess, Understand and Improve Traffic Light Detection.

Traffic light detection is crucial for environment perception and decision-making in autonomous driving. State-of-the-art detectors are built upon deep Convolutional Neural Networks (CNNs) and have exhibited promising performance. However, one looming concern with CNN based detectors is how to thoroughly evaluate the performance of accuracy and robustness before they can be deployed to autonomous vehicles. In this work, we propose a visual analytics system, VATLD, equipped with a disentangled representation learning and semantic adversarial learning, to assess, understand, and improve the accuracy and robustness of traffic light detectors in autonomous driving applications. The disentangled representation learning extracts data semantics to augment human cognition with human-friendly visual summarization, and the semantic adversarial learning efficiently exposes interpretable robustness risks and enables minimal human interaction for actionable insights. We also demonstrate the effectiveness of various performance improvement strategies derived from actionable insights with our visual analytics system, VATLD, and illustrate some practical implications for safety-critical applications in autonomous driving.

DeepVID: Deep Visual Interpretation and Diagnosis for Image Classifiers via Knowledge Distillation.

Deep Neural Networks (DNNs) have been extensively used in multiple disciplines due to their superior performance. However, in most cases, DNNs are considered as black-boxes and the interpretation of their internal working mechanism is usually challenging. Given that model trust is often built on the understanding of how a model works, the interpretation of DNNs becomes more important, especially in safety-critical applications (e.g., medical diagnosis, autonomous driving). In this paper, we propose DeepVID, a Deep learning approach to Visually Interpret and Diagnose DNN models, especially image classifiers. In detail, we train a small locally-faithful model to mimic the behavior of an original cumbersome DNN around a particular data instance of interest, and the local model is sufficiently simple such that it can be visually interpreted (e.g., a linear model). Knowledge distillation is used to transfer the knowledge from the cumbersome DNN to the small model, and a deep generative model (i.e., variational auto-encoder) is used to generate neighbors around the instance of interest. Those neighbors, which come with small feature variances and semantic meanings, can effectively probe the DNN's behaviors around the interested instance and help the small model to learn those behaviors. Through comprehensive evaluations, as well as case studies conducted together with deep learning experts, we validate the effectiveness of DeepVID.

DQNViz: A Visual Analytics Approach to Understand Deep Q-Networks.

Deep Q-Network (DQN), as one type of deep reinforcement learning model, targets to train an intelligent agent that acquires optimal actions while interacting with an environment. The model is well known for its ability to surpass professional human players across many Atari 2600 games. Despite the superhuman performance, in-depth understanding of the model and interpreting the sophisticated behaviors of the DQN agent remain to be challenging tasks, due to the long-time model training process and the large number of experiences dynamically generated by the agent. In this work, we propose DQNViz, a visual analytics system to expose details of the blind training process in four levels, and enable users to dive into the large experience space of the agent for comprehensive analysis. As an initial attempt in visualizing DQN models, our work focuses more on Atari games with a simple action space, most notably the Breakout game. From our visual analytics of the agent's experiences, we extract useful action/reward patterns that help to interpret the model and control the training. Through multiple case studies conducted together with deep learning experts, we demonstrate that DQNViz can effectively help domain experts to understand, diagnose, and potentially improve DQN models.

GANViz: A Visual Analytics Approach to Understand the Adversarial Game.

Generative models bear promising implications to learn data representations in an unsupervised fashion with deep learning. Generative Adversarial Nets (GAN) is one of the most popular frameworks in this arena. Despite the promising results from different types of GANs, in-depth understanding on the adversarial training process of the models remains a challenge to domain experts. The complexity and the potential long-time training process of the models make it hard to evaluate, interpret, and optimize them. In this work, guided by practical needs from domain experts, we design and develop a visual analytics system, GANViz, aiming to help experts understand the adversarial process of GANs in-depth. Specifically, GANViz evaluates the model performance of two subnetworks of GANs, provides evidence and interpretations of the models' performance, and empowers comparative analysis with the evidence. Through our case studies with two real-world datasets, we demonstrate that GANViz can provide useful insight into helping domain experts understand, interpret, evaluate, and potentially improve GAN models.

[Transfected miR-1908 inhibits renal fibrosis via targeting transforming growth factor beta 1].

OBJECTIVE: To explore the regulatory role of miR-1908 in renal fibrosis. METHODS: The level of miR-1908 and transforming growth factor beta 1 (TGF-ß1) mRNA during renal fibrosis were detected with real-time quantitative PCR. Bioinformatics and luciferase reporter gene analyses were applied to determine the targeting relationship between miR-1908 and TGF-ß1 mRNA. After primary human renal interstitial fibroblasts were transfected with miR-1908 adenoviral expression vector in vitro, Western blotting was used to detect the protein levels of TGF-ß1, smad2/3 and matrix metalloproteinase 2 (MMP-2) in the cells. Six weeks after intraperitoneal injection of miR-1908 adenoviral vector, the renal tissue sections of the renal fibrosis mouse models were stained with Masson staining. RESULTS: Human miR-1908 showed a gradually decreasing expression during renal fibrosis process, which was completely contrary to the changes of TGF-ß1 mRNA. Overexpression of miR-1908 suppressed the expressions of TGF-ß1, smad2/3 and MMP-2 in human primary renal interstitial cells. The renal fibrosis was significantly relieved in the mice injected with miR-1908 adenovirus vector injection compared with the ones without injection. CONCLUSION: miR-1908 could inhibit renal fibrosis through targeting TGF-ß1.

Complete mitochondrial genome of a new subspecies of the blue sheep, Pseudois nayaur (Cetartiodactyla: Caprinae) from Helan Mountain in China.

The population of blue sheep, Pseudois nayaur, from Helan Mountain in China is considered as a new subspecies. We first determined and annotated its complete mitochondrial genome. The mitogenome is 16,795 bp in length, consisting of 13 protein-coding genes, 22 transfer RNA (tRNA) genes, 2 ribosomal RNA (rRNA) genes and a control region. As in other mammals, most mitochondrial genes are encoded on the heavy strand, except for ND6 and eight tRNA genes, which are encoded on the light strand. Its overall base composition is A: 33.2%, T: 26.6%, C: 26.8% and G: 13.3%. The complete mitogenome of the new subspecies of P. nayaur could provide an important data to further explore the taxonomic status of the subspecies.

TreeNetViz: revealing patterns of networks over tree structures.

Network data often contain important attributes from various dimensions such as social affiliations and areas of expertise in a social network. If such attributes exhibit a tree structure, visualizing a compound graph consisting of tree and network structures becomes complicated. How to visually reveal patterns of a network over a tree has not been fully studied. In this paper, we propose a compound graph model, TreeNet, to support visualization and analysis of a network at multiple levels of aggregation over a tree. We also present a visualization design, TreeNetViz, to offer the multiscale and cross-scale exploration and interaction of a TreeNet graph. TreeNetViz uses a Radial, Space-Filling (RSF) visualization to represent the tree structure, a circle layout with novel optimization to show aggregated networks derived from TreeNet, and an edge bundling technique to reduce visual complexity. Our circular layout algorithm reduces both total edge-crossings and edge length and also considers hierarchical structure constraints and edge weight in a TreeNet graph. These experiments illustrate that the algorithm can reduce visual cluttering in TreeNet graphs. Our case study also shows that TreeNetViz has the potential to support the analysis of a compound graph by revealing multiscale and cross-scale network patterns.