How Trustworthy are Performance Evaluations for Basic Vision Tasks?

This article examines performance evaluation criteria for basic vision tasks involving sets of objects namely, object detection, instance-level segmentation and multi-object tracking. The rankings of algorithms by a criterion can fluctuate with different choices of parameters, e.g. Intersection over Union (IoU) threshold, making their evaluations unreliable. More importantly, there is no means to verify whether we can trust the evaluations of a criterion. This work suggests a notion of trustworthiness for performance criteria, which requires (i) robustness to parameters for reliability, (ii) contextual meaningfulness in sanity tests, and (iii) consistency with mathematical requirements such as the metric properties. We observe that these requirements were overlooked by many widely-used criteria, and explore alternative criteria using metrics for sets of shapes. We also assess all these criteria based on the suggested requirements for trustworthiness.

CellTrackVis: interactive browser-based visualization for analyzing cell trajectories and lineages.

BACKGROUND: Automatic cell tracking methods enable practitioners to analyze cell behaviors efficiently. Notwithstanding the continuous development of relevant software, user-friendly visualization tools have room for further improvements. Typical visualization mostly comes with main cell tracking tools as a simple plug-in, or relies on specific software/platforms. Although some tools are standalone, limited visual interactivity is provided, or otherwise cell tracking outputs are partially visualized. RESULTS: This paper proposes a self-reliant visualization system, CellTrackVis, to support quick and easy analysis of cell behaviors. Interconnected views help users discover meaningful patterns of cell motions and divisions in common web browsers. Specifically, cell trajectory, lineage, and quantified information are respectively visualized in a coordinated interface. In particular, immediate interactions among modules enable the study of cell tracking outputs to be more effective, and also each component is highly customizable for various biological tasks. CONCLUSIONS: CellTrackVis is a standalone browser-based visualization tool. Source codes and data sets are freely available at http://github.com/scbeom/celltrackvis with the tutorial at http://scbeom.github.io/ctv_tutorial .

Tracking Multiple Marine Ships via Multiple Sensors with Unknown Backgrounds.

In multitarget tracking, knowledge of the backgrounds plays a crucial role in the accuracy of the tracker. Clutter and detection probability are the two essential background parameters which are usually assumed to be known constants although they are, in fact, unknown and time varying. Incorrect values of these parameters lead to a degraded or biased performance of the tracking algorithms. This paper proposes a method for online tracking multiple targets using multiple sensors which jointly adapts to the unknown clutter rate and the probability of detection. An effective filter is developed from parallel estimation of these parameters and then feeding them into the state-of-the-art generalized labeled multi-Bernoulli filter. Provided that the fluctuation of these unknown backgrounds is slowly-varying in comparison to the rate of measurement-update data, the validity of the proposed method is demonstrated via numerical study using multistatic Doppler data.

GLMB Tracker with Partial Smoothing.

In this paper, we introduce a tracking algorithm based on labeled Random Finite Sets (RFS) and Rauch-Tung-Striebel (RTS) smoother via a Generalized Labeled Multi-Bernoulli (GLMB) multi-scan estimator to track multiple objects in a wide range of tracking scenarios. In the forward filtering stage, we use the GLMB filter to generate a set of labels and the association history between labels and the measurements. In the trajectory-estimating stage, we apply a track management strategy to eliminate tracks with short lifespan compared to a threshold value. Subsequently, we apply the information of trajectories captured from the forward GLMB filtering stage to carry out standard forward filtering and RTS backward smoothing on each estimated trajectory. For the experiment, we implement the tracker with standard GLMB filter, the hybrid track-before-detect (TBD) GLMB filter, and the GLMB filter with objects spawning. The results show improvements in tracking performance for all implemented trackers given negligible extra computational effort compared to standard GLMB filters.