A competitive scheme for storing sparse representation of X-Ray medical images.

A competitive scheme for economic storage of the informational content of an X-Ray image, as it can be used for further processing, is presented. It is demonstrated that sparse representation of that type of data can be encapsulated in a small file without affecting the quality of the recovered image. The proposed representation, which is inscribed within the context of data reduction, provides a format for saving the image information in a way that could assist methodologies for analysis and classification. The competitiveness of the resulting file is compared against the compression standards JPEG and JPEG2000.

Data Streaming for Metabolomics: Accelerating Data Processing and Analysis from Days to Minutes.

The speed and throughput of analytical platforms has been a driving force in recent years in the "omics" technologies and while great strides have been accomplished in both chromatography and mass spectrometry, data analysis times have not benefited at the same pace. Even though personal computers have become more powerful, data transfer times still represent a bottleneck in data processing because of the increasingly complex data files and studies with a greater number of samples. To meet the demand of analyzing hundreds to thousands of samples within a given experiment, we have developed a data streaming platform, XCMS Stream, which capitalizes on the acquisition time to compress and stream recently acquired data files to data processing servers, mimicking just-in-time production strategies from the manufacturing industry. The utility of this XCMS Online-based technology is demonstrated here in the analysis of T cell metabolism and other large-scale metabolomic studies. A large scale example on a 1000 sample data set demonstrated a 10 000-fold time savings, reducing data analysis time from days to minutes. Further, XCMS Stream has the capability to increase the efficiency of downstream biochemical dependent data acquisition (BDDA) analysis by initiating data conversion and data processing on subsets of data acquired, expanding its application beyond data transfer to smart preliminary data decision-making prior to full acquisition.

Fast lossless compression via cascading Bloom filters.

BACKGROUND: Data from large Next Generation Sequencing (NGS) experiments present challenges both in terms of costs associated with storage and in time required for file transfer. It is sometimes possible to store only a summary relevant to particular applications, but generally it is desirable to keep all information needed to revisit experimental results in the future. Thus, the need for efficient lossless compression methods for NGS reads arises. It has been shown that NGS-specific compression schemes can improve results over generic compression methods, such as the Lempel-Ziv algorithm, Burrows-Wheeler transform, or Arithmetic Coding. When a reference genome is available, effective compression can be achieved by first aligning the reads to the reference genome, and then encoding each read using the alignment position combined with the differences in the read relative to the reference. These reference-based methods have been shown to compress better than reference-free schemes, but the alignment step they require demands several hours of CPU time on a typical dataset, whereas reference-free methods can usually compress in minutes. RESULTS: We present a new approach that achieves highly efficient compression by using a reference genome, but completely circumvents the need for alignment, affording a great reduction in the time needed to compress. In contrast to reference-based methods that first align reads to the genome, we hash all reads into Bloom filters to encode, and decode by querying the same Bloom filters using read-length subsequences of the reference genome. Further compression is achieved by using a cascade of such filters. CONCLUSIONS: Our method, called BARCODE, runs an order of magnitude faster than reference-based methods, while compressing an order of magnitude better than reference-free methods, over a broad range of sequencing coverage. In high coverage (50-100 fold), compared to the best tested compressors, BARCODE saves 80-90% of the running time while only increasing space slightly.

Ensemble analysis of adaptive compressed genome sequencing strategies.

BACKGROUND: Acquiring genomes at single-cell resolution has many applications such as in the study of microbiota. However, deep sequencing and assembly of all of millions of cells in a sample is prohibitively costly. A property that can come to rescue is that deep sequencing of every cell should not be necessary to capture all distinct genomes, as the majority of cells are biological replicates. Biologically important samples are often sparse in that sense. In this paper, we propose an adaptive compressed method, also known as distilled sensing, to capture all distinct genomes in a sparse microbial community with reduced sequencing effort. As opposed to group testing in which the number of distinct events is often constant and sparsity is equivalent to rarity of an event, sparsity in our case means scarcity of distinct events in comparison to the data size. Previously, we introduced the problem and proposed a distilled sensing solution based on the breadth first search strategy. We simulated the whole process which constrained our ability to study the behavior of the algorithm for the entire ensemble due to its computational intensity. RESULTS: In this paper, we modify our previous breadth first search strategy and introduce the depth first search strategy. Instead of simulating the entire process, which is intractable for a large number of experiments, we provide a dynamic programming algorithm to analyze the behavior of the method for the entire ensemble. The ensemble analysis algorithm recursively calculates the probability of capturing every distinct genome and also the expected total sequenced nucleotides for a given population profile. Our results suggest that the expected total sequenced nucleotides grows proportional to log of the number of cells and proportional linearly with the number of distinct genomes. The probability of missing a genome depends on its abundance and the ratio of its size over the maximum genome size in the sample. The modified resource allocation method accommodates a parameter to control that probability. AVAILABILITY: The squeezambler 2.0 C++ source code is available at http://sourceforge.net/projects/hyda/.

An improved YEF-DCT based compression algorithm for video capsule endoscopy.

Video capsule endoscopy is a non-invasive technique to receive images of intestine for medical diagnostics. The main design challenges of endoscopy capsule are accruing and transmitting acceptable quality images by utilizing as less hardware and battery power as possible. In order to save wireless transmission power and bandwidth, an efficient image compression algorithm needs to be implemented inside the endoscopy electronic capsule. In this paper, an integer discrete-cosine-transform (DCT) based algorithm is presented that works on a low-complexity color-space specially designed for wireless capsule endoscopy application. First of all, thousands of human endoscopic images and video frames have been analyzed to identify special intestinal features present in those frames. Then a color space, referred as YEF, is used. The YEF converter is lossless and takes only a few adders and shift operation to implement. A low-cost quantization scheme with variable chroma sub-sampling options is also implemented to achieve higher compression. Comparing with the existing works, the proposed transform coding based compressor performs strongly with an average compression ratio of 85% and a high image quality index, peak-signal-to-noise ratio (PSNR) of 52 dB.

Transformations for the compression of FASTQ quality scores of next-generation sequencing data.

MOTIVATION: The growth of next-generation sequencing means that more effective and efficient archiving methods are needed to store the generated data for public dissemination and in anticipation of more mature analytical methods later. This article examines methods for compressing the quality score component of the data to partly address this problem. RESULTS: We compare several compression policies for quality scores, in terms of both compression effectiveness and overall efficiency. The policies employ lossy and lossless transformations with one of several coding schemes. Experiments show that both lossy and lossless transformations are useful, and that simple coding methods, which consume less computing resources, are highly competitive, especially when random access to reads is needed. AVAILABILITY AND IMPLEMENTATION: Our C++ implementation, released under the Lesser General Public License, is available for download at http://www.cb.k.u-tokyo.ac.jp/asailab/members/rwan. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Efficient encoding and rapid decoding for interactive visualization of large three-dimensional hyperspectral chemical images.

Interactive visualization of data from a new generation of chemical imaging systems requires coding that is efficient and accessible. New technologies for secondary ion mass spectrometry (SIMS) generate large three-dimensional, hyperspectral datasets with high spatial and spectral resolution. Interactive visualization is important for chemical analysis, but the raw dataset size exceeds the memory capacities of typical current computer systems and is a significant obstacle. This paper reports the development of a lossless coding method that is memory efficient, enabling large SIMS datasets to be held in fast memory, and supports quick access for interactive visualization. The approach provides pixel indexing, as required for chemical imaging applications, and is based on the statistical characteristics of the data. The method uses differential time-of-flight to effect mass-spectral run-length-encoding and uses a scheme for variable-length, byte-unit representations for both mass-spectral time-of-flight and intensity values. Experiments demonstrate high compression rates and fast access.

Cost-effective handling of digital medical images in the telemedicine environment.

BACKGROUND: This paper concentrates on strategies for less costly handling of medical images. Aspects of digitization using conventional digital cameras, lossy compression with good diagnostic quality, and visualization through less costly monitors are discussed. METHOD: For digitization of film-based media, subjective evaluation of the suitability of digital cameras as an alternative to the digitizer was undertaken. To save on storage, bandwidth and transmission time, the acceptable degree of compression with diagnostically no loss of important data was studied through randomized double-blind tests of the subjective image quality when compression noise was kept lower than the inherent noise. A diagnostic experiment was undertaken to evaluate normal low cost computer monitors as viable viewing displays for clinicians. RESULTS: The results show that conventional digital camera images of X-ray images were diagnostically similar to the expensive digitizer. Lossy compression, when used moderately with the imaging noise to compression noise ratio (ICR) greater than four, can bring about image improvement with better diagnostic quality than the original image. Statistical analysis shows that there is no diagnostic difference between expensive high quality monitors and conventional computer monitors. CONCLUSION: The results presented show good potential in implementing the proposed strategies to promote widespread cost-effective telemedicine and digital medical environments.