DeepVariant-on-Spark: Small-Scale Genome Analysis Using a Cloud-Based Computing Framework.

Although sequencing a human genome has become affordable, identifying genetic variants from whole-genome sequence data is still a hurdle for researchers without adequate computing equipment or bioinformatics support. GATK is a gold standard method for the identification of genetic variants and has been widely used in genome projects and population genetic studies for many years. This was until the Google Brain team developed a new method, DeepVariant, which utilizes deep neural networks to construct an image classification model to identify genetic variants. However, the superior accuracy of DeepVariant comes at the cost of computational intensity, largely constraining its applications. Accordingly, we present DeepVariant-on-Spark to optimize resource allocation, enable multi-GPU support, and accelerate the processing of the DeepVariant pipeline. To make DeepVariant-on-Spark more accessible to everyone, we have deployed the DeepVariant-on-Spark to the Google Cloud Platform (GCP). Users can deploy DeepVariant-on-Spark on the GCP following our instruction within 20 minutes and start to analyze at least ten whole-genome sequencing datasets using free credits provided by the GCP. DeepVaraint-on-Spark is freely available for small-scale genome analysis using a cloud-based computing framework, which is suitable for pilot testing or preliminary study, while reserving the flexibility and scalability for large-scale sequencing projects.

A static VM placement and hybrid job scheduling model for green data centers.

Reducing energy consumption has become a critical issue in today data centers. Reducing the number of required physical and Virtual Machines results in energy-efficiency. In this paper, to avoid the disadvantages of VM migration, a static VM placement algorithm is proposed which places VMs on hosts in a Worst-Fit-Decreasing (WFD) fashion. To reduce energy consumption further, the effect of job scheduling policy on the number of VMs needed for maintaining QoS requirements is studied. Each VM is modeled by an M/M/* queue in space-shared, time-shared, and hybrid job scheduling policies, and energy consumption of real-time as well as non-real-time applications is analyzed. Numerical results show that the hybrid policy outperforms space-shared and time-shared policies, in terms of energy consumption as well as Service Level Agreement (SLA) violations. Moreover, our non-migration method outperforms three different algorithms which use VM migration, in terms of reducing both energy consumption and SLA Violations.

Cumulus provides cloud-based data analysis for large-scale single-cell and single-nucleus RNA-seq.

Massively parallel single-cell and single-nucleus RNA sequencing has opened the way to systematic tissue atlases in health and disease, but as the scale of data generation is growing, so is the need for computational pipelines for scaled analysis. Here we developed Cumulus-a cloud-based framework for analyzing large-scale single-cell and single-nucleus RNA sequencing datasets. Cumulus combines the power of cloud computing with improvements in algorithm and implementation to achieve high scalability, low cost, user-friendliness and integrated support for a comprehensive set of features. We benchmark Cumulus on the Human Cell Atlas Census of Immune Cells dataset of bone marrow cells and show that it substantially improves efficiency over conventional frameworks, while maintaining or improving the quality of results, enabling large-scale studies.

Scalability and cost-effectiveness analysis of whole genome-wide association studies on Google Cloud Platform and Amazon Web Services.

OBJECTIVE: Advancements in human genomics have generated a surge of available data, fueling the growth and accessibility of databases for more comprehensive, in-depth genetic studies. METHODS: We provide a straightforward and innovative methodology to optimize cloud configuration in order to conduct genome-wide association studies. We utilized Spark clusters on both Google Cloud Platform and Amazon Web Services, as well as Hail (http://doi.org/10.5281/zenodo.2646680) for analysis and exploration of genomic variants dataset. RESULTS: Comparative evaluation of numerous cloud-based cluster configurations demonstrate a successful and unprecedented compromise between speed and cost for performing genome-wide association studies on 4 distinct whole-genome sequencing datasets. Results are consistent across the 2 cloud providers and could be highly useful for accelerating research in genetics. CONCLUSIONS: We present a timely piece for one of the most frequently asked questions when moving to the cloud: what is the trade-off between speed and cost?

Cloud-based services for electronic civil registration and vital statistics systems.

This paper examines the hosting options for electronic civil registration and vital statistics (CRVS) systems, particularly the use of data centers versus cloud-based solutions. A data center is a facility that houses computer systems and associated hardware and software components, such as network and storage systems, power supplies, environment controls, and security devices. An alternative to using a data center is cloud-based hosting, which is a virtual data center hosted by a public cloud provider. The cloud is used on a pay-as-you-go basis and does not require purchasing and maintaining of hardware for data centers. It also provides more flexibility for continuous innovation in line with evolving information and communications technology.

Cost Comparison of an On-Premise IT Solution with a Cloud-Based Solution for Electronic Health Records in a Dental School Clinic.

Electronic health records (EHRs) are increasingly moving towards cloud-based web environments. While cloud-based EHRs claim substantial benefits at reduced cost, little cost-benefit research exists for dental schools. The aim of this study was to examine the cost-benefits of a cloud-based EHR compared to an on-premise client-server EHR in the University of Michigan School of Dentistry (U-M Dent). Data were collected in 2016 from the U-M Dent cost-benefit comparison of tangible and intangible factors associated with implementing a new EHR, using the Total Cost of Ownership (TCO) framework from EDUCAUSE. The TCO framework assessed three factors: foundational (overarching aspects: three items), qualitative (intangibles: 56 items), and quantitative (actual costs). Stakeholders performed factor grading, and relative assessment scores were derived for each item as well as the overall factor. The cloud-based EHR solution received higher foundational and qualitative factor summary scores. The overall cost of an on-premise solution over a two-year period was approximately $2,000,000 higher than a cloud-based solution. Cloud solutions did not carry any hidden costs, while such costs accounted for 8% (~$540,000) of the overall costs of the on-premise solution. Across the two-year period, both one-time and ongoing costs were higher for the on-premise solution than the cloud-based solution (by 40.5% and 20.5%, respectively). This study found that a cloud-based EHR system in the U-M Dent offered significant cost savings and unique benefits that were not available with the on-premise EHR solution. Based on cost, the U-M Dent has made a case for cloud-based EHR systems.

Analysis of Factors Influencing Hospitals' Implementation of a Green E-Procurement System Using a Cloud Model.

Currently, the green procurement activities of private hospitals in Taiwan follow the self-built green electronic-procurement (e-procurement) system. This requires professional personnel to take the time to regularly update the green specification and software and hardware of the e-procurement system, and the information system maintenance cost is high. In the case of a green e-procurement system crash, the efficiency of green procurement activities for hospitals is affected. If the green e-procurement can be moved to a convenient and trusty cloud computing model, this will enhance the efficiency of procurement activities and reduce the information maintenance cost for private hospitals. However, implementing a cloud model is an issue of technology innovation application and the technology-organization-environment (TOE) framework has been widely applied as the theoretical framework in technology innovation application. In addition, finding the weight of factors is a multi-criteria decision-making (MCDM) issue. Therefore, the present study first collected factors influencing implementation of the cloud mode together with the TOE as the theoretical framework, by reviewing the literature. Therefore, an expert questionnaire was designed and distributed to top managers of 20 private hospitals in southern Taiwan. The fuzzy analysis hierarchical process (FAHP), which is a MCDM tool, finds the weights of the factors influencing private hospitals in southern Taiwan when they implement a cloud green e-procurement system. The research results can enable private hospitals to successfully implement a green e-procurement system through a cloud model by optimizing resource allocation according to the weight of each factor. In addition, the results of this research can help cloud service providers of green e-procurement understand users' needs and develop relevant cloud solutions and marketing strategies.

Across-the-World Automated Optimization and Continuous-Flow Synthesis of Pharmaceutical Agents Operating Through a Cloud-Based Server.

The power of the Cloud has been harnessed for pharmaceutical compound production with remote servers based in Tokyo, Japan being left to autonomously find optimal synthesis conditions for three active pharmaceutical ingredients (APIs) in laboratories in Cambridge, UK. A researcher located in Los Angeles, USA controlled the entire process via an internet connection. The constituent synthetic steps for Tramadol, Lidocaine, and Bupropion were thus optimized with minimal intervention from operators within hours, yielding conditions satisfying customizable evaluation functions for all examples.

Monte Carlo verification of radiotherapy treatments with CloudMC.

BACKGROUND: A new implementation has been made on CloudMC, a cloud-based platform presented in a previous work, in order to provide services for radiotherapy treatment verification by means of Monte Carlo in a fast, easy and economical way. A description of the architecture of the application and the new developments implemented is presented together with the results of the tests carried out to validate its performance. METHODS: CloudMC has been developed over Microsoft Azure cloud. It is based on a map/reduce implementation for Monte Carlo calculations distribution over a dynamic cluster of virtual machines in order to reduce calculation time. CloudMC has been updated with new methods to read and process the information related to radiotherapy treatment verification: CT image set, treatment plan, structures and dose distribution files in DICOM format. Some tests have been designed in order to determine, for the different tasks, the most suitable type of virtual machines from those available in Azure. Finally, the performance of Monte Carlo verification in CloudMC is studied through three real cases that involve different treatment techniques, linac models and Monte Carlo codes. RESULTS: Considering computational and economic factors, D1_v2 and G1 virtual machines were selected as the default type for the Worker Roles and the Reducer Role respectively. Calculation times up to 33 min and costs of 16 € were achieved for the verification cases presented when a statistical uncertainty below 2% (2σ) was required. The costs were reduced to 3-6 € when uncertainty requirements are relaxed to 4%. CONCLUSIONS: Advantages like high computational power, scalability, easy access and pay-per-usage model, make Monte Carlo cloud-based solutions, like the one presented in this work, an important step forward to solve the long-lived problem of truly introducing the Monte Carlo algorithms in the daily routine of the radiotherapy planning process.

GT-WGS: an efficient and economic tool for large-scale WGS analyses based on the AWS cloud service.

BACKGROUND: Whole-genome sequencing (WGS) plays an increasingly important role in clinical practice and public health. Due to the big data size, WGS data analysis is usually compute-intensive and IO-intensive. Currently it usually takes 30 to 40 h to finish a 50× WGS analysis task, which is far from the ideal speed required by the industry. Furthermore, the high-end infrastructure required by WGS computing is costly in terms of time and money. In this paper, we aim to improve the time efficiency of WGS analysis and minimize the cost by elastic cloud computing. RESULTS: We developed a distributed system, GT-WGS, for large-scale WGS analyses utilizing the Amazon Web Services (AWS). Our system won the first prize on the Wind and Cloud challenge held by Genomics and Cloud Technology Alliance conference (GCTA) committee. The system makes full use of the dynamic pricing mechanism of AWS. We evaluate the performance of GT-WGS with a 55× WGS dataset (400GB fastq) provided by the GCTA 2017 competition. In the best case, it only took 18.4 min to finish the analysis and the AWS cost of the whole process is only 16.5 US dollars. The accuracy of GT-WGS is 99.9% consistent with that of the Genome Analysis Toolkit (GATK) best practice. We also evaluated the performance of GT-WGS performance on a real-world dataset provided by the XiangYa hospital, which consists of 5× whole-genome dataset with 500 samples, and on average GT-WGS managed to finish one 5× WGS analysis task in 2.4 min at a cost of $3.6. CONCLUSIONS: WGS is already playing an important role in guiding therapeutic intervention. However, its application is limited by the time cost and computing cost. GT-WGS excelled as an efficient and affordable WGS analyses tool to address this problem. The demo video and supplementary materials of GT-WGS can be accessed at https://github.com/Genetalks/wgs_analysis_demo .

Adaptive Resource Utilization Prediction System for Infrastructure as a Service Cloud.

Infrastructure as a Service (IaaS) cloud provides resources as a service from a pool of compute, network, and storage resources. Cloud providers can manage their resource usage by knowing future usage demand from the current and past usage patterns of resources. Resource usage prediction is of great importance for dynamic scaling of cloud resources to achieve efficiency in terms of cost and energy consumption while keeping quality of service. The purpose of this paper is to present a real-time resource usage prediction system. The system takes real-time utilization of resources and feeds utilization values into several buffers based on the type of resources and time span size. Buffers are read by R language based statistical system. These buffers' data are checked to determine whether their data follows Gaussian distribution or not. In case of following Gaussian distribution, Autoregressive Integrated Moving Average (ARIMA) is applied; otherwise Autoregressive Neural Network (AR-NN) is applied. In ARIMA process, a model is selected based on minimum Akaike Information Criterion (AIC) values. Similarly, in AR-NN process, a network with the lowest Network Information Criterion (NIC) value is selected. We have evaluated our system with real traces of CPU utilization of an IaaS cloud of one hundred and twenty servers.

TomoMiner and TomoMinerCloud: A Software Platform for Large-Scale Subtomogram Structural Analysis.

Cryo-electron tomography (cryo-ET) captures the 3D electron density distribution of macromolecular complexes in close to native state. With the rapid advance of cryo-ET acquisition technologies, it is possible to generate large numbers (>100,000) of subtomograms, each containing a macromolecular complex. Often, these subtomograms represent a heterogeneous sample due to variations in the structure and composition of a complex in situ form or because particles are a mixture of different complexes. In this case subtomograms must be classified. However, classification of large numbers of subtomograms is a time-intensive task and often a limiting bottleneck. This paper introduces an open source software platform, TomoMiner, for large-scale subtomogram classification, template matching, subtomogram averaging, and alignment. Its scalable and robust parallel processing allows efficient classification of tens to hundreds of thousands of subtomograms. In addition, TomoMiner provides a pre-configured TomoMinerCloud computing service permitting users without sufficient computing resources instant access to TomoMiners high-performance features.

The Economic and Social Value of an Image Exchange Network: A Case for the Cloud.

As the health care environment continually changes, radiologists look to the ACR's Imaging 3.0® initiative to guide the search for value. By leveraging new technology, a cloud-based image exchange network could provide secure universal access to prior images, which were previously siloed, to facilitate accurate interpretation, improved outcomes, and reduced costs. The breast imaging department represents a viable starting point given the robust data supporting the benefit of access to prior imaging studies, existing infrastructure for image sharing, and the current workflow reliance on prior images. This concept is scalable not only to the remainder of the radiology department but also to the broader medical record.

A cloud-based workflow to quantify transcript-expression levels in public cancer compendia.

Public compendia of sequencing data are now measured in petabytes. Accordingly, it is infeasible for researchers to transfer these data to local computers. Recently, the National Cancer Institute began exploring opportunities to work with molecular data in cloud-computing environments. With this approach, it becomes possible for scientists to take their tools to the data and thereby avoid large data transfers. It also becomes feasible to scale computing resources to the needs of a given analysis. We quantified transcript-expression levels for 12,307 RNA-Sequencing samples from the Cancer Cell Line Encyclopedia and The Cancer Genome Atlas. We used two cloud-based configurations and examined the performance and cost profiles of each configuration. Using preemptible virtual machines, we processed the samples for as little as $0.09 (USD) per sample. As the samples were processed, we collected performance metrics, which helped us track the duration of each processing step and quantified computational resources used at different stages of sample processing. Although the computational demands of reference alignment and expression quantification have decreased considerably, there remains a critical need for researchers to optimize preprocessing steps. We have stored the software, scripts, and processed data in a publicly accessible repository (https://osf.io/gqrz9).

VM Capacity-Aware Scheduling within Budget Constraints in IaaS Clouds.

Recently, cloud computing has drawn significant attention from both industry and academia, bringing unprecedented changes to computing and information technology. The infrastructure-as-a-Service (IaaS) model offers new abilities such as the elastic provisioning and relinquishing of computing resources in response to workload fluctuations. However, because the demand for resources dynamically changes over time, the provisioning of resources in a way that a given budget is efficiently utilized while maintaining a sufficing performance remains a key challenge. This paper addresses the problem of task scheduling and resource provisioning for a set of tasks running on IaaS clouds; it presents novel provisioning and scheduling algorithms capable of executing tasks within a given budget, while minimizing the slowdown due to the budget constraint. Our simulation study demonstrates a substantial reduction up to 70% in the overall task slowdown rate by the proposed algorithms.