Hadoop and PySpark for reproducibility and scalability of genomic sequencing studies.

Modern genomic studies are rapidly growing in scale, and the analytical approaches used to analyze genomic data are increasing in complexity. Genomic data management poses logistic and computational challenges, and analyses are increasingly reliant on genomic annotation resources that create their own data management and versioning issues. As a result, genomic datasets are increasingly handled in ways that limit the rigor and reproducibility of many analyses. In this work, we examine the use of the Spark infrastructure for the management, access, and analysis of genomic data in comparison to traditional genomic workflows on typical cluster environments. We validate the framework by reproducing previously published results from the Alzheimer's Disease Sequencing Project. Using the framework and analyses designed using Jupyter notebooks, Spark provides improved workflows, reduces user-driven data partitioning, and enhances the portability and reproducibility of distributed analyses required for large-scale genomic studies.

A multicenter study to map genes for Fuchs endothelial corneal dystrophy: baseline characteristics and heritability.

PURPOSE: To describe the methods for family and case-control recruitment for a multicenter genetic and associated heritability analyses of Fuchs endothelial corneal dystrophy (FECD). METHODS: Twenty-nine enrolling sites with 62 trained investigators and coordinators gathered individual and family information, graded the phenotype, and collected blood and/or saliva for genetic analysis on all individuals with and without FECD. The degree of FECD was assessed in a 0 to 6 semiquantitative scale using standardized clinical methods with pathological verification of FECD on at least 1 member of each family. Central corneal thickness was measured by ultrasonic pachymetry. RESULTS: Three hundred twenty-two families with 330 affected sibling pairs with FECD were enrolled and included a total of 650 sibling pairs of all disease grades. Using the entire 7-step FECD grading scale or a dichotomous definition of severe disease, heritability was assessed in families via sib-sib correlations. Both binary indicators of severe disease and semiquantitative measures of disease severity were significantly heritable, with heritability estimates of 30% for severe disease, 37% to 39% for FECD score, and 47% for central corneal thickness. CONCLUSIONS: Genetic risk factors have a strong role in the severity of the FECD phenotype and corneal thickness. Genotyping this cohort with high-density genetic markers followed by appropriate statistical analyses should lead to novel loci for disease susceptibility.