Evaluation of software impact designed for biomedical research: Are we measuring what's meaningful?

Software is vital for the advancement of biology and medicine. Through analysis of usage and impact metrics of software, developers can help determine user and community engagement. These metrics can be used to justify additional funding, encourage additional use, and identify unanticipated use cases. Such analyses can help define improvement areas and assist with managing project resources. However, there are challenges associated with assessing usage and impact, many of which vary widely depending on the type of software being evaluated. These challenges involve issues of distorted, exaggerated, understated, or misleading metrics, as well as ethical and security concerns. More attention to the nuances, challenges, and considerations involved in capturing impact across the diverse spectrum of biological software is needed. Furthermore, some tools may be especially beneficial to a small audience, yet may not have comparatively compelling metrics of high usage. Although some principles are generally applicable, there is not a single perfect metric or approach to effectively evaluate a software tool's impact, as this depends on aspects unique to each tool, how it is used, and how one wishes to evaluate engagement. We propose more broadly applicable guidelines (such as infrastructure that supports the usage of software and the collection of metrics about usage), as well as strategies for various types of software and resources. We also highlight outstanding issues in the field regarding how communities measure or evaluate software impact. To gain a deeper understanding of the issues hindering software evaluations, as well as to determine what appears to be helpful, we performed a survey of participants involved with scientific software projects for the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We also investigated software among this scientific community and others to assess how often infrastructure that supports such evaluations is implemented and how this impacts rates of papers describing usage of the software. We find that although developers recognize the utility of analyzing data related to the impact or usage of their software, they struggle to find the time or funding to support such analyses. We also find that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seem to be associated with increased usage rates. Our findings can help scientific software developers make the most out of the evaluations of their software so that they can more fully benefit from such assessments.

Open-source Tools for Training Resources - OTTR.

Data science and informatics tools are developing at a blistering rate, but their users often lack the educational background or resources to efficiently apply the methods to their research. Training resources and vignettes that accompany these tools often deprecate because their maintenance is not prioritized by funding, giving teams little time to devote to such endeavors. Our group has developed Open-source Tools for Training Resources (OTTR) to offer greater efficiency and flexibility for creating and maintaining these training resources. OTTR empowers creators to customize their work and allows for a simple workflow to publish using multiple platforms. OTTR allows content creators to publish training material to multiple massive online learner communities using familiar rendering mechanics. OTTR allows the incorporation of pedagogical practices like formative and summative assessments in the form of multiple choice questions and fill in the blank problems that are automatically graded. No local installation of any software is required to begin creating content with OTTR. Thus far, 15 training courses have been created with OTTR repository template. By using the OTTR system, the maintenance workload for updating these courses across platforms has been drastically reduced. For more information about OTTR and how to get started, go to ottrproject.org.

Locating longitudinal study participants 10 years after last contact: Contemporary approaches to sample retention.

AIMS: Despite their advantages, longitudinal studies often face high rates of attrition. This study documents the extensive efforts associated with retaining a longitudinal cohort last contacted 10 years earlier. METHOD: We examine the processes and outcomes of attempts to reach 1736 individuals who have been part of a multiwave study about growing up in Ontario, Canada. Contact methods include email, phone, text, social media, postal mail, announcements in newspapers, subway stations, and music streaming services. RESULTS: Challenges included a lack of consistent annual communication with participants, children moving out of the parental home, and changes in email addresses and phone numbers. The most effective contact method was phone; text messages and friend referrals were the least effective. Overall, 41.5% of the original sample was reached. Locating former research participants years later necessitated multiple and repeated contact attempts, and intensive human resources. CONCLUSION: Ten lessons for effective sample retention are discussed. In summary, reducing attrition depends on a comprehensive study design and an organized and flexible protocol that adapts to a study's ever-changing needs.

Motivation, inclusivity, and realism should drive data science education.

Data science education provides tremendous opportunities but remains inaccessible to many communities. Increasing the accessibility of data science to these communities not only benefits the individuals entering data science, but also increases the field's innovation and potential impact as a whole. Education is the most scalable solution to meet these needs, but many data science educators lack formal training in education. Our group has led education efforts for a variety of audiences: from professional scientists to high school students to lay audiences. These experiences have helped form our teaching philosophy which we have summarized into three main ideals: 1) motivation, 2) inclusivity, and 3) realism. 20 we also aim to iteratively update our teaching approaches and curriculum as we find ways to better reach these ideals. In this manuscript we discuss these ideals as well practical ideas for how to implement these philosophies in the classroom.

Boron Isotope Analysis Reveals Borate Selectivity in Seaweeds.

The role of boron in terrestrial plant physiology is diverse and increasingly well understood, but its role in marine aquatic eukaryotes is less clear. Our research reveals a distinctive and large offset in boron isotopes from seawater, irrespective of seaweed type or season. We show that the offset is consistent with the incorporation of borate from seawater. Boron is a known micronutrient in plants but very few studies have used boron isotopes to investigate boron's role in plant physiology. Seaweed, as the most primitive multicellular plant, has an important role in investigating wider plant adaptations that use boron to meet functional needs. Furthermore, seaweed and other plants are a key base nutrient provider in food webs, supplying boron to consumers and playing a critical role in boron environmental cycling.

Comprehensive assessment of multiple biases in small RNA sequencing reveals significant differences in the performance of widely used methods.

BACKGROUND: RNA sequencing offers advantages over other quantification methods for microRNA (miRNA), yet numerous biases make reliable quantification challenging. Previous evaluations of these biases have focused on adapter ligation bias with limited evaluation of reverse transcription bias or amplification bias. Furthermore, evaluations of the quantification of isomiRs (miRNA isoforms) or the influence of starting amount on performance have been very limited. No study had yet evaluated the quantification of isomiRs of altered length or compared the consistency of results derived from multiple moderate starting inputs. We therefore evaluated quantifications of miRNA and isomiRs using four library preparation kits, with various starting amounts, as well as quantifications following removal of duplicate reads using unique molecular identifiers (UMIs) to mitigate reverse transcription and amplification biases. RESULTS: All methods resulted in false isomiR detection; however, the adapter-free method tested was especially prone to false isomiR detection. We demonstrate that using UMIs improves accuracy and we provide a guide for input amounts to improve consistency. CONCLUSIONS: Our data show differences and limitations of current methods, thus raising concerns about the validity of quantification of miRNA and isomiRs across studies. We advocate for the use of UMIs to improve accuracy and reliability of miRNA quantifications.

xMD-miRNA-seq to generate near in vivo miRNA expression estimates in colon epithelial cells.

Accurate, RNA-seq based, microRNA (miRNA) expression estimates from primary cells have recently been described. However, this in vitro data is mainly obtained from cell culture, which is known to alter cell maturity/differentiation status, significantly changing miRNA levels. What is needed is a robust method to obtain in vivo miRNA expression values directly from cells. We introduce expression microdissection miRNA small RNA sequencing (xMD-miRNA-seq), a method to isolate cells directly from formalin fixed paraffin-embedded (FFPE) tissues. xMD-miRNA-seq is a low-cost, high-throughput, immunohistochemistry-based method to capture any cell type of interest. As a proof-of-concept, we isolated colon epithelial cells from two specimens and performed low-input small RNA-seq. We generated up to 600,000 miRNA reads from the samples. Isolated epithelial cells, had abundant epithelial-enriched miRNA expression (miR-192; miR-194; miR-200b; miR-200c; miR-215; miR-375) and overall similar miRNA expression patterns to other epithelial cell populations (colonic enteroids and flow-isolated colon epithelium). xMD-derived epithelial cells were generally not contaminated by other adjacent cells of the colon as noted by t-SNE analysis. xMD-miRNA-seq allows for simple, economical, and efficient identification of cell-specific miRNA expression estimates. Further development will enhance rapid identification of cell-specific miRNA expression estimates in health and disease for nearly any cell type using archival FFPE material.

Whole Genome Expression Analysis in a Mouse Model of Tauopathy Identifies MECP2 as a Possible Regulator of Tau Pathology.

Increasing evidence suggests that hyperphosphorylation and aggregation of microtubule-associated protein tau (MAPT or tau) correlates with the development of cognitive impairment in Alzheimer's disease (AD) and related tauopathies. While numerous attempts have been made to model AD-relevant tau pathology in various animal models, there has been very limited success for these models to fully recapitulate the progression of disease as seen in human tauopathies. Here, we performed whole genome gene expression in a genomic mouse model of tauopathy that expressed human MAPT gene under the control of endogenous human MAPT promoter and also were complete knockout for endogenous mouse tau [referred to as 'hTau MaptKO(Duke)' mice]. First, whole genome expression analysis revealed 64 genes, which were differentially expressed (32 up-regulated and 32 down-regulated) in the hippocampus of 6-month-old hTau MaptKO(Duke) mice compared to age-matched non-transgenic controls. Genes relevant to neuronal function or neurological disease include up-regulated genes: PKC-alpha (Prkca), MECP2 (Mecp2), STRN4 (Strn4), SLC40a1 (Slc40a1), POLD2 (Pold2), PCSK2 (Pcsk2), and down-regulated genes: KRT12 (Krt12), LASS1 (Cers1), PLAT (Plat), and NRXN1 (Nrxn1). Second, network analysis suggested anatomical structure development, cellular metabolic process, cell death, signal transduction, and stress response were significantly altered biological processes in the hTau MaptKO(Duke) mice as compared to age-matched non-transgenic controls. Further characterization of a sub-group of significantly altered genes revealed elevated phosphorylation of MECP2 (methyl-CpG-binding protein-2), which binds to methylated CpGs and associates with chromatin, in hTau MaptKO(Duke) mice compared to age-matched controls. Third, phoshpho-MECP2 was elevated in autopsy brain samples from human AD compared to healthy controls. Finally, siRNA-mediated knockdown of MECP2 in human tau expressing N2a cells resulted in a significant decrease in total and phosphorylated tau. Together, these results suggest that MECP2 is a potential novel regulator of tau pathology relevant to AD and tauopathies.

Primary angle-closure glaucoma: an update.

Primary angle-closure glaucoma is potentially a devastating disease, responsible for half of glaucoma-related blindness worldwide. Angle closure is characterized by appositional approximation or contact between the iris and trabecular meshwork. It tends to develop in eyes with shallow anterior chambers, anteriorly positioned or pushed lenses, and angle crowding. Risk of primary angle-closure glaucoma is high among women, the elderly and the hyperopic, and it is most prevalent in Asia. Investigation into genetic mechanisms of glaucoma inheritance is underway. Diagnosis relies on gonioscopy and may be aided by anterior segment optical coherence tomography and ultrasound biomicroscopy. Treatment is designed to control intraocular pressure while monitoring changes to the angle and optic nerve head. Treatment typically begins with medical management through pressure-reducing topical medications. Peripheral iridotomy is often performed to alleviate pupillary block, while laser iridoplasty has been found effective for mechanisms of closure other than pupillary block, such as plateau iris syndrome. Phacoemulsification, with or without goniosynechialysis, both in eyes with existing cataracts and in those with clear lenses, is thus far a viable treatment alternative. Long-term research currently underway will examine its efficacy in cases of angle closure in early stages of the disease. Endoscopic cyclophotocoagulation is another treatment option, which can be combined with cataract surgery. Trabeculectomy remains effective therapy for more advanced cases.

MIR137HG risk variant rs1625579 genotype is related to corpus callosum volume in schizophrenia.

Genome-wide association studies implicate the MIR137HG risk variant rs1625579 (MIR137HGrv) within the host gene for microRNA-137 as a potential regulator of schizophrenia susceptibility. We examined the influence of MIR137HGrv genotype on 17 subcortical and callosal volumes in a large sample of individuals with schizophrenia and healthy controls (n=841). Although the volumes were overall reduced relative to healthy controls, for individuals with schizophrenia the homozygous MIR137HGrv risk genotype was associated with attenuated reduction of mid-posterior corpus callosum volume (p=0.001), along with trend-level effects in the adjacent central and posterior corpus callosum. These findings are unique in the literature and remain robust after analysis in ethnically homogenous and single-scanner subsets of the larger sample. Thus, our study suggests that the mechanisms whereby MIR137HGrv works to increase schizophrenia risk are not those that generate the corpus callosum volume reductions commonly found in the disorder.

Meta gene set enrichment analyses link miR-137-regulated pathways with schizophrenia risk.

BACKGROUND: A single nucleotide polymorphism (SNP) within MIR137, the host gene for miR-137, has been identified repeatedly as a risk factor for schizophrenia. Previous genetic pathway analyses suggest that potential targets of this microRNA (miRNA) are also highly enriched in schizophrenia-relevant biological pathways, including those involved in nervous system development and function. METHODS: In this study, we evaluated the schizophrenia risk of miR-137 target genes within these pathways. Gene set enrichment analysis of pathway-specific miR-137 targets was performed using the stage 1 (21,856 subjects) schizophrenia genome wide association study data from the Psychiatric Genomics Consortium and a small independent replication cohort (244 subjects) from the Mind Clinical Imaging Consortium and Northwestern University. RESULTS: Gene sets of potential miR-137 targets were enriched with variants associated with schizophrenia risk, including target sets involved in axonal guidance signaling, Ephrin receptor signaling, long-term potentiation, PKA signaling, and Sertoli cell junction signaling. The schizophrenia-risk association of SNPs in PKA signaling targets was replicated in the second independent cohort. CONCLUSIONS: These results suggest that these biological pathways may be involved in the mechanisms by which this MIR137 variant enhances schizophrenia risk. SNPs in targets and the miRNA host gene may collectively lead to dysregulation of target expression and aberrant functioning of such implicated pathways. Pathway-guided gene set enrichment analyses should be useful in evaluating the impact of other miRNAs and target genes in different diseases.

Genetic markers of white matter integrity in schizophrenia revealed by parallel ICA.

It is becoming a consensus that white matter integrity is compromised in schizophrenia (SZ), however the underlying genetics remains elusive. Evidence suggests a polygenic basis of the disorder, which involves various genetic variants with modest individual effect sizes. In this work, we used a multivariate approach, parallel independent component analysis (P-ICA), to explore the genetic underpinnings of white matter abnormalities in SZ. A pre-filtering step was first applied to locate 6527 single nucleotide polymorphisms (SNPs) discriminating patients from controls with a nominal uncorrected p-value of 0.01. These potential susceptibility loci were then investigated for associations with fractional anisotropy (FA) images in a cohort consisting of 73 SZ patients and 87 healthy controls (HC). A significant correlation (r = -0.37, p = 1.25 × 10(-6)) was identified between one genetic factor and one FA component after controlling for scanning site, ethnicity, age, and sex. The identified FA-SNP association remained stable in a 10-fold validation. A 5000-run permutation test yielded a p-value of 2.00 × 10(-4). The FA component reflected decreased white matter integrity in the forceps major for SZ patients. The SNP component was overrepresented in genes whose products are involved in corpus callosum morphology (e.g., CNTNAP2, NPAS3, and NFIB) as well as canonical pathways of synaptic long term depression and protein kinase A signaling. Taken together, our finding delineates a part of genetic architecture underlying SZ-related FA reduction, emphasizing the important role of genetic variants involved in neural development.

Associations between DNA methylation and schizophrenia-related intermediate phenotypes - a gene set enrichment analysis.

Multiple genetic approaches have identified microRNAs as key effectors in psychiatric disorders as they post-transcriptionally regulate expression of thousands of target genes. However, their role in specific psychiatric diseases remains poorly understood. In addition, epigenetic mechanisms such as DNA methylation, which affect the expression of both microRNAs and coding genes, are critical for our understanding of molecular mechanisms in schizophrenia. Using clinical, imaging, genetic, and epigenetic data of 103 patients with schizophrenia and 111 healthy controls of the Mind Clinical Imaging Consortium (MCIC) study of schizophrenia, we conducted gene set enrichment analysis to identify markers for schizophrenia-associated intermediate phenotypes. Genes were ranked based on the correlation between DNA methylation patterns and each phenotype, and then searched for enrichment in 221 predicted microRNA target gene sets. We found the predicted hsa-miR-219a-5p target gene set to be significantly enriched for genes (EPHA4, PKNOX1, ESR1, among others) whose methylation status is correlated with hippocampal volume independent of disease status. Our results were strengthened by significant associations between hsa-miR-219a-5p target gene methylation patterns and hippocampus-related neuropsychological variables. IPA pathway analysis of the respective predicted hsa-miR-219a-5p target genes revealed associated network functions in behavior and developmental disorders. Altered methylation patterns of predicted hsa-miR-219a-5p target genes are associated with a structural aberration of the brain that has been proposed as a possible biomarker for schizophrenia. The (dys)regulation of microRNA target genes by epigenetic mechanisms may confer additional risk for developing psychiatric symptoms. Further study is needed to understand possible interactions between microRNAs and epigenetic changes and their impact on risk for brain-based disorders such as schizophrenia.

Potential Impact of miR-137 and Its Targets in Schizophrenia.

The significant impact of microRNAs (miRNAs) on disease pathology is becoming increasingly evident. These small non-coding RNAs have the ability to post-transcriptionally silence the expression of thousands of genes. Therefore, dysregulation of even a single miRNA could confer a large polygenic effect. Schizophrenia is a genetically complex illness thought to involve multiple genes each contributing a small risk. Large genome-wide association studies identified miR-137, a miRNA shown to be involved in neuronal maturation, as one of the top risk genes. To assess the potential mechanism of impact of miR-137 in this disorder and identify its targets, we used a combination of literature searches, ingenuity pathway analysis (IPA), and freely accessible bioinformatics resources. Using TargetScan and the schizophrenia gene resource (SZGR) database, we found that in addition to CSMD1, C10orf26, CACNA1C, TCF4, and ZNF804A, five schizophrenia risk genes whose transcripts are also validated miR-137 targets, there are other schizophrenia-associated genes that may be targets of miR-137, including ERBB4, GABRA1, GRIN2A, GRM5, GSK3B, NRG2, and HTR2C. IPA analyses of all the potential targets identified several nervous system (NS) functions as the top canonical pathways including synaptic long-term potentiation, a process implicated in learning and memory mechanisms and recently shown to be altered in patients with schizophrenia. Among the subset of targets involved in NS development and function, the top scoring pathways were ephrin receptor signaling and axonal guidance, processes that are critical for proper circuitry formation and were shown to be disrupted in schizophrenia. These results suggest that miR-137 may indeed play a substantial role in the genetic etiology of schizophrenia by regulating networks involved in neural development and brain function.

The value of intraocular pressure asymmetry in diagnosing glaucoma.

PURPOSE: To investigate the amount of intraocular pressure (IOP) asymmetry in a large group of ethnically diverse patients with and without glaucoma, and to delineate the risk for glaucoma which increasing amounts of IOP asymmetry confer upon the patient. PATIENTS AND METHODS: Collaborative retrospective study of 326 glaucoma patients and 326 controls. Former Wills Eye Institute fellows collected single pre-treatment measurements of IOP on patients diagnosed as having definite glaucoma based on characteristic optic nerve damage and confirmatory visual field damage. Patients with a normal eye examination who had normal-appearing optic discs and no apparent glaucoma, or who had a normal eye examination in association with refractive error or cataract, were used as controls. RESULTS: Intraocular pressure asymmetry is a significant risk factor for having glaucoma (odds ratio, 2.14; 95% confidence interval, 1.86-2.47; P<0.001). Absence of IOP asymmetry between the fellow eyes is associated with a 1% probability of having glaucoma. A difference of 3 mm Hg is associated with a 6% probability of having glaucoma, and a difference of >6 mm Hg with a 57% probability of having glaucoma. The association between IOP asymmetry and glaucoma status is significant for subjects with both elevated IOP (P=0.014) and statistically normal IOP (maximum IOP ≤ 21 mm Hg; P<0.001). CONCLUSIONS: Inter-eye asymmetry of IOP is a common finding in patients with glaucoma. There is a direct relationship between the amount of IOP asymmetry between the fellow eyes and the likelihood of having glaucoma.

Efficacy of surveillance for 2009 H1N1 pandemic within a healthcare system.

In the fall of 2009, our hospital introduced a surveillance system to monitor the increase in cases of H1N1 pandemic influenza A virus infection. The system involved tracking cases of influenza-like illness in the emergency department, the outpatient clinics, and the inpatient wards as well as specimens with positive polymerase chain reaction results reported by the microbiology laboratory. Our data correlated well with national and regional data.

Effects of a novel cognition-enhancing agent on fetal ethanol-induced learning deficits.

BACKGROUND: Drinking during pregnancy has been associated with learning disabilities in affected offspring. At present, there are no clinically effective pharmacotherapeutic interventions for these learning deficits. Here, we examined the effects of ABT-239, a histamine H3 receptor antagonist, on fetal ethanol-induced fear conditioning and spatial memory deficits. METHODS AND RESULTS: Long-Evans rat dams stably consumed a mean of 2.82 g ethanol/kg during a 4-hour period each day during pregnancy. This voluntary drinking pattern produced a mean peak serum ethanol level of 84 mg/dl. Maternal weight gain, litter size and birth weights were not different between the ethanol-consuming and control groups. Female adult offspring from the control and fetal alcohol-exposed (FAE) groups received saline or 1 mg ABT-239/kg 30 minutes prior to fear conditioning training. Three days later, freezing time to the context was significantly reduced in saline-treated FAE rats compared to control. Freezing time in ABT-239-treated FAE rats was not different than that in controls. In the spatial navigation study, adult male offspring received a single injection of saline or ABT-239 30 minutes prior to 12 training trials on a fixed platform version of the Morris Water Task. All rats reached the same performance asymptote on Trials 9 to 12 on Day 1. However, 4 days later, first-trial retention of platform location was significantly worse in the saline-treated FAE rats compared control offspring. Retention by ABT-239-treated FAE rats was similar to that by controls. ABT-239's effect on spatial memory retention in FAE rats was dose dependent. CONCLUSIONS: These results suggest that ABT-239 administered prior to training can improve retention of acquired information by FAE offspring on more challenging versions of hippocampal-sensitive learning tasks. Further, the differential effects of ABT-239 in FAE offspring compared to controls raises questions about the impact of fetal ethanol exposure on histaminergic neurotransmission in affected offspring.