An assessment of the autism neuroimaging literature for the prospects of re-executability.

Background: The degree of reproducibility of the neuroimaging literature in psychiatric application areas has been called into question and the issues that relate to this reproducibility are extremely complex. Some of these complexities have to do with the underlying biology of the disorders that we study and others arise due to the technology we apply to the analysis of the data we collect. Ultimately, the observations we make get communicated to the rest of the community through publications in the scientific literature. Methods: We sought to perform a 're-executability survey' to evaluate the recent neuroimaging literature with an eye toward seeing if the technical aspects of our publication practices are helping or hindering the overall quest for a more reproducible understanding of brain development and aging. The topic areas examined include availability of the data, the precision of the imaging method description and the reporting of the statistical analytic approach, and the availability of the complete results. We applied the survey to 50 publications in the autism neuroimaging literature that were published between September 16, 2017 to October 1, 2018. Results: The results of the survey indicate that for the literature examined, data that is not already part of a public repository is rarely available, software tools are usually named but versions and operating system are not, it is expected that reasonably skilled analysts could approximately perform the analyses described, and the complete results of the studies are rarely available.  Conclusions: We have identified that there is ample room for improvement in research publication practices. We hope exposing these issues in the retrospective literature can provide guidance and motivation for improving this aspect of our reporting practices in the future.

Everything Matters: The ReproNim Perspective on Reproducible Neuroimaging.

There has been a recent major upsurge in the concerns about reproducibility in many areas of science. Within the neuroimaging domain, one approach is to promote reproducibility is to target the re-executability of the publication. The information supporting such re-executability can enable the detailed examination of how an initial finding generalizes across changes in the processing approach, and sampled population, in a controlled scientific fashion. ReproNim: A Center for Reproducible Neuroimaging Computation is a recently funded initiative that seeks to facilitate the "last mile" implementations of core re-executability tools in order to reduce the accessibility barrier and increase adoption of standards and best practices at the neuroimaging research laboratory level. In this report, we summarize the overall approach and tools we have developed in this domain.

Corrigendum: Data Citation in Neuroimaging: Proposed Best Practices for Data Identification and Attribution.

[This corrects the article on p. 34 in vol. 10, PMID: 27570508.].

Data Citation in Neuroimaging: Proposed Best Practices for Data Identification and Attribution.

Data sharing and reuse, while widely accepted as good ideas, have been slow to catch on in any concrete and consistent way. One major hurdle within the scientific community has been the lack of widely accepted standards for citing that data, making it difficult to track usage and measure impact. Within the neuroimaging community, there is a need for a way to not only clearly identify and cite datasets, but also to derive new aggregate sets from multiple sources while clearly maintaining lines of attribution. This work presents a functional prototype of a system to integrate Digital Object Identifiers (DOI) and a standardized metadata schema into a XNAT-based repository workflow, allowing for identification of data at both the project and image level. These item and source level identifiers allow any newly defined combination of images, from any number of projects, to be tagged with a new group-level DOI that automatically inherits the individual attributes and provenance information of its constituent parts. This system enables the tracking of data reuse down to the level of individual images. The implementation of this type of data identification system would impact researchers and data creators, data hosting facilities, and data publishers, but the benefit of having widely accepted standards for data identification and attribution would go far toward making data citation practical and advantageous.

The NITRC image repository.

The Neuroimaging Informatics Tools and Resources Clearinghouse (NITRC - www.nitrc.org) suite of services include a resources registry, image repository and a cloud computational environment to meet the needs of the neuroimaging researcher. NITRC provides image-sharing functionality through both the NITRC Resource Registry (NITRC-R), where bulk data files can be released through the file release system (FRS), and the NITRC Image Repository (NITRC-IR), a XNAT-based image data management system. Currently hosting 14 projects, 6845 subjects, and 8285 MRI imaging sessions, NITRC-IR provides a large array of structural, diffusion and resting state MRI data. Designed to be flexible about management of data access policy, NITRC provides a simple, free, NIH-funded service to support resource sharing in general, and image sharing in particular.

Connectivity in Autism: A Review of MRI Connectivity Studies.

Autism spectrum disorder (ASD) affects 1 in 50 children between the ages of 6 and 17 years. The etiology of ASD is not precisely known. ASD is an umbrella term, which includes both low- (IQ < 70) and high-functioning (IQ > 70) individuals. A better understanding of the disorder and how it manifests in individual subjects can lead to more effective intervention plans to fulfill the individual's treatment needs.Magnetic resonance imaging (MRI) is a non-invasive investigational tool that can be used to study the ways in which the brain develops or deviates from the typical developmental trajectory. MRI offers insights into the structure, function, and metabolism of the brain. In this article, we review published studies on brain connectivity changes in ASD using either resting state functional MRI or diffusion tensor imaging.The general findings of decreases in white matter integrity and in long-range neural coherence are well known in the ASD literature. Nevertheless, the detailed localization of these findings remains uncertain, and few studies link these changes in connectivity with the behavioral phenotype of the disorder. With the help of data sharing and large-scale analytic efforts, however, the field is advancing toward several convergent themes, including the reduced functional coherence of long-range intra-hemispheric cortico-cortical default mode circuitry, impaired inter-hemispheric regulation, and an associated, perhaps compensatory, increase in local and short-range cortico-subcortical coherence.

Interacting with the National Database for Autism Research (NDAR) via the LONI Pipeline workflow environment.

Under the umbrella of the National Database for Clinical Trials (NDCT) related to mental illnesses, the National Database for Autism Research (NDAR) seeks to gather, curate, and make openly available neuroimaging data from NIH-funded studies of autism spectrum disorder (ASD). NDAR has recently made its database accessible through the LONI Pipeline workflow design and execution environment to enable large-scale analyses of cortical architecture and function via local, cluster, or "cloud"-based computing resources. This presents a unique opportunity to overcome many of the customary limitations to fostering biomedical neuroimaging as a science of discovery. Providing open access to primary neuroimaging data, workflow methods, and high-performance computing will increase uniformity in data collection protocols, encourage greater reliability of published data, results replication, and broaden the range of researchers now able to perform larger studies than ever before. To illustrate the use of NDAR and LONI Pipeline for performing several commonly performed neuroimaging processing steps and analyses, this paper presents example workflows useful for ASD neuroimaging researchers seeking to begin using this valuable combination of online data and computational resources. We discuss the utility of such database and workflow processing interactivity as a motivation for the sharing of additional primary data in ASD research and elsewhere.

A simple tool for neuroimaging data sharing.

Data sharing is becoming increasingly common, but despite encouragement and facilitation by funding agencies, journals, and some research efforts, most neuroimaging data acquired today is still not shared due to political, financial, social, and technical barriers to sharing data that remain. In particular, technical solutions are few for researchers that are not a part of larger efforts with dedicated sharing infrastructures, and social barriers such as the time commitment required to share can keep data from becoming publicly available. We present a system for sharing neuroimaging data, designed to be simple to use and to provide benefit to the data provider. The system consists of a server at the International Neuroinformatics Coordinating Facility (INCF) and user tools for uploading data to the server. The primary design principle for the user tools is ease of use: the user identifies a directory containing Digital Imaging and Communications in Medicine (DICOM) data, provides their INCF Portal authentication, and provides identifiers for the subject and imaging session. The user tool anonymizes the data and sends it to the server. The server then runs quality control routines on the data, and the data and the quality control reports are made public. The user retains control of the data and may change the sharing policy as they need. The result is that in a few minutes of the user's time, DICOM data can be anonymized and made publicly available, and an initial quality control assessment can be performed on the data. The system is currently functional, and user tools and access to the public image database are available at http://xnat.incf.org/.

Structure-centered portal for child psychiatry research.

The real world needs of the clinical community require a domain-specific solution to integrate disparate information available from various web-based resources for data, materials, and tools into routine clinical and clinical research setting. We present a child-psychiatry oriented portal as an effort to deliver a knowledge environment wrapper that provides organization and integration of multiple information and data sources. Organized semantically by resource context, the portal groups information sources by context type, and permits the user to interactively "narrow" or "broaden" the scope of the information resources that are available and relevant to the specific context. The overall objective of the portal is to bring information from multiple complex resources into a simple single uniform framework and present it to the user in a single window format.

Data sharing in neuroimaging research.

Significant resources around the world have been invested in neuroimaging studies of brain function and disease. Easier access to this large body of work should have profound impact on research in cognitive neuroscience and psychiatry, leading to advances in the diagnosis and treatment of psychiatric and neurological disease. A trend toward increased sharing of neuroimaging data has emerged in recent years. Nevertheless, a number of barriers continue to impede momentum. Many researchers and institutions remain uncertain about how to share data or lack the tools and expertise to participate in data sharing. The use of electronic data capture (EDC) methods for neuroimaging greatly simplifies the task of data collection and has the potential to help standardize many aspects of data sharing. We review here the motivations for sharing neuroimaging data, the current data sharing landscape, and the sociological or technical barriers that still need to be addressed. The INCF Task Force on Neuroimaging Datasharing, in conjunction with several collaborative groups around the world, has started work on several tools to ease and eventually automate the practice of data sharing. It is hoped that such tools will allow researchers to easily share raw, processed, and derived neuroimaging data, with appropriate metadata and provenance records, and will improve the reproducibility of neuroimaging studies. By providing seamless integration of data sharing and analysis tools within a commodity research environment, the Task Force seeks to identify and minimize barriers to data sharing in the field of neuroimaging.

The internet brain volume database: a public resource for storage and retrieval of volumetric data.

Every month, numerous publications appear that include neuroanatomic volumetric observations. The current and past literature that includes volumetric measurements is vast, but variable with respect to specific species, structures, and subject characteristics (such as gender, age, pathology, etc.). In this report we introduce the Internet Brain Volume Database (IBVD), www.nitrc.org/projects/ibvd , a site devoted to facilitating access to and utilization of neuroanatomic volumetric observations as published in the literature. We review the design and functionality of the site. The IBVD is the first database dedicated to integrating, exposing and sharing brain volumetric observations across species and disease. It offers valuable functionality for quality assurance assessment of results as well as support for meta-analysis across large segments of the published literature that are obscured from traditional text-based search engines.

WebParc: a tool for analysis of the topography and volume of stroke from MRI.

The quantitative assessment of the anatomic consequences of cerebral infarction is critical in the study of the etiology and therapeutic response in patients with stroke. We present here an overview of the operation of "WebParc," a computational system that provides measures of stroke lesion volume and location with respect to canonical forebrain neural systems nomenclature. Using a web-based interface, clinical imaging data can be registered to a template brain that contains a comprehensive set of anatomic structures. Upon delineation of the lesion, we can express the size and localization of the lesion in terms of the regions that are intersected within the template. We demonstrate the application of the system using MRI-based diffusion-weighted imaging and document measures of the validity and reliability of its uses. Intra- and inter-rater reliability is demonstrated, and characterized relative to the various classes of anatomic regions that can be assessed. The WebParc system has been developed to meet criteria of both efficiency and intuitive operator use in the real time analysis of stroke anatomy, so as to be useful in support of clinical care and clinical research studies. This article is an overview of its base-line operation with quantitative anatomic characterization of lesion size and location in terms of stroke distribution within the separate gray and white matter compartments of the brain.

The Neuroimaging Informatics Tools and Resources Clearinghouse (NITRC).

The Neuroimaging Informatics Tools and Resources Clearinghouse, NITRC is a newly established Web site for organizing knowledge about the resources publicly available functional MRI and related structural imaging analysis. Based on GForge, it provides a common environment for downloading, discussion, education, rating, and documentation for a growing number of resources. Its design and current status is presented.

Human cerebral cortex: a system for the integration of volume- and surface-based representations.

We describe an MRI-based system for topological analysis followed by measurements of topographic features for the human cerebral cortex that takes as its starting point volumetric segmentation data. This permits interoperation between volume-based and surface-based topographic analysis and extends the functionality of many existing segmentation schemes. We demonstrate the utility of these operations in individual as well as to group analysis. The methodology integrates analyses of cortical segmentation data generated by manual and semi-automated volumetric morphometry routines (such as the program cardviews) with the procedures of the FreeSurfer program to generate a cortical ribbon of the cerebrum and perform cortical topographic measurements (including thickness, surface area and curvature) in individual subjects as well as in subject populations. This system allows the computation of topographical cortical measurements for segmentation data generated from manual and semi-automated volumetric sources other than FreeSurfer. These measurements can be regionally specific and integrated with systems of cortical parcellation that subdivides the neocortex into gyral-based parcellation units (PUs). This system of topographical analysis of the cerebral cortex is consistent with current views of cortical development and neural systems organization of the human and non-human primate brain.

The integrated response of the human cerebro-cerebellar and limbic systems to acupuncture stimulation at ST 36 as evidenced by fMRI.

Clinical and experimental data indicate that most acupuncture clinical results are mediated by the central nervous system, but the specific effects of acupuncture on the human brain remain unclear. Even less is known about its effects on the cerebellum. This fMRI study demonstrated that manual acupuncture at ST 36 (Stomach 36, Zusanli), a main acupoint on the leg, modulated neural activity at multiple levels of the cerebro-cerebellar and limbic systems. The pattern of hemodynamic response depended on the psychophysical response to needle manipulation. Acupuncture stimulation typically elicited a composite of sensations termed deqi that is related to clinical efficacy according to traditional Chinese medicine. The limbic and paralimbic structures of cortical and subcortical regions in the telencephalon, diencephalon, brainstem and cerebellum demonstrated a concerted attenuation of signal intensity when the subjects experienced deqi. When deqi was mixed with sharp pain, the hemodynamic response was mixed, showing a predominance of signal increases instead. Tactile stimulation as control also elicited a predominance of signal increase in a subset of these regions. The study provides preliminary evidence for an integrated response of the human cerebro-cerebellar and limbic systems to acupuncture stimulation at ST 36 that correlates with the psychophysical response.