Approaches to protocol standardization and data harmonization in the ECHO-wide cohort study.

The United States (U.S.) National Institutes of Health-funded Environmental influences on Child Health Outcomes (ECHO)-wide Cohort was established to conduct high impact, transdisciplinary science to improve child health and development. The cohort is a collaborative research design in which both extant and new data are contributed by over 57,000 children across 69 cohorts. In this review article, we focus on two key challenging issues in the ECHO-wide Cohort: data collection standardization and data harmonization. Data standardization using a Common Data Model and derived analytical variables based on a team science approach should facilitate timely analyses and reduce errors due to data misuse. However, given the complexity of collaborative research designs, such as the ECHO-wide Cohort, dedicated time is needed for harmonization and derivation of analytic variables. These activities need to be done methodically and with transparency to enhance research reproducibility. IMPACT: Many collaborative research studies require data harmonization either prior to analyses or in the analyses of compiled data. The Environmental influences on Child Health Outcomes (ECHO) Cohort pools extant data with new data collection from over 57,000 children in 69 cohorts to conduct high-impact, transdisciplinary science to improve child health and development, and to provide a national database and biorepository for use by the scientific community at-large. We describe the tools, systems, and approaches we employed to facilitate harmonized data for impactful analyses of child health outcomes.

Toward an Anti-Racist Approach to Biomedical and Neuroscience Research.

Racism is a threat to public health. Race is a sociopolitical construct that has been used for generations to create disparities in educational access, housing conditions, exposure to environmental contaminants, and access to health care. Collectively, these disparities have a negative impact on the health of non-white Americans. The National Institutes of Health (NIH) funds biomedical research, including basic neuroscience research, aimed at understanding the mechanisms and consequences of health and disease in Americans. NIH has recently acknowledged its own structural racism, the disadvantage this perpetuates in the biomedical research enterprise, and has announced its commitment to eliminating these disparities. Here, we discuss different rates of disease in U.S. citizens from different racial backgrounds. We next describe ways in which the biomedical research enterprise (1) has contributed to health disparities and (2) can contribute to the solving this problem. Based on our own scientific expertise, we use neuroscience in general and mental health/addiction disorders more specifically as examples of a broader issue. The NIH, including its neuroscience-focused Institutes, and NIH-funded scientists, including neuroscientists, should prioritize research topics that reflect the health conditions that affect all Americans, not just white Americans.

Predicting 90-Day Outcome After Thrombectomy: Baseline-Adjusted 24-Hour NIHSS Is More Powerful Than NIHSS Score Change.

BACKGROUND AND PURPOSE: The National Institutes of Health Stroke Scale (NIHSS) measured at an early time point is an appealing surrogate marker for long-term functional outcome of stroke patients treated with endovascular therapy. However, definitions and analytical methods for an early NIHSS-based outcome measure that optimize power and precision in clinical studies are not well-established. METHODS: In this post-hoc analysis of our prospective observational study that enrolled endovascular therapy-treated patients at 12 comprehensive stroke centers across the US, we compared the ability of 24-hour NIHSS, ΔNIHSS (baseline minus 24-hour NIHSS), and percentage change (NIHSS×100/baseline NIHSS), analyzed as continuous and dichotomous measures, to predict 90-day modified Rankin Scale (mRS) using logistic regression (adjusted for age, baseline NIHSS, glucose, hypertension, Alberta Stroke Program Early CT Score, time to recanalization, recanalization status, and intravenous thrombolysis) and Spearman ρ. RESULTS: Of 485 patients in the BEST (Blood Pressure After Endovascular Stroke Therapy) cohort, 446 (92%) with 90-day follow-up data were included. An absolute 24-hour NIHSS, adjusted for baseline in multivariable modeling, had the highest predictive power of all definitions evaluated (aR2 0.368 and adjusted odds ratio 0.79 [0.75-0.84], P<0.001 for mRS score 0-2; aR2 0.444 and adjusted odds ratio 0.84 [0.8-0.86] for ordinal mRS). For predicting mRS score of 0-2 with a cut point, the second most efficient approach, the optimal threshold for 24-hour NIHSS score was ≤7 (sensitivity 80.1%, specificity 80.4%; adjusted odds ratio 12.5 [7.14-20], P<0.001), followed by percent change in NIHSS (sensitivity 79%, specificity 58.5%; adjusted odds ratio 4.55 [2.85-7.69], P<0.001). CONCLUSIONS: Twenty-four-hour NIHSS, adjusted for baseline, was the strongest predictor of both dichotomous and ordinal 90-day mRS outcomes for endovascular therapy-treated patients. A dichotomous 24-hour NIHSS score of ≤7 was the second-best predictor. Although ΔNIHSS, continuous and dichotomized at ≥4, predicted 90-day outcomes, absolute 24-hour NIHSS definitions performed better.

Accuracy of NIH Stroke Scale for diagnosing aphasia.

OBJECTIVES: The National Institutes of Health Stroke Scale (NIHSS) has not been validated to diagnose aphasia in the stroke population. We therefore examined the diagnostic accuracy of NIHSS for detecting aphasia in acute ischemic stroke. METHODS: Consecutive patients with acute first-ever ischemic stroke were included prospectively in Lund Stroke Register Study at Skåne University Hospital, Sweden. Exclusion criteria were: (a) non-native Swedish; (b) obtundation (c) dementia or psychiatric diagnosis. Patients were assessed with NIHSS item 9 (range 0-3, where 1-3 indicate aphasia) by a NIHSS certified research nurse in the acute phase after stroke onset (median 3 days). Within 24 h after this assessment, a speech therapist evaluated the patients' language function with the comprehensive language screening test (LAST, range 0-15 where 0-14 indicates aphasia). Data were analyzed using LAST as 'reference standard'. RESULTS: We examined 221 patients. Among these, 23% (n = 50) had aphasia according to NIHSS (distribution of scores 0, 1, 2, 3 were n = 171, n = 29, n = 12, n = 9) compared to 26% (n = 58) with aphasia according to LAST (score ≤14; median = 11). Assuming LAST as reference standard, NIHSS gave 16 false negatives (NIHSS item 9 = 0) for aphasia (LAST scores range 8-14), and 8 false positives (NIHSS item 9 score = 1) for aphasia, yielding a sensitivity of 72% (0.59-0.83) and a specificity of 95% (0.91-0.98). CONCLUSIONS: When using NIHSS for screening and diagnosing aphasia in adults with acute ischemic stroke, patients with severe aphasia can be detected, however, some mild aphasias might be misclassified. Given the 72% sensitivity, absence of aphasia on the NIHSS should not be used to guide stroke treatment.

NIH's scientific approach to inclusive excellence.

The representation of women and scientists from underrepresented groups (URGs), including Black/African Americans, Hispanic/Latinx, Pacific Islanders, and American Indians, diminishes as individuals advance in their careers from training to senior leadership positions. Correcting this imbalance requires integrated strategies to achieve inclusive excellence within the scientific workforce reflected by creating and sustaining environments, in which diverse talent thrives. The National Institutes of Health (NIH) Scientific Workforce Diversity office has led the charge to develop and implement evidence-informed interventions toward achieving this goal that undergirds NIH's mission to improve the nation's health. Past and current efforts aiming to enhance workforce diversity but targeted to individuals are necessary but insufficient for lasting change. Thus, NIH-funded institutions should develop and prioritize integrated, systems-targeted efforts as foundational components of a well-supported, productive workforce. At the heart of these endeavors is institutional accountability that ties progress toward inclusive excellence to institutional values and reward systems.

Contributors to Independent Research Funding Success from the Perspective of K12 BIRCWH Program Directors.

INTRODUCTION: Faculty training awards are an important means of advancing early career faculty in research. The National Institutes of Health (NIH) Building Interdisciplinary Research Careers in Women's Health (BIRCWH) is a long-running K12 career development program and has been integral in promoting the research success of faculty nationally. We surveyed BIRCWH program directors to understand factors likely to influence long-term research careers and funding success. MATERIALS AND METHODS: We developed an online survey containing open-ended questions about individual and programmatic attributes and activities that promote success in achieving independent research funding. Domains of interest included: 1) strategies for funding success; 2) traits for predicting success; 3) groups considered vulnerable to attrition; and 4) existing resources and means of support. RESULTS: Fifteen institutions (75%) were included in the final analysis. Passion for research, persistence, resilience, and strong mentorship relationships were identified by all directors as factors important to scholar success. Responses also revealed an important pattern: program directors attributed attrition either to individual or organizational characteristics. This distinction has meaningful consequences for framing efforts to diminish attrition. Faculty who were clinicians, women, parents and underrepresented minorities were identified as vulnerable to attrition from the research careers. Common perceived challenges in these groups included isolation/feeling alienated, juggling numerous priorities, inadequate research time, lack of role models, and work-life balance issues. CONCLUSION: K12 BIRCWH directors identified persistence and resilience and developing community, networks, and other support opportunities as elements of scholar success. Programs and mentors can help early career faculty by teaching skills and providing tools they can use to maximize the value of these opportunities and expand their mentees' research relationships. Our study also highlights the importance of social factors, particularly isolation, on clinicians, women, and minoritized scholars on career success.

Diet and Risk of Myeloproliferative Neoplasms in Older Individuals from the NIH-AARP Cohort.

BACKGROUND: The etiology of myeloproliferative neoplasms (MPN) is obscure, and no previous studies have evaluated the role of diet. METHODS: In the NIH-AARP Diet and Health Study, a prospective cohort of 463,049 participants ages 50 to 71 years at baseline (1995-1996), we identified 490 MPN cases after a median follow-up of 15.5 years, including 190 with polycythemia vera (PV) and 146 with essential thrombocythemia (ET). We examined possible associations between various dietary factors and the risk of MPN as a group, as well as PV and ET, using multivariable Cox proportional hazards models to estimate hazard ratios (HR) and 95% confidence intervals (CI) and adjust for potential confounding variables. RESULTS: An increased risk was observed between fruit consumption and the risk of MPN overall (third tertile vs. first tertile, HR = 1.32; 95% CI, 1.04-1.67; P trend = 0.02) and PV (third tertile vs. first tertile, HR = 2.00; 95% CI, 1.35-2.95; P trend < 0.01). Increased risk of PV was also observed among those with high intake of sugar (HR = 1.77; 95% CI, 1.12-2.79), sugar from natural sources (HR = 1.77; 95% CI, 1.16-2.71), sugar from natural beverage sources (HR = 1.57; 95% CI, 1.08-2.29), and fructose (HR = 1.84; 95% CI, 1.21-2.79). CONCLUSIONS: The intake of fat and protein did not appear to influence PV risk-neither did meat or vegetable consumption. None of the dietary factors studied was associated with the risk of ET. The role of sugar intake in the etiology of PV in older individuals warrants further investigation. IMPACT: Our results indicate that high sugar intake is associated with an increased risk of polycythemia vera.

Neurocognitive performance in alcohol use disorder using the NIH toolbox: Role of severity and sex differences.

AIMS: Sustained heavy alcohol consumption is associated with a range of neurocognitive deficits. Yet, past research centers on a severe profile of alcohol use disorder (AUD), with persons recruited from in-patient settings. The current project aims to compare neurocognitive performance between individuals seeking AUD outpatient treatment with healthy comparisons while considering the association between performance, disorder severity, and sex. METHODS: Enrollment included two matched groups (N = 125; 34 % female): 77 treatment-seeking individuals with AUD; 48 healthy comparison individuals with low drinking patterns. Neurocognitive performance on NIH Toolbox subtests measuring attention, inhibition, episodic memory, working memory, language, and processing speed were compared across groups. Within the AUD group, analyses examined the relationship between performance, disorder severity, recent alcohol consumption, and sex. RESULTS: AUD group did not perform significantly lower than healthy comparisons on neurocognition subtests assessed. Within AUD group, females displayed significantly higher processing speeds than males (p = .007). Disorder severity and alcohol consumption were not significantly related to performance. However, a significant interaction between disorder severity and sex emerged (p = .010), with higher severity associated with poorer performance in males but not females, on a subtest measuring attention and inhibition. CONCLUSIONS: Effect of heavy alcohol use on neurocognitive performance was not detected in this outpatient AUD sample. Weaknesses in domains of attention and inhibition may be correlated with AUD severity among males, but not females. Further research on AUD severity and sex in understanding individual differences in neurocognition is warranted, particularly using novel tools for large scale phenotyping, such as the NIH Toolbox.

Medical Mobile App Classification Using the National Institute for Health and Care Excellence Evidence Standards Framework for Digital Health Technologies: Interrater Reliability Study.

BACKGROUND: Clinical governance of medical mobile apps is challenging, and there is currently no standard method for assessing the quality of such apps. In 2018, the National Institute for Health and Care Excellence (NICE) developed a framework for assessing the required level of evidence for digital health technologies (DHTs), as determined by their clinical function. The framework can potentially be used to assess mobile apps, which are a subset of DHTs. To be used reliably in this context, the framework must allow unambiguous classification of an app's clinical function. OBJECTIVE: The objective of this study was to determine whether mobile health apps could be reliably classified using the NICE evidence standards framework for DHTs. METHODS: We manually extracted app titles, screenshots, and content descriptions for all apps listed on the National Health Service (NHS) Apps Library website on July 12, 2019; none of the apps were downloaded. Using this information, 2 mobile health (mHealth) researchers independently classified each app to one of the 4 functional tiers (ie, 1, 2, 3a, and 3b) described in the NICE digital technologies evaluation framework. Coders also answered contextual questions from the framework to identify whether apps were deemed to be higher risk. Agreement between coders was assessed using Cohen κ statistic. RESULTS: In total, we assessed 76 apps from the NHS Apps Library. There was classification agreement for 42 apps. Of these, 0 apps were unanimously classified into Tier 1; 24, into Tier 2; 15, into Tier 3a; and 3, into Tier 3b. There was disagreement between coders in 34/76 cases (45%); interrater agreement was poor (Cohen κ=0.32, 95% CI 0.16-0.47). Further investigation of disagreements highlighted 5 main explanatory themes: apps that did not correspond to any tier, apps that corresponded to multiple tiers, ambiguous tier descriptions, ambiguous app descriptions, and coder error. CONCLUSIONS: The current iteration of the NICE evidence standards framework for DHTs did not allow mHealth researchers to consistently and unambiguously classify digital health mobile apps listed on the NHS app library according to their functional tier.

When IRBs Say No to Participating in Research about Single IRBs.

In response to a policy of the National Institutes of Health and requirements in the revised Common Rule, a protocol for a multisite study must be reviewed by a single institutional review board (IRB), rather than by the IRB at each study site. The goal of the single IRB approach is to increase the efficiency of IRB review of multisite research without jeopardizing protections for research subjects. Yet the extent to which these joint goals are being achieved is unclear. To better understand how single IRBs function, we recruited academic, government, and commercial single IRBs (N = 49) to participate in a study involving observation of protocol review meetings and/or interviews with their members, chairs, and administrators. Twenty (40.8%) agreed to participate, of which 50% agreed to both interviews and observation. While 81.8% (9/11) of academic and 50% (4/8) of government single IRBs participated in some way, only 23.3% (7/30) of commercial single IRBs did so. The four largest commercial single IRBs declined to participate. Because evaluation of single IRBs is important to inform development, implementation, monitoring, and refinement of federal policies, single IRBs should be encouraged to participate in research that examines how they function.

Belmont in Context.

Given its outsized influence as a core document in bioethics, it is worth reminding ourselves of the historical context in which the Belmont Report came to be. This article examines the societal forces that helped bring about the Belmont Report and that shaped its conception of ethical research. A product of a public investigation that included many nonscientists and espoused philosophical principles, the Report internalized a growing call in the late 1960s for oversight over the research enterprise, which had long been the private realm of physician-investigators. Belmont helped bring about a regulatory and oversight apparatus to the research enterprise, as well as a language and discipline of bioethics that added a multidisciplinary set of voices and decision-makers to discussions of what constitutes ethical research. Because it reflected the spirit of protectionism engendered by events of the 1960s and 1970s, Belmont also helped emphasize the importance of informed consent and the protection of vulnerable populations. But because the Report was a product of its time, contingent on historical developments and highly publicized events, it is not necessarily responsive to new factors that now condition the research enterprise.

The Origins and Drafting of the Belmont Report.

This article describes the origins and drafting of the Belmont Report by members of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research.

The NIH Minimal Dataset for Chronic Low Back Pain: Responsiveness and Minimal Clinically Important Change.

STUDY DESIGN: Prospective cohort study. OBJECTIVE: To analyze responsiveness and minimal clinically important change (MCIC) of the US National Institutes of Health (NIH) minimal dataset for chronic low back pain (CLBP). SUMMARY OF BACKGROUND DATA: The NIH minimal dataset is a 40-item questionnaire developed to increase use of standardized definitions and measures for CLBP. Longitudinal validity of the total minimal dataset and the subscale Impact Stratification are unknown. METHODS: Total outcome scores on the NIH minimal dataset, Dutch Language Version, were calculated ranging from 0 to 100 points with higher scores representing worse functioning. Responsiveness and MCIC were determined with an anchor-based method, calculating the area under the receiver operating characteristics (ROC) curve (AUC) and by determining the optimal cut-off point. Smallest detectable change (SDC) was calculated as a parameter of measurement error. RESULTS: In total 223 patients with CLBP were included. Mean total score on the NIH minimal dataset was 44 ±â€Š14 points at baseline. The total outcome score was responsive to change with an AUC of 0.84. MCIC was 14 points with a sensitivity of 72% and specificity 82%, and SDC was 23 points. Mean total score on Impact Stratification (scale 8-50) was 34.4 ±â€Š7.4 points at baseline, with an AUC of 0.91, an MCIC of 7.5 with a sensitivity 96% of and specificity of 78%, and an SDC of 14 points. CONCLUSION: The longitudinal validity of the NIH minimal dataset is adequate. An improvement of 14 points in total outcome score and 7.5 points in Impact Stratification can be interpreted as clinically important in individual patients. However, MCIC depends on baseline values and the method that is chosen to determine the optimal cut-off point. Furthermore, measurement error is larger than the MCIC. This means that individual change scores should be interpreted with caution. LEVEL OF EVIDENCE: 2.

Transitioning to the National Institutes of Health single institutional review board model: Piloting the use of the Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance.

BACKGROUND/AIMS: Obtaining ethical approval from multiple institutional review boards is a long-standing challenge to multi-site clinical trials and often leads to significant delays in study activation and enrollment. As of 25 January 2018, the National Institutes of Health began requiring use of a single institutional review board for US multi-site trials. To learn more and further inform the research and regulatory communities around aspects of transitioning to single institutional review board review, this study evaluated the efficiency, resource use, and user perceptions of a nascent institutional review board reliance model (Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance). METHODS: This research was embedded within the Influenza Vaccine to Effectively Stop Cardio Thoracic Events and Decompensated Heart Failure trial-a multi-site trial of two influenza vaccine formulations. In the first year of the trial, a sample of sites agreed to use the developing Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance model and participated in its evaluation. In keeping with a least burdensome approach, short surveys were developed and obtained from each reporting entity (relying sites, non-relying site, lead site, and reviewing institutional review board). Data regarding time to institutional review board approval and site activation, costs, and user perceptions of reliant review were self-reported and collected via the survey form. Quantitative and qualitative analyses were performed, with costs analyzed as actual versus estimated due to the lack of established baseline cost data. RESULTS: A total of 13 sites ceded review and received institutional review board approval. Mean time to approval was substantially faster in sites that ceded review using the Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance model versus the site that did not cede review (81 vs 121 days). The mean time to approval was also faster than published averages for academic medical centers (81 vs 103 days). Time to first enrollment was faster for ceding sites versus the non-ceding site, and also faster than published averages (126 vs 149 and 169 days, respectively). Costs were higher than estimates for local institutional review board review and approval. Nearly half (47%) the stakeholders reported being very satisfied or satisfied with the reliance experience, although many noted the challenge related to institutional culture change. CONCLUSION: Implementation of a single institutional review board represents a shift in practice and culture for many institutions. Evaluation of the reliance arrangements for this study highlights both the potential of, and challenges for, institutions as they transition to single institutional review board review. Although efficiencies were observed for study start-up, we anticipate a learning curve as institutions and research teams implement necessary process and resource changes to adapt to single institutional review board oversight. Findings may inform research teams but are, however, limited by the relatively small number of sites and lack of a control group.

The selection of comparators for randomized controlled trials of health-related behavioral interventions: recommendations of an NIH expert panel.

OBJECTIVES: To provide recommendations for the selection of comparators for randomized controlled trials of health-related behavioral interventions. STUDY DESIGN AND SETTING: The National Institutes of Health Office of Behavioral and Social Science Research convened an expert panel to critically review the literature on control or comparison groups for behavioral trials and to develop strategies for improving comparator choices and for resolving controversies and disagreements about comparators. RESULTS: The panel developed a Pragmatic Model for Comparator Selection in Health-Related Behavioral Trials. The model indicates that the optimal comparator is the one that best serves the primary purpose of the trial but that the optimal comparator's limitations and barriers to its use must also be taken into account. CONCLUSION: We developed best practice recommendations for the selection of comparators for health-related behavioral trials. Use of the Pragmatic Model for Comparator Selection in Health-Related Behavioral Trials can improve the comparator selection process and help resolve disagreements about comparator choices.

The NIH BRAIN Initiative: Integrating Neuroethics and Neuroscience.

The NIH Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative is focused on developing new tools and neurotechnologies to transform our understanding of the brain, and neuroethics is an essential component of this research effort. Coordination with other brain projects around the world will help maximize success.

Validation of a Novel Telehealth Administration Protocol for the NIH Toolbox-Cognition Battery.

BACKGROUND: Cognition is an important outcome in many clinical trials. The NIH Toolbox-Cognition Battery (NIHTB-CB) is a computerized cognitive assessment designed for clinical research that is administered in-person. Here, we evaluated the equivalency of a novel videoconference protocol for administering the NIHTB-CB. Since our protocol required repeated assessments, we further explored the NIHTB-CB's practice effect. MATERIALS AND METHODS: Twenty-five healthy participants completed the NIHTB-CB under two separate conditions 4 weeks apart. The standard condition followed the recommended administration protocol, whereas the videoconference condition had the examiner and participant in separate rooms but able to communicate over videoconference. A linear mixed-model analysis was performed to explore the fixed effect of testing condition and time on NIHTB-CB performance. RESULTS: Across all three NIHTB-CB composite scores (total, fluid, and crystallized cognition), no significant fixed effect of administration condition was found. A significant practice effect was observed for the fluid and total cognition composite scores over a 29.0 (±2.1) day test-retest interval. CONCLUSIONS: Our novel videoconference protocol for the NIHTB-CB is equivalent to the standard protocol in healthy participants, and may provide a solution for researchers seeking to engage study participants at remote sites. If the NIHTB-CB is used longitudinally to monitor patients, corrections for repeated measures may be required.