The Health Equity Leadership Institute (HELI): Developing workforce capacity for health disparities research.

INTRODUCTION: Efforts to address health disparities and achieve health equity are critically dependent on the development of a diverse research workforce. However, many researchers from underrepresented backgrounds face challenges in advancing their careers, securing independent funding, and finding the mentorship needed to expand their research. METHODS: Faculty from the University of Maryland at College Park and the University of Wisconsin-Madison developed and evaluated an intensive week-long research and career-development institute-the Health Equity Leadership Institute (HELI)-with the goal of increasing the number of underrepresented scholars who can sustain their ongoing commitment to health equity research. RESULTS: In 2010-2016, HELI brought 145 diverse scholars (78% from an underrepresented background; 81% female) together to engage with each other and learn from supportive faculty. Overall, scholar feedback was highly positive on all survey items, with average agreement ratings of 4.45-4.84 based on a 5-point Likert scale. Eighty-five percent of scholars remain in academic positions. In the first three cohorts, 73% of HELI participants have been promoted and 23% have secured independent federal funding. CONCLUSIONS: HELI includes an evidence-based curriculum to develop a diverse workforce for health equity research. For those institutions interested in implementing such an institute to develop and support underrepresented early stage investigators, a resource toolbox is provided.

Using publication data to evaluate a Clinical and Translational Science Award (CTSA) career development program: Early outcomes from KL2 scholars.

INTRODUCTION: This study uses KL2 scholars' publications to evaluate the types of research the KL2 program supports and to assess the initial productivity and impact of its scholars. METHODS: We illustrate the feasibility of 3 different approaches to bibliometrics, one viable method for determining the types of research a program or hub supports, and demonstrate how these data can be further combined with internal data records. RESULTS: Gender differences were observed in the types of research scholars undertake. Overall KL2 scholars are performing well, with their publications being cited more than the norm for National Institutes of Health publications. Favorable results were also observed in scholars' continued engagement in research. CONCLUSION: This study illustrates that linking bibliometric data and data categorizing publications along the translational spectrum with a Clinical and Translational Science Award hub's internal data records is feasible and offers a number of innovative possibilities for the evaluation of a Clinical and Translational Science Award hub's programs and investigators.

Classifying publications from the clinical and translational science award program along the translational research spectrum: a machine learning approach.

BACKGROUND: Translational research is a key area of focus of the National Institutes of Health (NIH), as demonstrated by the substantial investment in the Clinical and Translational Science Award (CTSA) program. The goal of the CTSA program is to accelerate the translation of discoveries from the bench to the bedside and into communities. Different classification systems have been used to capture the spectrum of basic to clinical to population health research, with substantial differences in the number of categories and their definitions. Evaluation of the effectiveness of the CTSA program and of translational research in general is hampered by the lack of rigor in these definitions and their application. This study adds rigor to the classification process by creating a checklist to evaluate publications across the translational spectrum and operationalizes these classifications by building machine learning-based text classifiers to categorize these publications. METHODS: Based on collaboratively developed definitions, we created a detailed checklist for categories along the translational spectrum from T0 to T4. We applied the checklist to CTSA-linked publications to construct a set of coded publications for use in training machine learning-based text classifiers to classify publications within these categories. The training sets combined T1/T2 and T3/T4 categories due to low frequency of these publication types compared to the frequency of T0 publications. We then compared classifier performance across different algorithms and feature sets and applied the classifiers to all publications in PubMed indexed to CTSA grants. To validate the algorithm, we manually classified the articles with the top 100 scores from each classifier. RESULTS: The definitions and checklist facilitated classification and resulted in good inter-rater reliability for coding publications for the training set. Very good performance was achieved for the classifiers as represented by the area under the receiver operating curves (AUC), with an AUC of 0.94 for the T0 classifier, 0.84 for T1/T2, and 0.92 for T3/T4. CONCLUSIONS: The combination of definitions agreed upon by five CTSA hubs, a checklist that facilitates more uniform definition interpretation, and algorithms that perform well in classifying publications along the translational spectrum provide a basis for establishing and applying uniform definitions of translational research categories. The classification algorithms allow publication analyses that would not be feasible with manual classification, such as assessing the distribution and trends of publications across the CTSA network and comparing the categories of publications and their citations to assess knowledge transfer across the translational research spectrum.

Success case studies contribute to evaluation of complex research infrastructure.

The success case studies approach examines in depth what works well in a program by describing cases and examining factors leading to successful outcomes. In this article, we describe the use of success case studies as part of an evaluation of the transformation of a health sciences research support infrastructure. Using project-specific descriptions and the researchers' perceptions of the impact of improved research infrastructure, we added depth of understanding to the quantitative data required by funding agencies. Each case study included an interview with the lead researcher, along with review of documents about the research, the investigator, and their collaborators. Our analyses elucidated themes regarding contributions of the Clinical and Translational Science Awards program of the National Institutes of Health to scientific achievements and career advancement of investigators in one academic institution.

Heterogeneity at work: implications of the 2012 Clinical Translational Science Award evaluators survey.

The Clinical and Translational Science Award (CTSA) program is an ambitious multibillion dollar initiative sponsored by the National Institutes of Health (NIH) organized around the mission of facilitating the improved quality, efficiency, and effectiveness of translational health sciences research across the country. Although the NIH explicitly requires internal evaluation, funded CTSA institutions are given wide latitude to choose the structure and methods for evaluating their local CTSA program. The National Evaluators Survey was developed by a peer-led group of local CTSA evaluators as a voluntary effort to understand emerging differences and commonalities in evaluation teams and techniques across the 61 CTSA institutions funded nationwide. This article presents the results of the 2012 National Evaluators Survey, finding significant heterogeneity in evaluation staffing, organization, and methods across the 58 CTSAs institutions responding. The variety reflected in these findings represents both a liability and strength. A lack of standardization may impair the ability to make use of common metrics, but variation is also a successful evolutionary response to complexity. Additionally, the peer-led approach and simple design demonstrated by the questionnaire itself has value as an example of an evaluation technique with potential for replication in other areas across the CTSA institutions or any large-scale investment where multiple related teams across a wide geographic area are given the latitude to develop specialized approaches to fulfilling a common mission.

Uganda's HIV prevention success: the role of sexual behavior change and the national response.

There has been considerable interest in understanding what may have led to Uganda's dramatic decline in HIV prevalence, one of the world's earliest and most compelling AIDS prevention successes. Survey and other data suggest that a decline in multi-partner sexual behavior is the behavioral change most likely associated with HIV decline. It appears that behavior change programs, particularly involving extensive promotion of "zero grazing" (faithfulness and partner reduction), largely developed by the Ugandan government and local NGOs including faith-based, women's, people-living-with-AIDS and other community-based groups, contributed to the early declines in casual/multiple sexual partnerships and HIV incidence and, along with other factors including condom use, to the subsequent sharp decline in HIV prevalence. Yet the debate over "what happened in Uganda" continues, often involving divisive abstinence-versus-condoms rhetoric, which appears more related to the culture wars in the USA than to African social reality.