ABSTRACT
Introduction: We evaluate a CTSA program hub by applying bibliometrics, social network analysis (SNA), and altmetrics and examine the changes in research productivity, citation impact, research collaboration, and CTSA-supported research topics since our pilot study in 2017. Methods: The sampled data included North Carolina Translational and Clinical Science Institute (NC TraCS)-supported publications produced between September 2008 and March 2021. We applied measures and metrics from bibliometrics, SNA, and altmetrics to the dataset. In addition, we analyzed research topics and correlations between different metrics. Results: 1154 NC TraCS-supported publications generated over 53,560 citation counts by April 2021. The average cites per year and the relative citation ratio (RCR) mean of these publications improved from 33 and 2.26 in 2017 to 48 and 2.58 in 2021. The number of involved UNC units in the most published authors' collaboration network increased from 7 (2017) to 10 (2021). NC TraCS-supported co-authorship involved 61 NC organizations. PlumX metrics identified articles with the highest altmetrics scores. About 96% NC TraCS-supported publications have above the average SciVal Topic Prominence Percentile; the average approximate potential to translate of the included publication was 54.2%; and 177 publications addressed health disparity issues. Bibliometric measures (e.g., citation counts, RCR) and PlumX metrics (i.e., Citations, Captures, and Social-Media) are positively correlated (p < .05). Conclusion: Bibliometrics, SNA, and altmetrics offer distinctive but related perspectives to examine CTSA research performance and longitudinal growth, especially at the individual program hub level. These perspectives can help CTSAs build program foci.
ABSTRACT
INTRODUCTION: To enhance the performance evaluation of Clinical and Translational Science Award (CTSA) hubs, we examined the utility of advanced bibliometric measures that go beyond simple publication counts to demonstrate the impact of translational research output. METHODS: The sampled data included North Carolina Translational and Clinical Science Institute (NC TraCS)-supported publications produced between September 2008 and March 2017. We adopted advanced bibliometric measures and a state-of-the-art bibliometric network analysis tool to assess research productivity, citation impact, the scope of research collaboration, and the clusters of research topics. RESULTS: Totally, 754 NC TraCS-supported publications generated over 24,000 citation counts by April 2017 with an average of 33 cites per article. NC TraCS-supported research papers received more than twice as many cites per year as the average National Institute of Health-funded research publications from the same field and time. We identified the top productive researchers and their networks within the CTSA hub. Findings demonstrated the impact of NC TraCS in facilitating interdisciplinary collaborations within the CTSA hub and across the CTSA consortium and connecting researchers with right peers and organizations. CONCLUSION: Both improved bibliometrics measures and bibliometric network analysis can bring new perspectives to CTSA evaluation via citation influence and the scope of research collaborations.
ABSTRACT
INTRODUCTION: Pilot programs are integral to catalyzing and accelerating research at Clinical and Translational Science Award (CTSA) hubs. However, little has been published about the structure and operationalization of pilot programs or how they impact the translational research enterprise at CTSAs. The North Carolina Translational and Clinical Science Institute (NC TraCS), the CTSA hub at the University of North Carolina at Chapel Hill (UNC-CH) conducted an evaluation case study to describe the pilot program structure, assess process outcomes, and provide a framework for other institutions to utilize for the evaluation of their respective pilot programs. METHODS: We describe the operationalization of our pilot program, the evaluation framework utilized to evaluate the program, and how we analyzed available data to understand how our pilot funding opportunities were utilized by investigators. We calculated application volumes and funding rates by investigator position title and pilot application type. We also reviewed feedback provided by pilot Principal Investigators (PIs) to understand how many pilot projects were completed, NC TraCS service utilization, and barriers to research. Limited data on publications and subsequent funding was also reviewed. RESULTS: Between 2009 and 2019 the NC TraCS Pilot Program received 2343 applications and funded 933 projects, ranging from $2000 to $100,000 in amount, with an overall funding rate of 39.8%. Utilization of NC TraCS services had positive impacts on both resubmission funding and project completion rates. CONCLUSION: This process evaluation indicates that the program is being operationalized in a way that successfully fulfills the program mission while meeting the needs of a diverse group of researchers.
ABSTRACT
The 61 CTSA Consortium sites are home to valuable programs and infrastructure supporting translational science and all are charged with ensuring that such investments translate quickly to improved clinical care. Catalog of Assets for Translational and Clinical Health Research (CATCHR) is the Consortium's effort to collect and make available information on programs and resources to maximize efficiency and facilitate collaborations. By capturing information on a broad range of assets supporting the entire clinical and translational research spectrum, CATCHR aims to provide the necessary infrastructure and processes to establish and maintain an open-access, searchable database of consortium resources to support multisite clinical and translational research studies. Data are collected using rigorous, defined methods, with the resulting information made visible through an integrated, searchable Web-based tool. Additional easy-to-use Web tools assist resource owners in validating and updating resource information over time. In this paper, we discuss the design and scope of the project, data collection methods, current results, and future plans for development and sustainability. With increasing pressure on research programs to avoid redundancy, CATCHR aims to make available information on programs and core facilities to maximize efficient use of resources.
