Measuring the impact of scientific publications and publication extenders: examples of novel approaches.

Different stakeholders, such as authors, research institutions, and healthcare professionals (HCPs) may determine the impact of peer-reviewed publications in different ways. Commonly-used measures of research impact, such as the Journal Impact Factor or the H-index, are not designed to evaluate the impact of individual articles. They are heavily dependent on citations, and therefore only measure impact of the overall journal or researcher respectively, taking months or years to accrue. The past decade has seen the development of article-level metrics (ALMs), that measure the online attention received by an individual publication in contexts including social media platforms, news media, citation activity, and policy and patent citations. These new tools can complement traditional bibliometric data and provide a more holistic evaluation of the impact of a publication. This commentary discusses the need for ALMs, and summarizes several examples - PlumX Metrics, Altmetric, the Better Article Metrics score, the EMPIRE Index, and scite. We also discuss how metrics may be used to evaluate the value of "publication extenders" - educational microcontent such as animations, videos and plain-language summaries that are often hosted on HCP education platforms. Publication extenders adapt a publication's key data to audience needs and thereby extend a publication's reach. These new approaches have the potential to address the limitations of traditional metrics, but the diversity of new metrics requires that users have a keen understanding of which forms of impact are relevant to a specific publication and select and monitor ALMs accordingly.

Different readers have different ways of deciding how important scientific articles are. The usual methods used to measure the impact of research, like the Journal Impact Factor or the H-index, are not meant to measure this for individual articles. These methods mainly look at how many times the articles are mentioned by others, and it can take a long time to see the impact.But in the past ten years, new tools called article-level metrics (ALMs) have been created. These tools measure how much attention an article gets online, like on social media, in the news, or when other researchers talk about it. ALMs are better at explaining how important a specific article is. They can work together with the usual methods to measure impact.This paper talks about why ALMs are important and gives examples of these tools, like PlumX Metrics, Altmetric, the Better Article Metrics score, the EMPIRE Index, and scite. It also explains how these tools can help us see the value of animations, videos, or summaries in simple language. These make it easier for more people to understand and learn from the articles.These new ways of measuring impact can help us see how important articles are in a more complete way. But because there are many different ways to measure this, it's important for users to understand which methods are relevant for a specific article and keep track of them.

Publication outcomes of abstracts presented at the Argentine Congress of Rheumatology / Tasa de publicación de los resúmenes presentados al Congreso Argentino de Reumatología

Objective: To analyse the outcome of scientific abstracts submitted to the Argentine Congress of Rheumatology (ACOR) in 2000, 2005, 2010, and 2015. Methods: Every abstract submitted to the ACOR was analysed. The number of these manuscripts published was determined through Google Scholar and PubMed searches. The impact of the scientific journals was established through the SCImago Journal (SJR) indicator. Results: Considering the 727 abstracts evaluated, 10.2% of the articles were found in journals indexed by Google Scholar, and 6.6% in PubMed: 4.7% were published in 2000, 9.4% in 2005, 14.6% in 2010, and 11.9% in 2015 (Log Rank test 0.008), with a statistically significant increase between 2010 and 2015 compared to 2000 (HR 3.3; 95% CI 1.5–7; p 0.002 and HR 2.9; CI 1.4–6.3; p 0.005 respectively). The median SJR of the journals was 0.46 and 67.6% had SJR available. Conclusions: The publication rate was low, and only a few articles were published in the most prestigious journals within the speciality.(AU)

Objetivo: Analizar la tasa de publicación de los resúmenes presentados al Congreso Argentino de Reumatología (ACOR) en 2000, 2005, 2010 y 2015. Métodos: Todos los resúmenes enviados al ACOR fueron evaluados. Se determinó la tasa de publicación mediante una búsqueda en Google Scholar y PubMed. Se examinó la relevancia de las revistas científicas a través del indicador SCImago Journal (SJR). Resultados: Se evaluaron 727 resúmenes. Se encontró un 10,2% de artículos publicados en revistas indexadas por Google Scholar y un 6,6% en PubMed. El 4,7% fueron publicados en 2000, el 9,4% en 2005, el 14,6% en 2010 y el 11,9% en 2015 (Log Rank test: 0,008), con un aumento estadísticamente significativo entre 2010 y 2015 frente al 2000 (HR: 3,3; IC95%: 1,5-7; p=0,002 y HR: 2,9; IC95%: 1,4-6,3; p=0,005, respectivamente). La mediana del SJR de dichas revistas fue de 0,46, y el 67,6% tenían SJR disponible. Conclusiones: La tasa de publicación es baja, y solo unos pocos trabajos fueron publicados en las revistas más prestigiosas de la especialidad.(AU)

Publication outcomes of abstracts presented at the Argentine Congress of Rheumatology.

OBJECTIVE: To analyse the outcome of scientific abstracts submitted to the Argentine Congress of Rheumatology (ACOR) in 2000, 2005, 2010, and 2015. METHODS: Every abstract submitted to the ACOR was analysed. The number of these manuscripts published was determined through Google Scholar and PubMed searches. The impact of the scientific journals was established through the SCImago Journal (SJR) indicator. RESULTS: Considering the 727 abstracts evaluated, 10.2% of the articles were found in journals indexed by Google Scholar, and 6.6% in PubMed: 4.7% were published in 2000, 9.4% in 2005, 14.6% in 2010, and 11.9% in 2015 (Log Rank test 0.008), with a statistically significant increase between 2010 and 2015 compared to 2000 (HR 3.3; 95% CI 1.5-7; p 0.002 and HR 2.9; CI 1.4-6.3; p 0.005 respectively). The median SJR of the journals was 0.46 and 67.6% had SJR available. CONCLUSIONS: The publication rate was low, and only a few articles were published in the most prestigious journals within the speciality.

Recent Changes in Characteristics of Applicants and Matriculants to Thoracic Surgery Fellowships.

The match rate for traditional thoracic surgery fellowships decreased from 97.5% in 2012 to 59.1% in 2021, reflecting an increase in applications. We queried whether characteristics of applicants and matriculants to traditional thoracic surgery fellowships changed during this time period. Applicant data from the 2008 through 2018 application cycles were extracted from the Electronic Residency Application System (ERAS) and Graduate Medical Education (GME) Track Resident Survey and stratified by period of application (2008-2014 vs 2015-2018). Characteristics of applicants and matriculants were analyzed. There were 697 applicant records in the early period and 530 in the recent period (application rate 99.6/year vs 132.5/year; P = 0.0005), and 607 matriculant records in the early period and 383 in the recent period (matriculation rate 87% vs 72%; P < 0.0001). There was no difference in representation of university-affiliated versus community-based general surgery residency programs among applicants comparing the periods. Higher proportions of applicants and matriculants in the early period trained in general surgery programs affiliated with a comprehensive cancer center or a thoracic surgery fellowship. Applicants and matriculants of the recent period had higher median numbers of journal publications and had higher impact factor journal publications. The increase in applicants for thoracic surgery training is primarily from general surgery trainees in residency programs not affiliated with a comprehensive cancer center or a thoracic surgery fellowship. The increased interest in thoracic surgery training was accompanied by overall enhanced scholarship production among the applicants and matriculants regardless of their residency characteristics.

The publication rates of abstracts presented at the Turkish Respiratory Society (TUSAD) annual congress: adequate or not?

OBJECTIVE: In this study, we aimed to determine the publication status of the abstracts presented at the Turkish Respiratory Society Congress between 2011 and 2014. METHODS: The abstracts were classified according to presentation type (oral presentations, poster discussion, etc.), study type, study design, topic, publication status, time interval between the presentation and the publication date, and the journal in which the article was published. The conversion rate of presentations into full-text articles in peer-reviewed journals were surveyed through Elsevier's Scopus. RESULTS: The total number of abstracts submitted in the congress was 2 009. In terms of study type, the majority of abstracts were case reports (56.4%) and the remainder was original research. Totally, 179 abstracts were published in an indexed journal with an overall publication rate of 8.9%. 18.3% of oral presentations were converted into full-text article. Publication rates according to study types were 14.8% for original researches and 4.4% for case reports. The first three subspecialties with the highest publication rates were "sleep related breathing disorders" (16.9%), "interventional pulmonology" (16.7%) and "pleural diseases" (15.2%). Median publication/acceptance time was 8.0 months (0-38). CONCLUSION: This is the first study evaluating the publication rates of abstracts presented in a respiratory congress. Although the number of presented abstracts in the congress increased year by year, prominently in the case reports, over all conversion rates were decreased. We put forward that encouraging the authors to conduct higher-quality investigations would raise the publication rate as well as improve the scientific quality of congress.

Conversion of Society for Maternal-Fetal Medicine abstract presentations to manuscript publications.

OBJECTIVE: We sought to evaluate the rate of conversion of Society for Maternal-Fetal Medicine (SMFM) annual meeting abstract presentations to full manuscript publications over time. STUDY DESIGN: Full manuscript publications corresponding to all SMFM oral abstracts 2003 through 2010 inclusive, and SMFM poster abstracts in 2003, 2005, 2007, and 2009 were manually searched in PubMed. An abstract was considered to "match" a full publication if the abstract and publication titles as well as main methods and results were similar and the abstract first author was a publication author. In cases of uncertainty, the abstract-publication match was reviewed by a second physician researcher. Time to publication, publication rates over time, and publication rates among US vs non-US authors were examined. PubMed identification numbers were also collected to determine if >1 abstract contributed to a manuscript. Data were analyzed using Wilcoxon rank sum, analysis of variance, t test, and logistic regression. RESULTS: In all, 3281 abstracts presented at SMFM over the study period, including 629 orals (63 main plenary, 64 fellows plenary, 502 concurrent), were reviewed. Of 3281, 1780 (54.3%) were published, generating 1582 unique publications. Oral abstracts had a consistently higher rate of conversion to publications vs posters (77.1% vs 48.8%, P < .001). The median time to publication was 19 (interquartile range, 9-36) months, and was significantly shorter for orals vs posters (11 vs 21 months, P < .001). Over the study period, rates of publication of orals remained constant, but rates of publication of posters were lower in 2007 and 2009 compared to 2003 and 2005. Publications related to SMFM abstracts were published in 194 different journals, most commonly American Journal of Obstetrics and Gynecology (39.8%), Obstetrics and Gynecology (9.7%), and Journal of Maternal-Fetal and Neonatal Medicine (6.5%). Publication rates were higher if the abstract's first author was affiliated with a non-US institution (64.8% vs 51.1%, P < .001) and if the abstract received an award (82.7% vs 53.3%, P < .001). In regression models, oral presentation at SMFM, first author affiliation with a non-US institution, submission for American Journal of Obstetrics and Gynecology SMFM special issue, and year of abstract presentation at SMFM were associated with full manuscript publication. CONCLUSION: From 2003 through 2010, full manuscript publication rates of SMFM abstracts were high and consistent, and time to publication decreased/improved across the study period for oral presentations.

Factors influencing successful peer-reviewed publication of original research presentations from the American Society of Pediatric Otolaryngology (ASPO).

OBJECTIVE: To analyze factors associated with progression of an original scientific presentation at the American Society of Pediatric Otolaryngology (ASPO) Annual Meeting to publication in a peer-reviewed journal. A dataset of presenters was created to enable prospective follow-up to assess early impressions regarding publication success with longer-term publication outcomes. METHODS: Prior to the Annual ASPO meeting in 2013, a survey consisting of 10 questions was e-mailed to all 59 presenters. Questions were designed to assess presenter expectations on publication, barriers to publishing, and experience in presenting, publishing and clinical practice. A second survey was sent 12 months later to those respondents of the first survey who were amenable to follow-up. RESULTS: Overall, 46 of 59 (78%) presenters responded to the initial survey prior to their ASPO 2013 presentation. Of these, 34 agreed to participate in a longer-term follow-up of their presentation to publication experience. Of these 34, there were 17 who participated in the follow-up survey 1 year later. Just under half of the original respondents were residents (46%). All presenters (100%) planned to re-submit a revised manuscript if initially rejected. However, 35% of follow-up respondents did not make initial submission to a peer-reviewed journal. Results of a descriptive analysis suggest that more experienced researchers expect their submitted manuscript to be accepted for publication within a shorter time frame than those that have published fewer papers. Time was ranked as the greatest barrier to publication (60%) of those surveyed both in the initial prospective survey and for those who did not publish a paper in the follow-up survey (83%). CONCLUSION: This study suggests a strong desire and expectation of publishing ASPO presentations. Despite this expectation, past research and this data set suggest this expectation often does not materialize. "Time constraints" were the most commonly identified barrier to publication. To enhance dissemination of new findings from ASPO meetings, institutions and individuals should examine methods that facilitate and incentivize publication of findings in peer-reviewed publications.

La revisión editorial por pares: rechazo del manuscrito, deficiencias del proceso de revisión, sistemas para su gestión y uso como indicador científico / Editorial peer review: manuscript rejection, peer-review deficiencies, systems for management and use as scientific performance indicator

Las plataformas digitales y la dinámica y evolución de las revistas científicas han permitido desarrollar diversos modelos del proceso de revisión editorial por pares para la evaluación de manuscritos científicos previo a su publicación. En este artículo se continúa el análisis de la revisión por pares, con énfasis en la gestión de artículos rechazados, la designación de revisores, las deficiencias principales de la revisión por pares (según la asignación de revisores, el desempeño de los roles de autor, revisor y editor, y los intentos por paliar las deficiencias del proceso), los sistemas para su gestión en línea y el empleo de la revisión por pares como indicador del desempeño investigativo. Todos estos temas se analizan en el contexto de los sistemas y comunidades de ciencia, su impacto en la citación, y para facilitar su posible integración con fines prácticos según los requerimientos de cada revista(AU)

The digital platforms and the dynamics and evolution of scientific journals have allowed the development of different models of the editorial peer review to evaluate scientific manuscripts before their publication. In this article, the PR analysis is continued, on the models to manage rejected papers, the proper designation of reviewers and PR main limitations (reviewers' assignment, author-reviewer-editor performance and the attempts to palliate process' deficiencies), the online PR management systems and the use of PR as scientific performance indicator. All these topics are analyzed through the context of the systems and communities of science and their impact in the citation, to facilitate their possible incorporation with practical aims according to the requirements of each journal(AU)

La revisión editorial por pares: rechazo del manuscrito, deficiencias del proceso de revisión, sistemas para su gestión y uso como indicador científico / Editorial peer review: manuscript rejection, peer-review deficiencies, systems for management and use as scientific performance indicator

Las plataformas digitales y la dinámica y evolución de las revistas científicas han permitido desarrollar diversos modelos del proceso de revisión editorial por pares para la evaluación de manuscritos científicos previo a su publicación. En este artículo se continúa el análisis de la revisión por pares, con énfasis en la gestión de artículos rechazados, la designación de revisores, las deficiencias principales de la revisión por pares (según la asignación de revisores, el desempeño de los roles de autor, revisor y editor, y los intentos por paliar las deficiencias del proceso), los sistemas para su gestión en línea y el empleo de la revisión por pares como indicador del desempeño investigativo. Todos estos temas se analizan en el contexto de los sistemas y comunidades de ciencia, su impacto en la citación, y para facilitar su posible integración con fines prácticos según los requerimientos de cada revista

The digital platforms and the dynamics and evolution of scientific journals have allowed the development of different models of the editorial peer review to evaluate scientific manuscripts before their publication. In this article, the PR analysis is continued, on the models to manage rejected papers, the proper designation of reviewers and PR main limitations (reviewers' assignment, author-reviewer-editor performance and the attempts to palliate process' deficiencies), the online PR management systems and the use of PR as scientific performance indicator. All these topics are analyzed through the context of the systems and communities of science and their impact in the citation, to facilitate their possible incorporation with practical aims according to the requirements of each journal

La revisión editorial por pares: roles y procesos / Editorial peer review: roles and processes

Las plataformas digitales y la dinámica y evolución de las revistas científicas han permitido desarrollar diversos modelos del proceso editorial de revisión por pares para la evaluación de manuscritos científicos previo a su publicación. En este artículo se aborda la naturaleza, evolución y características principales de la revisión por pares. Se hace un análisis comparativo e integrador de los modelos de revisión por pares convencionales (a simple y doble ciegas) y abiertos (ya sea por identidad o por proceso), sus ventajas y limitaciones. Se propone un nuevo sistema de clasificación de la revisión por pares por publicación (divulgación de la información del proceso convencional) y revisión por pares de proceso abierto (cuya información se divulga según se revisa el manuscrito previo al dictamen académico en una plataforma digital). Todos estos temas se analizan en el contexto de los sistemas y comunidades de ciencia, su impacto en la citación, y para facilitar su posible integración con fines prácticos según los requerimientos de cada revista.(AU)

The digital platforms and the dynamics and evolution of scientific journals have allowed the development of different models of the editorial peer review to evaluate scientific manuscripts before their publication. The nature, evolution and principal characteristics of peer review are presented in this article. A comparative and integrated analysis of the models of conventional peer review (simple and double blind) and open (either by identity or by process) is made, also considering their advantages and limitations. A new classification system is proposed for the Open peer review, divided in Open published peer review (spreading the information of the conventional process) and Open process peer review (in which the information is spread in a digital platform as the manuscript is reviewed prior to an academic report). All these topics are analyzed through the context of the systems and communities of science and their impact in the citation, to facilitate their possible incorporation with practical aims according to the requirements of each journal.(AU)

La revisión editorial por pares: roles y procesos / Editorial peer review: roles and processes

Las plataformas digitales y la dinámica y evolución de las revistas científicas han permitido desarrollar diversos modelos del proceso editorial de revisión por pares para la evaluación de manuscritos científicos previo a su publicación. En este artículo se aborda la naturaleza, evolución y características principales de la revisión por pares. Se hace un análisis comparativo e integrador de los modelos de revisión por pares convencionales (a simple y doble ciegas) y abiertos (ya sea por identidad o por proceso), sus ventajas y limitaciones. Se propone un nuevo sistema de clasificación de la revisión por pares por publicación (divulgación de la información del proceso convencional) y revisión por pares de proceso abierto (cuya información se divulga según se revisa el manuscrito previo al dictamen académico en una plataforma digital). Todos estos temas se analizan en el contexto de los sistemas y comunidades de ciencia, su impacto en la citación, y para facilitar su posible integración con fines prácticos según los requerimientos de cada revista.

The digital platforms and the dynamics and evolution of scientific journals have allowed the development of different models of the editorial peer review to evaluate scientific manuscripts before their publication. The nature, evolution and principal characteristics of peer review are presented in this article. A comparative and integrated analysis of the models of conventional peer review (simple and double blind) and open (either by identity or by process) is made, also considering their advantages and limitations. A new classification system is proposed for the Open peer review, divided in Open published peer review (spreading the information of the conventional process) and Open process peer review (in which the information is spread in a digital platform as the manuscript is reviewed prior to an academic report). All these topics are analyzed through the context of the systems and communities of science and their impact in the citation, to facilitate their possible incorporation with practical aims according to the requirements of each journal.