Peer Review in a General Medical Research Journal Before and During the COVID-19 Pandemic.

Importance: Although peer review is an important component of publication for new research, the viability of this process has been questioned, particularly with the added stressors of the COVID-19 pandemic. Objective: To characterize rates of peer reviewer acceptance of invitations to review manuscripts, reviewer turnaround times, and editor-assessed quality of reviews before and after the start of the COVID-19 pandemic at a large, open-access general medical journal. Design, Setting, and Participants: This retrospective, pre-post cohort study examined all research manuscripts submitted to JAMA Network Open between January 1, 2019, and June 29, 2021, either directly or via transfer from other JAMA Network journals, for which at least 1 peer review of manuscript content was solicited. Measures were compared between the period before the World Health Organization declaration of a COVID-19 pandemic on March 11, 2020 (14.3 months), and the period during the pandemic (15.6 months) among all reviewed manuscripts and between pandemic-period manuscripts that did or did not address COVID-19. Main Outcomes and Measures: For each reviewed manuscript, the number of invitations sent to reviewers, proportions of reviewers accepting invitations, time in days to return reviews, and editor-assessed quality ratings of reviews were determined. Results: In total, the journal sought review for 5013 manuscripts, including 4295 Original Investigations (85.7%) and 718 Research Letters (14.3%); 1860 manuscripts were submitted during the prepandemic period and 3153 during the pandemic period. Comparing the prepandemic with the pandemic period, the mean (SD) number of reviews rated as high quality (very good or excellent) per manuscript increased slightly from 1.3 (0.7) to 1.5 (0.7) (P < .001), and the mean (SD) time for reviewers to return reviews was modestly shorter (from 15.8 [7.6] days to 14.4 [7.0] days; P < .001), a difference that persisted in linear regression models accounting for manuscript type, study design, and whether the manuscript addressed COVID-19. Conclusions and Relevance: In this cohort study, the speed and editor-reported quality of peer reviews in an open-access general medical journal improved modestly during the initial year of the pandemic. Additional study will be necessary to understand how the pandemic has affected reviewer burden and fatigue.

What the American Journal of Critical Care Junior Peer Reviewers Were Reading During Year 2 of the Program.

The American Journal of Critical Care's Junior Peer Reviewer program aims to mentor novice reviewers in the peer review process. To grow their critical appraisal skills, the participants take part in discussion sessions in which they review articles published in other journals. Here we summarize the articles reviewed during the second year of the program, which again focused on the care of critically ill patients with COVID-19. This article aims to share these reviews and the reviewers' thoughts regarding the relevance, design, and applicability of the findings from the selected studies. High rates of delirium associated with COVID-19 may be impacted by optimizing sedation strategies and allowing safe family visitation. Current methodology in crisis standards of care may result in inequity and further research is needed. The use of extracorporeal carbon dioxide removal to facilitate super low tidal volume ventilation does not improve 90-day mortality outcomes. Continued research to better understand the natural history of COVID-19 and interventions useful for improving outcomes is imperative.

What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask.

Coincident with the tsunami of COVID-19-related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field.

General medical publications during COVID-19 show increased dissemination despite lower validation.

BACKGROUND: The COVID-19 pandemic has yielded an unprecedented quantity of new publications, contributing to an overwhelming quantity of information and leading to the rapid dissemination of less stringently validated information. Yet, a formal analysis of how the medical literature has changed during the pandemic is lacking. In this analysis, we aimed to quantify how scientific publications changed at the outset of the COVID-19 pandemic. METHODS: We performed a cross-sectional bibliometric study of published studies in four high-impact medical journals to identify differences in the characteristics of COVID-19 related publications compared to non-pandemic studies. Original investigations related to SARS-CoV-2 and COVID-19 published in March and April 2020 were identified and compared to non-COVID-19 research publications over the same two-month period in 2019 and 2020. Extracted data included publication characteristics, study characteristics, author characteristics, and impact metrics. Our primary measure was principal component analysis (PCA) of publication characteristics and impact metrics across groups. RESULTS: We identified 402 publications that met inclusion criteria: 76 were related to COVID-19; 154 and 172 were non-COVID publications over the same period in 2020 and 2019, respectively. PCA utilizing the collected bibliometric data revealed segregation of the COVID-19 literature subset from both groups of non-COVID literature (2019 and 2020). COVID-19 publications were more likely to describe prospective observational (31.6%) or case series (41.8%) studies without industry funding as compared with non-COVID articles, which were represented primarily by randomized controlled trials (32.5% and 36.6% in the non-COVID literature from 2020 and 2019, respectively). CONCLUSIONS: In this cross-sectional study of publications in four general medical journals, COVID-related articles were significantly different from non-COVID articles based on article characteristics and impact metrics. COVID-related studies were generally shorter articles reporting observational studies with less literature cited and fewer study sites, suggestive of more limited scientific support. They nevertheless had much higher dissemination.

The collective wisdom in the COVID-19 research: Comparison and synthesis of epidemiological parameter estimates in preprints and peer-reviewed articles.

OBJECTIVES: We aimed to explore the collective wisdom of preprints related to COVID-19 by comparing and synthesizing them with results of peer-reviewed publications. METHODS: PubMed, Google Scholar, medRxiv, bioRxiv, arXiv, and SSRN were searched for papers regarding the estimation of four epidemiological parameters of COVID-19: the basic reproduction number, incubation period, infectious period, and case-fatality-rate. Distributions of parameters and timeliness of preprints and peer-reviewed papers were compared. Four parameters in two groups were synthesized by bootstrapping, and their validities were evaluated by simulated cumulative cases of the susceptible-exposed-infectious-recovered-dead-cumulative (SEIRDC) model. RESULTS: A total of 106 papers were included for analysis. The distributions of four parameters in two literature groups were close, and the timeliness of preprints was better. Synthesized estimates of the basic reproduction number (3.18, 95% CI 2.85-3.53), incubation period (5.44 days, 95% CI 4.98-5.99), infectious period (6.25 days, 95% CI 5.09-7.51), and case-fatality-rate (4.51%, 95% CI 3.41%-6.29%) were obtained. Simulated cumulative cases of the SEIRDC model matched well with the onset cases in China. CONCLUSIONS: The validity of the COVID-19 parameter estimations of the preprints was on par with that of peer-reviewed publications, and synthesized results of literatures could reduce the uncertainty and be used for epidemic decision-making.

British Journal of Biomedical Science in 2020. What have we learned?

Each year the British Journal of Biomedical Science publishes a 'What have we learned' editorial designed to introduce readers within the major disciplines of laboratory medicine to developments outside their immediate area. In addition it is designed to inform a wider readership of the advances in the diagnosis and treatment of disease. To this end, in 2020 the journal published 39 articles covering the disciplines within Biomedical Science in the 4 issues comprising volume 77. These included a review of COVID-19 in this issue, 27 original articles, 6 Biomedical Science 'In Brief' and 4 case histories. 27 of the articles involved molecular techniques, with one of these comparing results with a mass spectrometry based method. The preponderance of molecular genetic studies gives us a good idea of the likely future direction of the disciplines.

Preprint manuscripts and servers in the era of coronavirus disease 2019.

RATIONALE, AIMS, AND OBJECTIVES: To both examine the impact of preprint publishing on health sciences research and survey popular preprint servers amidst the current coronavirus disease 2019 (COVID-19) pandemic. METHODS: The authors queried three biomedical databases (MEDLINE, Web of Science, and Google Scholar) and two preprint servers (MedRxiv and SSRN) to identify literature pertaining to preprints. Additionally, they evaluated 12 preprint servers featuring COVID-19 research through sample submission of six manuscripts. RESULTS: The realm of health sciences research has seen a dramatic increase in the presence and importance of preprint publications. By posting manuscripts on preprint servers, researchers are able to immediately communicate their findings, thereby facilitating prompt feedback and promoting collaboration. In doing so, they may also reduce publication bias and improve methodological transparency. However, by circumventing the peer-review process, academia incurs the risk of disseminating erroneous or misinterpreted data and suffering the downstream consequences. Never have these issues been better highlighted than during the ongoing COVID-19 pandemic. Researchers have flooded the literature with preprint publications as stopgaps to meet the desperate need for knowledge about the disease. These unreviewed articles initially outnumbered those published in conventional journals and helped steer the mainstream scientific community at the start of the pandemic. In surveying select preprint servers, the authors discovered varying usability, review practices, and acceptance polices. CONCLUSION: While vital in the rapid dispensation of science, preprint manuscripts promulgate their conclusions without peer review and possess the capacity to misinform. Undoubtedly part of the future of science, conscientious consumers will need to appreciate not only their utility, but also their limitations.

Preprint Servers in Kidney Disease Research: A Rapid Review.

Preprint servers, such as arXiv and bioRxiv, have disrupted the scientific communication landscape by providing rapid access to research before peer review. medRxiv was launched as a free online repository for preprints in the medical, clinical, and related health sciences in 2019. In this review, we present the uptake of preprint server use in nephrology and discuss specific considerations regarding preprint server use in medicine. Distribution of kidney-related research on preprint servers is rising at an exponential rate. Survey of nephrology journals identified that 15 of 17 (88%) are publishing original research accepted submissions that have been uploaded to preprint servers. After reviewing 52 clinically impactful trials in nephrology discussed in the online Nephrology Journal Club (NephJC), an average lag of 300 days was found between study completion and publication, indicating an opportunity for faster research dissemination. Rapid review of papers discussing benefits and risks of preprint server use from the researcher, publisher, or end user perspective identified 53 papers that met criteria. Potential benefits of biomedical preprint servers included rapid dissemination, improved transparency of the peer review process, greater visibility and recognition, and collaboration. However, these benefits come at the risk of rapid spread of results not yet subjected to the rigors of peer review. Preprint servers shift the burden of critical appraisal to the reader. Media may be especially at risk due to their focus on "late-breaking" information. Preprint servers have played an even larger role when late-breaking research results are of special interest, such as during the global coronavirus disease 2019 pandemic. Coronavirus disease 2019 has brought both the benefits and risks of preprint servers to the forefront. Given the prominent online presence of the nephrology community, it is poised to lead the medicine community in appropriate use of preprint servers.

Moving towards online rheumatology education in the era of COVID-19.

The coronavirus disease-2019 (COVID-19) pandemic has unsettled conventional medical education, hastening a switch to digital platforms and open-access publishing. Rheumatology is a fast evolving academic discipline that stands to gain by this switch. Most rheumatology textbooks are now available in digital formats, and these are complemented with live updating educational hubs such as UpToDate and ClinicalKey. Emerging topics of COVID-19 on these proprietary platforms are now freely available to all specialists. Social media channels, particularly Twitter, are becoming major players in the era of COVID-19 by offering online journal clubs, enabling fast dissemination of influential articles, and facilitating interactive education. Indexed rheumatology journals, in turn, aid online education by opening access to recommendations and other materials that are rapidly changing research and practice worldwide. Research peer review additionally offers learning experience to novice and seasoned researchers and authors. Global rheumatology societies have online learning resources, which are changing their format and geographic reach to meet the changing needs in the times of pandemic. While online teaching lacks emotional connections between mentors and mentees, switch to a more interactive format of education and regular contacts may partly solve the issue. Rheumatologists can take the lead in these challenging times and contribute more to online scholarly activities which are aimed to maintain and enrich education. Key Points • Disparities in rheumatology education are likely to be widened during the COVID-19 pandemic. • Barriers to rheumatology education include limited number of instructors and their limited experience in online teaching. • Online textbooks, didactic materials of indexed rheumatology journals, and frequently updated online educational hubs such as UpToDate serve as a foundation of online rheumatology education. • Online rheumatology education is enriched by peer review and social media activities, which are becoming major players in the time of the COVID-19 pandemic.