"Best Paper" awards lack transparency, inclusivity, and support for Open Science.

Awards can propel academic careers. They also reflect the culture and values of the scientific community. But do awards incentivize greater transparency, inclusivity, and openness in science? Our cross-disciplinary survey of 222 awards for the "best" journal articles across all 27 SCImago subject areas revealed that journals and learned societies administering such awards generally publish little detail on their procedures and criteria. Award descriptions were brief, rarely including contact details or information on the nominations pool. Nominations of underrepresented groups were not explicitly encouraged, and concepts that align with Open Science were almost absent from the assessment criteria. At the same time, 10% of awards, especially the recently established ones, tended to use article-level impact metrics. USA-affiliated researchers dominated the winner's pool (48%), while researchers from the Global South were uncommon (11%). Sixty-one percent of individual winners were men. Overall, Best Paper awards miss the global calls for greater transparency and equitable access to academic recognition. We provide concrete and implementable recommendations for scientific awards to improve the scientific recognition system and incentives for better scientific practice.

Structured peer review: pilot results from 23 Elsevier journals.

Background: Reviewers rarely comment on the same aspects of a manuscript, making it difficult to properly assess manuscripts' quality and the quality of the peer review process. The goal of this pilot study was to evaluate structured peer review implementation by: 1) exploring whether and how reviewers answered structured peer review questions, 2) analysing reviewer agreement, 3) comparing that agreement to agreement before implementation of structured peer review, and 4) further enhancing the piloted set of structured peer review questions. Methods: Structured peer review consisting of nine questions was piloted in August 2022 in 220 Elsevier journals. We randomly selected 10% of these journals across all fields and IF quartiles and included manuscripts that received two review reports in the first 2 months of the pilot, leaving us with 107 manuscripts belonging to 23 journals. Eight questions had open-ended fields, while the ninth question (on language editing) had only a yes/no option. The reviews could also leave Comments-to-Author and Comments-to-Editor. Answers were independently analysed by two raters, using qualitative methods. Results: Almost all the reviewers (n = 196, 92%) provided answers to all questions even though these questions were not mandatory in the system. The longest answer (Md 27 words, IQR 11 to 68) was for reporting methods with sufficient details for replicability or reproducibility. The reviewers had the highest (partial) agreement (of 72%) for assessing the flow and structure of the manuscript, and the lowest (of 53%) for assessing whether interpretation of the results was supported by data, and for assessing whether the statistical analyses were appropriate and reported in sufficient detail (52%). Two thirds of the reviewers (n = 145, 68%) filled out the Comments-to-Author section, of which 105 (49%) resembled traditional peer review reports. These reports contained a Md of 4 (IQR 3 to 5) topics covered by the structured questions. Absolute agreement regarding final recommendations (exact match of recommendation choice) was 41%, which was higher than what those journals had in the period from 2019 to 2021 (31% agreement, P = 0.0275). Conclusions: Our preliminary results indicate that reviewers successfully adapted to the new review format, and that they covered more topics than in their traditional reports. Individual question analysis indicated the greatest disagreement regarding the interpretation of the results and the conducting and the reporting of statistical analyses. While structured peer review did lead to improvement in reviewer final recommendation agreements, this was not a randomized trial, and further studies should be performed to corroborate this. Further research is also needed to determine whether structured peer review leads to greater knowledge transfer or better improvement of manuscripts.

Research integrity in Instructions for Authors in Japanese medical journals using ICMJE Recommendations: A descriptive literature study.

BACKGROUND: The International Committee of Medical Journal Editors (ICMJE) has published Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. These provide a global standard for writing and editing medical articles, including research integrity. However, no study has examined the research integrity-related content of Japanese medical journals' Instructions for Authors. We therefore compared research integrity content in ICMJE member journals with those in the English- and Japanese-language journals of the Japanese Association of Medical Sciences (JAMS). MATERIALS AND METHODS: This was a descriptive literature study. We obtained Instructions for Authors from English- and Japanese-language journals listed on the JAMS website and the ICMJE member journals listed on the ICMJE website as of September 1, 2021. We compared the presence of 20 topics (19 in the ICMJE Recommendations plus compliance with ICMJE) in the Instructions for Authors, and analyzed the content of the conflict of interest disclosure. RESULTS: We evaluated 12 ICMJE member journals, and 82 English-language and 99 Japanese-language subcommittee journals. The median number of topics covered was 10.5 for ICMJE member journals, 10 for English-language journals, and three for Japanese-language journals. Compliance with ICMJE was mentioned by 10 (83%) ICMJE member journals, 75 (91%) English-language journals, and 29 (29%) Japanese-language journals. The ICMJE Conflicts of Interest Disclosure Form was requested by seven (64%) ICMJE member journals, 15 (18%) English-language journals, and one (1%) Japanese-language journal. CONCLUSIONS: Although the topics in the JAMS English-language journals resembled those in the ICMJE member journals, the median value of ICMJE-related topic inclusion was approximately one-third lower in JAMS Japanese-language journals than in ICMJE member journals. It is hoped that Japanese-language journals whose conflict of interest disclosure policies differ from ICMJE standards will adopt international standards to deter misconduct and ensure publication quality.

[How to: correctly read scientific articles : Research design and methodology]. / How to: einen wissenschaftlichen Artikel richtig lesen : Forschungsdesign und Methodik.

Criteria for assessment of the significance of scientific articles are presented. The focus is on research design and methodology, illustrated by the classical study on prehospital volume treatment of severely injured individuals with penetrating torso injuries by Bickell et al. (1994). A well-thought out research design is crucial for the success of a scientific study and is documented in a study protocol beforehand. A hypothesis is a provisional explanation or prediction and must be testable, falsifiable, precise, and relevant. There are various types of randomization methods, with the randomized controlled trial being the gold standard for clinical interventional studies. When reading a scientific article it is important to verify whether the research design and setting align with the research question and whether potential sources of error have been considered and controlled. Critical scrutiny should also be applied to references, the funding and expertise of the researchers.

[Scientific, transparent and applicable rankings of Chinese pathological guidelines and consensus published in the medical journals in 2022].

The STAR tool was used to evaluate and analyze the science, transparency, and applicability of Chinese pathology guidelines and consensus published in medical journals in 2022. There were a total of 18 pathology guidelines and consensuses published in 2022, including 1 guideline and 17 consensuses. The results showed that the guideline score was 21.83 points, lower than the overall guideline average (43.4 points). Consensus ratings scored an average of 27.87 points, on par with the overall consensus level (28.3 points). Areas that scored above the overall level were "conflict of interest" and "working groups", while areas that scored below the overall level were "proposals", "funding", "evidence", "consensus approaches" and "accessibility". To sum up, the formulation of pathology guidelines and consensuses in 2022 is not standardized, and the evidence retrieval process, evidence evaluation methods and grading criteria for recommendations on clinical issues are not provided in the formulation process; the process and method for reaching consensus are not provided, the plan is lacking, and registration is not carried out. It is therefore suggested that guidelines/consensus makers in the field of pathology should attach importance to evidence-based medical evidence, strictly follow guideline formulation methods and processes, further improve the scientific, applicable and transparent guidelines/consensuses in the field, and better provide support for clinicians and patients.

Variability of Guidelines and Disclosures for AI-Generated Content in Top Surgical Journals.

Background: When properly utilized, artificial intelligence generated content (AIGC) may improve virtually every aspect of research, from data gathering to synthesis. Nevertheless, when used inappropriately, the use of AIGC may lead to the dissemination of inaccurate information and introduce potential ethical concerns.Research Design: Cross-sectional. Study Sample: 65 top surgical journals. Data Collection: Each journals submission guidelines and portal was queried for guidelines regarding AIGC use.Results: We found that, in July 2023, 60% of the top 65 surgical journals had introduced guidelines for use, with more surgical journals (68%) introducing guidelines than surgical subspecialty journals (52.5%), including otolaryngology (40%). Furthermore, of the 39 with guidelines, only 69.2% gave specific use guidelines. No included journal, at the time of analysis, explicitly disallowed AIGC use.Conclusions: Altogether, this data suggests that while many journals have quickly reacted to AIGC usage, the quality of such guidelines is still variable. This should be pre-emptively addressed within academia.

Quality assessment of paediatric randomized controlled trials published in China from 1999 to 2022: a cross-sectional study.

BACKGROUND: Randomized controlled trials (RCTs) are usually the basis of evidence-based medicine, but whether the results of RCTs can be correctly translated into clinical practice depends on the quality of the literature reported. In this study, we evaluated the general characteristics and quality of paediatric RCTs published in China to provide evidence for the reporting of paediatric RCTs and their application in clinical practice. METHODS: We conducted a cross-sectional observational study of paediatric RCTs published in paediatric journals in China between January 1, 1999, and December 30, 2022. All RCTs that included children (younger than 18 years old) were retrieved, and the general characteristics of the RCTs were extracted and analysed. The quality of the RCTs was assessed by the Cochrane quality assessment protocol. RESULTS: After screening 20 available paediatric journals, 3545 RCTs were included for analysis. The average annual growth rate of the number of published paediatric RCTs from 1999 to 2022 was 7.8% (P = 0.005, R2 = 0.311). Most of the studies were carried out in East China [1148 (32.4%]; the centres of the RCTs were mainly single-centre [3453 (97.4%], and the interventions were mainly medication [2442 (68.9%)]. Comparing RCTs published in 2017-2022 with RCTs published in 1999-2004, the quality of RCTs significantly improved in terms of random sequence generation, allocation concealment, blinding participants and personnel, incomplete outcome data and selective outcome reporting. RCTs published in multiple centres from the Chinese Science Citation Database were identified, and the approval of the ethics committee was of better quality for all the analysed risk of bias items. CONCLUSION: The number and quality of paediatric RCTs reported in China have improved in recent years, but the overall quality was relatively low. Special attention should be given to allocation concealment and blinding outcome assessment, and dropouts, adverse effects and sample size calculations should be reported. Promoting government policies, strengthening the standardization of journal publishing and advancing the registration of clinical trials are feasible measures.

Publish and perish: New issues in publication ethics.

The expression "Publish or perish," first appeared in 1942. It signified the rising importance of publication as a means to obtain research funds and establish a secure academic career. The expression is still highly relevant, but increasingly problematic. Perhaps it should be revised to read "Publish and Perish". We have reached a point where researchers, especially in non-English speaking countries, are no longer able to afford to publish their research. There seems little point in undertaking research if we can no longer disseminate or, indeed, apply the wisdom gained from it.

Investigating the nature of open science practices across complementary, alternative, and integrative medicine journals: An audit.

BACKGROUND: Open science practices are implemented across many scientific fields to improve transparency and reproducibility in research. Complementary, alternative, and integrative medicine (CAIM) is a growing field that may benefit from adoption of open science practices. The efficacy and safety of CAIM practices, a popular concern with the field, can be validated or refuted through transparent and reliable research. Investigating open science practices across CAIM journals by using the Transparency and Openness Promotion (TOP) guidelines can potentially promote open science practices across CAIM journals. The purpose of this study is to conduct an audit that compares and ranks open science practices adopted by CAIM journals against TOP guidelines laid out by the Center for Open Science (COS). METHODS: CAIM-specific journals with titles containing the words "complementary", "alternative" and/or "integrative" were included in this audit. Each of the eight TOP criteria were used to extract open science practices from each of the CAIM journals. Data was summarized by the TOP guideline and ranked using the TOP Factor to identify commonalities and differences in practices across the included journals. RESULTS: A total of 19 CAIM journals were included in this audit. Across all journals, the mean TOP Factor was 2.95 with a median score of 2. The findings of this study reveal high variability among the open science practices required by journals in this field. Four journals (21%) had a final TOP score of 0, while the total scores of the remaining 15 (79%) ranged from 1 to 8. CONCLUSION: While several studies have audited open science practices across discipline-specific journals, none have focused on CAIM journals. The results of this study indicate that CAIM journals provide minimal guidelines to encourage or require authors to adhere to open science practices and there is an opportunity to improve the use of open science practices in the field.