Supporting grant reviewers through the scientometric ranking of applicants.

INTRODUCTION: Comparing the scientific output of different researchers applying for a grant is a tedious work. In Hungary, to help reviewers to rapidly rank the scientific productivity of a researcher, a grant decision support tool was established and is available at www.scientometrics.org. In the present study, our goal was to assess the impact of this decision support tool on grant review procedures. METHODS: The established, publicly available scientometric portal uses four metrics, including the H-index, the yearly citations without self-citations, the number of publications in the last five years, and the number of highly cited publications of a researcher within eleven independent scientific disciplines. Publication-age matched researchers are then ranked and the results are provided to grant reviewers. A questionnaire was completed by reviewers regarding utilization of the scientometric ranking system. The outcome of the grant selection was analyzed by comparing scientometric parameters of applying and funded applicants. We compared three grant allocation rounds before to two grant allocation rounds after the introduction of the portal. RESULTS: The scientometric decision support tool was introduced in 2020 to assist grant selection in Hungary and all basic research grant applicants (n = 6,662) were screened. The average score of funded proposals compared to submitted proposals increased by 94% after the introduction of the ranking. Correlation between ranking scores and actual grant selection was strong in life and material sciences but some scientific panels had opposite correlation in social sciences and humanities. When comparing selection outcome to H-index across all applicants, both type I and type II errors decreased. All together 540 reviewers provided feedback representing all eleven scientific disciplines and 83.05% of the reviewers (especially younger reviewers) found the ranking useful. CONCLUSIONS: The scientometric decision support tool can save time and increase transparency of grant review processes. The majority of reviewers found the ranking-based scientometric analysis useful when assessing the publication performance of an applicant.

Correction: MultipleTesting.com: A tool for life science researchers for multiple hypothesis testing correction.

[This corrects the article DOI: 10.1371/journal.pone.0245824.].

MultipleTesting.com: A tool for life science researchers for multiple hypothesis testing correction.

Scientists from nearly all disciplines face the problem of simultaneously evaluating many hypotheses. Conducting multiple comparisons increases the likelihood that a non-negligible proportion of associations will be false positives, clouding real discoveries. Drawing valid conclusions require taking into account the number of performed statistical tests and adjusting the statistical confidence measures. Several strategies exist to overcome the problem of multiple hypothesis testing. We aim to summarize critical statistical concepts and widely used correction approaches while also draw attention to frequently misinterpreted notions of statistical inference. We provide a step-by-step description of each multiple-testing correction method with clear examples and present an easy-to-follow guide for selecting the most suitable correction technique. To facilitate multiple-testing corrections, we developed a fully automated solution not requiring programming skills or the use of a command line. Our registration free online tool is available at www.multipletesting.com and compiles the five most frequently used adjustment tools, including the Bonferroni, the Holm (step-down), the Hochberg (step-up) corrections, allows to calculate False Discovery Rates (FDR) and q-values. The current summary provides a much needed practical synthesis of basic statistical concepts regarding multiple hypothesis testing in a comprehensible language with well-illustrated examples. The web tool will fill the gap for life science researchers by providing a user-friendly substitute for command-line alternatives.

Dynamic classification using case-specific training cohorts outperforms static gene expression signatures in breast cancer.

The molecular diversity of breast cancer makes it impossible to identify prognostic markers that are applicable to all breast cancers. To overcome limitations of previous multigene prognostic classifiers, we propose a new dynamic predictor: instead of using a single universal training cohort and an identical list of informative genes to predict the prognosis of new cases, a case-specific predictor is developed for each test case. Gene expression data from 3,534 breast cancers with clinical annotation including relapse-free survival is analyzed. For each test case, we select a case-specific training subset including only molecularly similar cases and a case-specific predictor is generated. This method yields different training sets and different predictors for each new patient. The model performance was assessed in leave-one-out validation and also in 325 independent cases. Prognostic discrimination was high for all cases (n = 3,534, HR = 3.68, p = 1.67 E-56). The dynamic predictor showed higher overall accuracy (0.68) than genomic surrogates for Oncotype DX (0.64), Genomic Grade Index (0.61) or MammaPrint (0.47). The dynamic predictor was also effective in triple-negative cancers (n = 427, HR = 3.08, p = 0.0093) where the above classifiers all failed. Validation in independent patients yielded similar classification power (HR = 3.57). The dynamic classifier is available online at http://www.recurrenceonline.com/?q=Re_training. In summary, we developed a new method to make personalized prognostic prediction using case-specific training cohorts. The dynamic predictors outperform static models developed from single historical training cohorts and they also predict well in triple-negative cancers.