ABSTRACT
Introducción: La investigación científica tiende a ser presentada mediante reportes escritos, que inicialmente son denominados manuscritos, la revisión de su calidad es importante en el marco del rigor metodológico y científico. Objetivo: Diseñar y validar una rúbrica analítica para evaluar manuscritos científicos. Material y Métodos: Estudio instrumental. Donde se elabora una rúbrica analítica compuesta por 21 aspectos integrados en seis dimensiones y cuatro niveles de desempeño (Excelente, Bien, Regular, Por mejorar). Se utiliza la metodología de jueces expertos (n= 9), fiabilidad mediante el Alfa de Krippendorff (α) y relación prueba criterio externo e información cualitativa para la mejora de los indicadores. Resultados: El juicio de expertos indicó que los 21 aspectos a evaluar pueden ser considerados validos (V de Aiken ≥ 0.70) y fiables (α ≥ 0.70). Además, las sugerencias de los expertos permitieron mejoras cualitativas al instrumento. La aplicación piloto con un grupo de siete evaluadores indicó que la prueba cuenta con poder predictivo, porque se encontró que las puntuaciones de los jueces disminuyen mientras disminuye la calidad del manuscrito seleccionado. Conclusiones: La rúbrica analítica elaborada es un instrumento válido y fiable que puede ser utilizado para la valoración de manuscritos científicos en el ámbito de la educación médica. El proceso metodológico brinda evidencias solidas de su funcionamiento. A pesar de eso, se anima a continuar revisando el instrumento como parte de su proceso de mejora continua.
Introduction: Scientific research tends to be presented through written reports, which are initially called manuscripts. The review of their quality is important in the framework of methodological and scientific rigor. Objective: To design and validate an analytical rubric to evaluate scientific manuscripts. Material and Methods: Instrumental study. An analytical rubric composed of 21 aspects integrated into six dimensions and four performance levels (excellent, good, fair, to be improved) was developed. The methodology of expert judges (n = 9), reliability by means of Krippendorff's alpha (α), external criterion test, and qualitative information for the improvement of the indicators were used. Results: The expert judgment indicated that the 21 aspects to be evaluated can be considered valid (Aiken's V ≥ 0.70) and dependable (α ≥ 0.70). In addition, the experts' suggestions allowed qualitative improvements to the instrument. The pilot application with a group of seven evaluators indicated that the test has predictive power because it was found that the judges' scores decreased as the quality of the selected manuscript decreased. Conclusions: The analytical rubric elaborated is a valid and reliable instrument that can be used for the assessment of scientific manuscripts in the field of medical education. The methodological process provides solid evidence of its performance. Nevertheless, it is encouraged to continue revising the instrument as part of its continuous improvement process.
Subject(s)
Humans , Male , Female , Research Report/standardsABSTRACT
Reviewers play a vital role in ensuring quality control of scientific manuscripts published in any journal. The traditional double blind peer review, although a time-tested method, has come under increasing criticism in the face of emerging trends in the review process with the primary concern being the delays in completion of the review process. Other issues are the inability to detect errors/fraud, lack of transparency, lack of reliability, potential for bias, potential for unethical practices, lack of objectivity, inconsistencies amongst reviewers, lack of recognition and motivation of reviewers. Alternative options to classical peer review being propagated are: open review, immediate self-publication using preprint servers, nonselective review focusing primarily on the scientific content, and post-publication review. These alternative review processes, however, may suffer from the inability to validate quality control. In addition, anecdotal instances of peer review frauds are being reported more often than earlier. Suggested means to ensure quality of peer review process includes:(a) each journal to have its own database of reviewers, (b) verification of email IDs of reviewers provided by authors along with details of their institutions, (c) ensure credibility of reviewers before requesting for review, (d) check for plagiarism at the editorial level, (e) editors to distinguish between a good review from a possible biased/bad review, and (f) give recognition for reviewers once in a year. To conclude, quickness of review and publication should not dictate the scientific publication process at the cost of quality of contents.
