Gender biases in the evaluation of knowledge transfer: A meta-evaluative analysis of the Spanish "Knowledge Transfer and Innovation Sexennium".

This paper analyses from a gender perspective a pilot call for evaluating academics and researcher transfer and innovation activities, launched by the Spanish Government in 2018, known as the "Knowledge Transfer & Innovation Sexennium" (KT&IS). Not only women's participation was much lower than that of men (1 woman applicant for every 3 men applicants), but also, they showed lower success rates than men in all scientific fields, with an average gap of more than 13 points. The methodology combined an exploratory quantitative analysis of the almost 17,000 applications, with a meta-evaluative qualitative analysis through interviews to key actors of the evaluation program and focus groups with evaluators. Hidden biases operating throughout and in each of the different phases of the KT&IS evaluation process were identified. This article aims at contributing to how economic and social impact of research can be fairly and fully evaluated, as well as at facilitating the design of future evaluation calls that promote the effective advancement of gender equality in all science-related activities and transfer to society.

Correlation between journal metrics-based academic evaluation and researchers' ethics.

The "publish or perish" approach has become an integral part of an academic's life when seeking positions, striving for promotions, or competing for funding. This approach often hinges on journal-based metrics which push researchers to seek publication in journals indexed in the Web of Science. Due to the pressure to publish a certain number of publications in journals indexed in the Web of Science, researchers might attempt to find a journal with a lower impact factor, i.e., less popular and visible journals in the scientific community. Even more concerning is the fact that researchers might publish their results in predatory journals. This paper analyzes the consequence of introducing a journal indicators-based academic evaluation by analyzing productivity and publication patterns of researchers. Moreover, this paper investigates the correlation between journal-based academic evaluation rules and researchers' ethics. The analysis is based on bibliometric data collected from the Web of Science database. The case study subject is the Serbian research landscape before and after the introduction of a journal metrics-based academic evaluation.

Anxiety and heart rate in a real-life class test in undergraduates choosing real-time or prerecorded oral presentations.

Most studies on acute stress stem from works conducted under artificial laboratory conditions. Inducing stress for experimental scrutiny is problematic and can be unethical. In this study, a 'research methods' course's curriculum included a demonstration study testing anxiety and heart rate responses to the midterm test. Fifty-four university students (35 males and 19 females) presented a research topic in-person (n = 14) or prerecorded while being present (n = 40). Students selected a test format they could change until the week before the midterm test. The measures were trait anxiety, test anxiety, state anxiety, heart rate (HR), the last two being measured before and after examination, and grades. All students manifested decreased state anxiety and increased HR from before to after the test. Females exhibited higher HR and state anxiety than males. Real-time presentations were associated with higher HRs but not higher state anxiety. Those who changed their planned presentation mode from in-person to prerecorded exhibited higher test anxiety but not trait anxiety than those who presented as planned. Students who presented in-person obtained lower grades than those who prerecorded their presentations. Grades were negatively correlated with state anxiety after the test, but test anxiety did not significantly mediate the grades. Pre-test state anxiety was positively associated with both trait anxiety and test anxiety. The findings suggest that test anxiety is unlikely to affect grades, but test-anxious students need more control over academic evaluation. A choice in test format could be helpful in this regard.

Application of Statistical K-Means Algorithm for University Academic Evaluation.

With the globalization of higher education, academic evaluation is increasingly valued by the scientific and educational circles. Although the number of published papers of academic evaluation methods is increasing, previous research mainly focused on the method of assigning different weights for various indicators, which can be subjective and limited. This paper investigates the evaluation of academic performance by using the statistical K-means (SKM) algorithm to produce clusters. The core idea is mapping the evaluation data from Euclidean space to Riemannian space in which the geometric structure can be used to obtain accurate clustering results. The method can adapt to different indicators and make full use of big data. By using the K-means algorithm based on statistical manifolds, the academic evaluation results of universities can be obtained. Furthermore, through simulation experiments on the top 20 universities of China with the traditional K-means, GMM and SKM algorithms, respectively, we analyze the advantages and disadvantages of different methods. We also test the three algorithms on a UCI ML dataset. The simulation results show the advantages of the SKM algorithm.

Games academics play and their consequences: how authorship, h-index and journal impact factors are shaping the future of academia.

Research is a highly competitive profession where evaluation plays a central role; journals are ranked and individuals are evaluated based on their publication number, the number of times they are cited and their h-index. Yet such evaluations are often done in inappropriate ways that are damaging to individual careers, particularly for young scholars, and to the profession. Furthermore, as with all indices, people can play games to better their scores. This has resulted in the incentive structure of science increasingly mimicking economic principles, but rather than a monetary gain, the incentive is a higher score. To ensure a diversity of cultural perspectives and individual experiences, we gathered a team of academics in the fields of ecology and evolution from around the world and at different career stages. We first examine how authorship, h-index of individuals and journal impact factors are being used and abused. Second, we speculate on the consequences of the continued use of these metrics with the hope of sparking discussions that will help our fields move in a positive direction. We would like to see changes in the incentive systems, rewarding quality research and guaranteeing transparency. Senior faculty should establish the ethical standards, mentoring practices and institutional evaluation criteria to create the needed changes.

Interpretando a internacionalização dos periódicos científicos brasileiros / Interpreting the internationalization of brazilian scientific journals / Interpretando la internacionalización de los periódicos científicos brasileños

Explora o processo de internacionalização acadêmica dos periódicos científicos brasileiros. Mediante revisão bibliográfica, é analisado o cenário do desenvolvimento dessas publicações nas últimas décadas na América Latina, assim como as recentes ações das agências nacionais de fomento à Ciência, Tecnologia e Inovação (CT&I) para sua internacionalização. Conclui-se que, não existindo um plano para a internacionalização dos periódicos científicos brasileiros, é necessário o diálogo entre os diferentes atores para alcançar uma conjuntura inclusiva, em que prevaleça o princípio da informação científica como bem público e o livre acesso à produção científica a qualquer cidadão.

It explores the process of academic internationalization of Brazilian scientific journals. Through a bibliographic review, it analyzes the scenario of the development of these publications in the last decades in Latin America, as well as the recent actions of the national agencies for the promotion of science, technology and innovation (STI) for their internationalization. It is concluded that, in the absence of a plan for the internationalization of Brazilian scientific journals, dialogue between the different actors is necessary to achieve an inclusive situation, where the principle of scientific information as a public good prevails, together with free access to scientific production for any citizen.

Explora el proceso de internacionalización académica de las revistas científicas brasileñas. Mediante la revisión bibliográfica es analizado el escenario del desarrollo de estas publicaciones en las últimas décadas en América Latina, así como las recientes acciones de las agencias nacionales de fomento a la Ciencia, Tecnología e Innovación (CT&I) para su internacionalización. Se concluye que, no existiendo un plan para a internacionalización de las revistas científicas brasileñas, es necesario el diálogo entre los diferentes actores para alcanzar una coyuntura inclusiva, donde prevalezca el principio de la información científica como bien público y el libre acceso a la producción científica a cualquier ciudadano.

A warning for Chinese academic evaluation systems: short-term bibliometric measures misjudge the value of pioneering contributions.

Publication citation-based research evaluation, even if only in support of peer review, is not everywhere, on every level, or for everyone suitable, because of differences in scientific research, patterns of research output, stages of scientific evolution, and merits-scientific or societal-of scientific results.

Construction of an evaluation index system for determining the academic impact of military medical scholars.

INTRODUCTION: Academic papers are an essential manner for describing new ideas and consolidating existing concepts in the field of military medicine. The academic impact of military medical publications reflects the extent and depth of recognition, acceptance and utilisation of the concepts transmitted in these publications. The aim of this research was to construct an evaluation index system suitable for evaluating the academic influence of scholars in the field of military medicine. METHODS: Using the Delphi consensus methodology, 30 experts from the field of military medicine, military medical information and library and information science were asked during three rounds of questioning to score the feasibility and importance of indicators that could be used to determine academic impact. An analytic hierarchy process method was used to calculate the relative weighting of each indicator in determining the final level of academic impact. RESULTS: Eight evaluation indicators were agreed on to potentially determine academic impact. These comprised: 'Web of Science documents', 'Citation impact', 'h-index', 'Percentage of international collaborations', 'Percentage of the top 10% of the cited frequency', 'Category normalised citation impact', 'Percentage of documents cited' and 'The number of F1000 Recommended papers'. CONCLUSIONS: The evaluation index system determined from this study combines the advantages of both qualitative and quantitative recognised evaluation indicators, which are subsequently weighted according to their importance in the field of military medicine. It is hoped that this framework will provide a manner in the future for comparing the potential academic impact of military medical scholars.

A warning for Chinese academic evaluation systems: short-term bibliometric measures misjudge the value of pioneering contributions / 浙江大学学报（英文版）（B辑：生物医学和生物技术）

Publication citation-based research evaluation, even if only in support of peer review, is not everywhere, on every level, or for everyone suitable, because of differences in scientific research, patterns of research output, stages of scientific evolution, and merits-scientific or societal-of scientific results.

Design and implementation of Traditional Chinese Medicine-Paper Influence Evaluation System / 中华医学图书情报杂志

Evaluation of paper academic influence is an important component part for the scientific and technological evaluation of traditional Chinese medicine,and plays an important role in promoting the scientific and technological innovative development of traditional Chinese medicine. The Traditional Chinese Medicine-Paper Influence Evaluation System (TCM-PIES), which was thus designed according to the existing problems in scientific and technological evaluation of traditional Chinese medicine and in evaluation of papers on traditional Chinese medicine,and the characteristics of academic papers on traditional Chinese medicine by analyzing their data with information technology as its support, is applied in selection of excellent institutions and professional academic interests.

Quantitative Assessment of Theses at Mazandaran University of Medical Sciences Years-(1995-2014).

BACKGROUND: Review and evaluation of research for the correct steps towards real progress is essential which is a healthy and dynamic feature of the system. For the correct step toward real progress, evaluation research is essential which is feature of healthy and dynamic system. Considering the importance of scientific thesis in production and development and be aware of as the lack of structured information and qualitative and quantitative assessment at Mazandaran University of Medical Sciences, therefore we decided to do qualitative stud of theirs prepared 1995-2014. METHODS: This study was a descriptive survey, a sample of 325 graduate and PhD thesis and dissertation in clinical and basic science at the university of medical sciences of the population in 2060 is a thesis from 1994 to the end of 2014. To study the population, stratified sampling method was used. The descriptive study was conducted in terms of matching the degree thesis students, theses subjects, specialty of supervisors and Advisers. The data gathering tool was checklist of information (gender, discipline, degree and department education of students, School, year of dependence, title of theses and dissertations, specialty and departments of supervisors and advisers, type of research, grade obtained of students). Statistical analysis of the data was performed using 21 SPSS software. RESULTS: We studied 325 theses; 303 dissertations which 1 researcher; 21 dissertations which 2 researchers and 1 dissertation with 3 researchers. A total of 348 students (174 females and 174 males) researcher had theses. The number of students in the Department of Basic Science 82 (23.5%), 266 (76.5 %) in clinical group; 29(8.33%), 29 (8.33%) master degree; 260 (74.71%) general practitioner; 58 (16.67%) specialty and 1(29) at the PhD level. There was no relationship between research and level of education (p = 0.081). However, it was found that majority of the theses for the general practitioner (59.8%) wryer type 1(status condition). By matching and determining the overlapping of specialty of the advisor and the guide with the title, titles of the theses. It was found that 298 (91.69%) of the theses correspond with the specialty of the advisors and the supervisors. CONCLUSION: Based on the results obtained on the review of the theses to direct students to fundamental research and applying of specialized groups of supervisors and advisers in accordance with research is felt. No doubt that in the future will improve the quality of the students' theses.

Assessment of impact of papers on clinical research in view of evidence-based medicine / 中华医学图书情报杂志

Analyzed in this paper are the methods and their operability and efficiency for assessing the impact of papers on clinical research in view of evidence-based medicine, including citation analysis based on the clinical practice guidelines and evidence-based medicine database and comparative analysis of clinical trials registry data-base, with the model for assessing the impact of papers on clinical research in view of evidence-based medicine proposed.

Papers published in PLoS One by hospitals in China / 中华医学图书情报杂志

Objective To study the reasons why PLoS One has set off an upsurge in China especially in hospitals, the academic level of papers published in PLoS One, and how to correctly judge the SCI-covered papers published by hospitals.Methods Papers published in PLoS One by hospitals in China were retrieved.The number of papers published in PLoS One by hospitals in China, the institutions that published the papers in PLoS One, and the num-ber of fund-supported papers were analyzed.Results A total of 9739 papers were published in PLoS One by hospitals in China from 2007 to October 2015, with an average cited frequency of 5.29.The majority of authors who published the papers were from affiliated hospitals of scientific research institutions and teaching hospitals of colleges and uni-versities.Of the 9739 papers, 87.78%were supported by funds from different foundations.Conclusion Nothing is strike about the academic level of papers published in PLoS One by hospitals in China with their average cited fre-quency lower than that (8.14) in our country.The SCI-covered papers published by hospitals should not be con-sidered as the only criterion in assessing the performances of staff in hospitals.

Evaluation methods of individual technology maturity based on SCI-covered papers and patent data / 中华医学图书情报杂志

Objective To establish the evaluation methods of individual technology maturity based on SCI-covered papers and patent data using Gartner's Hype Cycle.Methods The SCI-covered papers and patent data were ana-lyzed by nonlinear regression analysis in combination with traditional X-radiography and photo-acoustic tomo-graphy, and represented as the bell-shaped curve and S-shaped curve for judging the technology maturity.Results The individual technology maturity was consistent with the qualitative review of experts.Conclusion The individual technology described in SCI-covered papers and patent data can be used in primary evaluation of its maturity.

A numerical algorithm with preference statements to evaluate the performance of scientists.

Academic evaluation committees have been increasingly receptive for using the number of published indexed articles, as well as citations, to evaluate the performance of scientists. It is, however, impossible to develop a stand-alone, objective numerical algorithm for the evaluation of academic activities, because any evaluation necessarily includes subjective preference statements. In a market, the market prices represent preference statements, but scientists work largely in a non-market context. I propose a numerical algorithm that serves to determine the distribution of reward money in Mexico's evaluation system, which uses relative prices of scientific goods and services as input. The relative prices would be determined by an evaluation committee. In this way, large evaluation systems (like Mexico's Sistema Nacional de Investigadores) could work semi-automatically, but not arbitrarily or superficially, to determine quantitatively the academic performance of scientists every few years. Data of 73 scientists from the Biology Institute of Mexico's National University are analyzed, and it is shown that the reward assignation and academic priorities depend heavily on those preferences. A maximum number of products or activities to be evaluated is recommended, to encourage quality over quantity.

Avaliação acadêmica, ideologia e poder / Evaluation, ideology and power / Évaluation académique, idéologie et pouvoir / Evaluación académica, ideología y poder

O artigo discute a avaliação acadêmica no contexto caracterizado pelo produtivismo. Como introdução, analisa o episódio da lista dos improdutivos, ocorrido em 1988 na USP, procurando apontar algumas tendências do debate sobre avaliação à época. Em seguida, apresenta, em linhas gerais, ideias que constituem a ideologia produtivista e examina seus desdobramentos na construção de um sistema nacional de avaliação no Brasil. Por último, considera os efeitos desse sistema no trabalho acadêmico, tomando como referência o lugar das exigências de publicação no processo de avaliação.

The article discusses the academic evaluation in the context characterized by high productivity. As an introduction, examines the episode of the unproductive list, which ocurred at USP in 1988, trying to point out some trends of the debate on evaluation at the time. It then presents, in general, ideas that constitute the productivist ideology and examines its implications in the construction of a national evaluation system in Brazil. Finally, consider the effects of this system in academic work, taking as reference publication requirements in the evaluation process.

Cet article traite de l’évaluation académique dans le contexte caractérisé par le productivisme. Dans l’introduction, on analyse l’épisode de la liste des improductifs, qui s’est tenu à l’Université de São Paulo en 1988, en essayant de souligner quelques tendances du débat sur l’évaluation dans l’époque. Ensuite, on présente, en général, des idées qui constituent l’idéologie productiviste et on examine ses implications dans la construction d’un système national d’évaluation au Brésil. Pour finir, on considère les effets de ce système dans le travail académique, en prenant pour référence le lieu des exigences de publication dans le processus d’évaluation.

El artículo discute la evaluación académica en el contexto caracterizado por el productivismo. Como introducción, analiza el episodio de la lista de los improductivos, ocurrido en 1988 en la USP, buscando apuntar algunas tendencias del debate sobre evaluación en la época. Enseguida, en líneas generales, presenta las ideas que constituyen la ideología productivista, y examina sus consecuencias en la construcción de un sistema nacional de evaluación en el Brasil. Por último, considera los efectos de este sistema en el trabajo académico, tomando por referencia el lugar de las exigencias de publicación en el proceso de evaluación.

Avaliação entre 'pares' na ciência e na academia: aspectos clandestinos de um julgamento nem sempre científico, acadêmico ou 'de avaliação' / Peer reviews in science and academy: aspects of a clandestine evaluation not always scientific, academic or assessment / L’évaluation entre 'paires' dans un contexte scientifique et universitaire: aspects d’un essai clandestin pas toujours scientifique, universitaire ou 'd’évaluation' / Evaluación entre pares en la ciencia y en el academia: aspectos de un juicio clandestino no siempre científico, académico o 'de evaluación'

A avaliação do trabalho acadêmico não pode ser reduzida apenas a medidas. Menos ainda reduzida a medidas de alguns aspectos. Há, porém, uma extensa “naturalização” de procedimentos que não são orientados por todos os aspectos que constituem essa “avaliação”. Um dos aspectos dessa “naturalização” tem sido a lenta e sutil substituição do debate e da crítica aos trabalhos acadêmicos pela sua simples exibição, apresentação ou publicação com exames mais burocráticos do que acadêmicos. As “avaliações entre pares” são geralmente exigência ou procedimento feito com problemas ainda não suficientemente resolvidos. Eles incluem o anonimato do avaliador com muitos “desvios” de uma avaliação acadêmica para uma avaliação política, ideológica ou pessoal. O debate, dessa forma, fica facilmente distorcido e as preferências do examinador (teóricas, ideológicas, políticas etc.) tendem a predominar e até serem perigosamente eliminatórias de trabalhos inovadores ou de uma corrente de contribuições diferente ou não popular no meio científico.

The assessment of academic work cannot be reduced to the measures. But there is an extensive “naturalization” of procedures that are not driven every respect constituting this “evaluation”. One aspect of this “naturalization has been slow and subtle substitution of debate and criticism of academic studies for its simple display, publication or presentation with examination more administrative than academic. The “peer reviews” has been a requirement with problems. They include the anonymity of the appraiser and it allows many “deviations” for an assessment of political, ideological or even personal. The tendency toward hegemony of groups is one of the problems. The recent proliferation of academic societies (or scientific), academic or professional journals, the publication requirements (number of titles) as a criterion for contests, promotions, financial aid or scholarships has increased the bureaucracy to administer it with a possible prejudice to the existing distortions to the development of Science.

L’évaluation du travail universitaire ne se réduit pas seulement aux mesures. Un aspect de cette évaluation est la substitution lente et subtile du débat et la critique des études universitaires pour son affichage, publication ou présentation simple. Les évaluations par les pairs ont divers problèmes. Ils comprennent l’anonymat de l’évaluateur et il permet une évaluation politique, idéologique ou même personnel. Le débat devient ainsi facilement faux et les préférences théorique, idéologique, politique etc. tendent à prédominer et à être dangereusement contraire aux oeuvres qui sont innovateurs ou qui ne font part d´un flux de contributions connue ou populaire au milieu scientifique. L´hégémonie des groupes est l’un des problèmes. La prolifération des sociétés scientifiques, des revues académiques, l’obligation de publication en tant que critère pour les concours, promotions, aide financière, bourses etc. a augmenté la bureaucratie et distorsions au développement scientifique.

La evaluación del trabajo académico no puede reducirse a medidas. Un aspecto de esta evaluación ha sido la sustitución del debate y de la crítica por su exhibición, simple publicación o presentación con una especie de examen más administrativo que académico. Las “revisiones por pares” han sido un requisito o procedimiento que se realiza con problemas. Las “evaluaciones” incluyen el anonimato del evaluador y permiten una evaluación política, ideológica o incluso personal. Las preferencias teóricas, ideológicas, políticas, etc. tienden a predominar. La hegemonía de los grupos es uno de los problemas. La reciente proliferación de sociedades científicas, revistas académicas, requisitos de publicación (cantidad) como criterio para concursos, promociones, ayuda financiera, becas, etc. ha aumentado la burocracia e contiene distorsiones que pueden llevar a un perjuicio posible por el desarrollo de la Ciencia.

Avaliação entre 'pares' na ciência e na academia: aspectos clandestinos de um julgamento nem sempre científico, acadêmico ou 'de avaliação' / Peer reviews in science and academy: aspects of a clandestine evaluation not always scientific, academic or assessment / L’évaluation entre 'paires' dans un contexte scientifique et universitaire: aspects d’un essai clandestin pas toujours scientifique, universitaire ou 'd’évaluation' / Evaluación entre pares en la ciencia y en el academia: aspectos de un juicio clandestino no siempre científico, académico o 'de evaluación'

A avaliação do trabalho acadêmico não pode ser reduzida apenas a medidas. Menos ainda reduzida a medidas de alguns aspectos. Há, porém, uma extensa “naturalização” de procedimentos que não são orientados por todos os aspectos que constituem essa “avaliação”. Um dos aspectos dessa “naturalização” tem sido a lenta e sutil substituição do debate e da crítica aos trabalhos acadêmicos pela sua simples exibição, apresentação ou publicação com exames mais burocráticos do que acadêmicos. As “avaliações entre pares” são geralmente exigência ou procedimento feito com problemas ainda não suficientemente resolvidos. Eles incluem o anonimato do avaliador com muitos “desvios” de uma avaliação acadêmica para uma avaliação política, ideológica ou pessoal. O debate, dessa forma, fica facilmente distorcido e as preferências do examinador (teóricas, ideológicas, políticas etc.) tendem a predominar e até serem perigosamente eliminatórias de trabalhos inovadores ou de uma corrente de contribuições diferente ou não popular no meio científico.(AU)

The assessment of academic work cannot be reduced to the measures. But there is an extensive “naturalization” of procedures that are not driven every respect constituting this “evaluation”. One aspect of this “naturalization has been slow and subtle substitution of debate and criticism of academic studies for its simple display, publication or presentation with examination more administrative than academic. The “peer reviews” has been a requirement with problems. They include the anonymity of the appraiser and it allows many “deviations” for an assessment of political, ideological or even personal. The tendency toward hegemony of groups is one of the problems. The recent proliferation of academic societies (or scientific), academic or professional journals, the publication requirements (number of titles) as a criterion for contests, promotions, financial aid or scholarships has increased the bureaucracy to administer it with a possible prejudice to the existing distortions to the development of Science.(AU)

L’évaluation du travail universitaire ne se réduit pas seulement aux mesures. Un aspect de cette évaluation est la substitution lente et subtile du débat et la critique des études universitaires pour son affichage, publication ou présentation simple. Les évaluations par les pairs ont divers problèmes. Ils comprennent l’anonymat de l’évaluateur et il permet une évaluation politique, idéologique ou même personnel. Le débat devient ainsi facilement faux et les préférences théorique, idéologique, politique etc. tendent à prédominer et à être dangereusement contraire aux oeuvres qui sont innovateurs ou qui ne font part d´un flux de contributions connue ou populaire au milieu scientifique. L´hégémonie des groupes est l’un des problèmes. La prolifération des sociétés scientifiques, des revues académiques, l’obligation de publication en tant que critère pour les concours, promotions, aide financière, bourses etc. a augmenté la bureaucratie et distorsions au développement scientifique.(AU)

La evaluación del trabajo académico no puede reducirse a medidas. Un aspecto de esta evaluación ha sido la sustitución del debate y de la crítica por su exhibición, simple publicación o presentación con una especie de examen más administrativo que académico. Las “revisiones por pares” han sido un requisito o procedimiento que se realiza con problemas. Las “evaluaciones” incluyen el anonimato del evaluador y permiten una evaluación política, ideológica o incluso personal. Las preferencias teóricas, ideológicas, políticas, etc. tienden a predominar. La hegemonía de los grupos es uno de los problemas. La reciente proliferación de sociedades científicas, revistas académicas, requisitos de publicación (cantidad) como criterio para concursos, promociones, ayuda financiera, becas, etc. ha aumentado la burocracia e contiene distorsiones que pueden llevar a un perjuicio posible por el desarrollo de la Ciencia.(AU)

Measuring scientists’ performance: a view from organismal biologist / Midiendo el desempeño de los científicos: un punto de vista de biólogos organísmicos / Medindo desempeenho dos cientistas: um ponto de vista de biólogos organísmioss

Increasingly, academic evaluations quantify performance in science by giving higher rank to scientists (as well as journals and institutions) who publish more articles and have more citations. In Mexico, for example, a centralized federal agency uses such bibliometric statistics for evaluating the performance of all Mexican scientists. In this article we caution against using this form of evaluation as an almost exclusive tool of measuring and comparing scientists’ performance. We argue that from an economic viewpoint, maximizing the number of journal articles and their citations does not necessarily correspond to the preferences and needs of society. The traditional peer review process is much better suited for that purpose, and we propose "rule-based peer review" for evaluating a large number of scientists.

En la ciencia hay una fuerte tendencia global de cuantificar el desempeño de los científicos (así como a las revistas e instituciones), dando mayor jerarquía a aquellos científicos que publican más artículos y son más citados. En México, por ejemplo, una institución federal centralizada usa tales estadísticas bibliométricas para evaluar el desempeño de todos los científicos del país. En este artículo advertimos sobre los inconvenientes de esta forma de evaluación como una herramienta casi única para medir y comparar el desempeño de los científicos. Argumentamos que, desde un punto de vista económico, la maximización del número de artículos científicos y de la frecuencia de sus citas no necesariamente corresponde a las preferencias y necesidades de la sociedad en general. El proceso tradicional de arbitraje por pares es más adecuado para este propósito, y proponemos el "arbitraje por pares basado en reglas" para evaluar a un número alto de científicos.

Há uma forte tendência global para avaliações acadêmicas que quantifiquem o desempenho nas ciências através de ranquear os cientistas (assim como revistas e instituições) que publicam mais artigos e têm mais citações. No México, por exemplo, um órgão centralizado do governo utiliza tais estatísticas bibliomêtricas para avaliar o desempenho de todos os cientistas mexicanos. No presente artigo, chamamos atenção ao uso desta forma de avaliação como ferramenta quase que exclusiva para medir e comparar o desempenho dos cientistas. Argumentamos de um ponto de vista econômico que maximizar o número de artigos e as suas citações não corresponde necessariamente às preferências e necessidades da sociedade. O processo tradicional de avaliação pelos pares é mais apropriado para esta finalidade, e propomos um sistema baseado em "avaliação pelos pares seguindo regras", que podem avaliar um grande numéro de cientistas.

Analysis on Applying H -index to Evaluate the Quality of Academic Journals / 医学信息学杂志

Applying H-index combined with other evaluation index of qualitative and quantitative evaluation for seientifie research ability, levels of researchers as well as academie journals, is becoming inereasingly concerned and will gradually be applied. The paper discusses the connotation, extension, feasibility, limitations on H-index and further analyzes the applicability of using H-index to evaluate the quality of academic journals.