Continuously variable rating: a new, simple and logical procedure to evaluate original scientific publications

ABSTRACT

OBJECTIVE: Impact Factors (IF) are widely used surrogates to evaluate single articles, in spite of known shortcomings imposed by cite distribution skewness. We quantify this asymmetry and propose a simple computer-based procedure for evaluating individual articles. METHOD: (a) Analysis of symmetry. Journals clustered around nine Impact Factor points were selected from the medical ‘‘Subject Categories’’ in Journal Citation Reports 2010. Citable items published in 2008 were retrieved and ranked by granted citations over the Jan/2008 - Jun/2011 period. Frequency distribution of cites, normalized cumulative cites and absolute cites/decile were determined for each journal cluster. (b) Positive Predictive Value. Three arbitrarily established evaluation classes were generated LOW (1.3Positive Predictive Value for journal clusters within each class range was estimated. (c) Continuously Variable Rating. An alternative evaluation procedure is proposed to allow the rating of individually published articles in comparison to all articles published in the same journal within the same year of publication. The general guiding lines for the construction of a totally dedicated software program are delineated. RESULTS AND

CONCLUSIONS:

Skewness followed the Pareto Distribution for (1Positive Predictive Values ranged from 24 - 43 percent for over 98 percent of the selected journals in the ISI database. Continuously Variable Rating is shown to be a simple computer based procedure capable of accurately providing a valid rating for each article within the journal and time frame in which it was published.