Citation

Langendorfer SJ. Int. J. Aquatic Res. Educ. 2011; 5(4): 372-375.

Copyright

(Copyright © 2011, Bowling Green State University)

DOI

10.25035/ijare.05.04.02

PMID

unavailable

Abstract

Regular readers of my editorials may recognize my ongoing concern over recent trends in how we evaluate scholarly papers and the scholarly journals in which they are published. In particular, I am concerned about a rush to quantify journals, and thus by implication, the papers contained within. Many academic institutions have begun requiring their faculty to identify such scores as rejection rates, citation indices, and impact factors for the journals in which they publish. As I wrote in the previous issue of IJARE, composite measures such as impact factors are notoriously invalid and unreliable, especially when the means by which they are calculated are not shared transparently by the commercial agencies that formulate and disseminate the calculations. To reiterate, a composite score typically is a single quantity based on a collection of weighted or un-weighted sub-scores. Even if each of the subscores possesses strong validity and reliability, the composite score rarely shares the same level of validity and reliability.

So, what is my specific gripe with this state of rating journals quantitatively? Based on my professional and measurement experience, the quality of research studies and the manuscripts that disseminate research information is highly individu- alized. While the summed quality of individual papers may reflect on the general quality of the journal itself, a journal ought to be and certainly is greater than the sum of its individual papers. A relatively new and specialized niche journal such as the International Journal of Aquatic Research and Education with a mission to publish both research and professional/educational articles ought not be rated and quantified in the same fashion or with the same algorithm as journals with longer histories that represent broader or more, well-regarded disciplines of study. And conversely, the quality of individual papers appearing in a scholarly journal cannot be adequately assessed simply by a journal's impact factor or rejection rate. In fact, it has long been recognized that even the most reputable journals tend to have a bias toward papers reporting statistically significant differences (Rosenthal, 1979). The implication is that the existing literature likely is skewed away from studies and papers that failed to find statistical differences even if their results are accurate.

Language: en