Nowadays, we have a wide debate regarding the systems of assessment of research, researchers, projects, groups and, even, institutions. Among the more frequents criticism there is the fact that the systems of assessment and finance give a very high value to the measures derived from the impact factor (IF) of the journals. The judgments to this discrimination indicator for article citations of a journal in a determined period (Beltrán, 2017), are going in various directions:

1. The impact factor does not correct the problems derived from the citation distribution, that tend to be biased in many ways, the sources of the cites are highly diverse, and in a journal there are not only articles derived from research or reviews. Nonetheless, this indicator only takes into account this type of publications, what usually distorts it (Rossner, Van Epps, & Hill, 2008).

2. The impact factor can be manipulated by editorial policies, for example, decreasing considerably the number of articles, or, as editorial houses having platforms with hundreds of journals do: they recommend or suggest articles on their platforms (this evidently can be assessed as a service of the editorial emporiums to the researchers, though it clearly bias the citations). A journal that publishes few articles per year and receive a medium number of cites cannot be compared to one that publishes a high number of articles per year. This aspect made them unequitable.

3. This indicator contrasts in a naive way journals with robust editorial infrastructures, with high financial and technological resources, and decades of history of their positioning in communities, with university journals that are barely trying to consolidate themselves.

4. The impact factor data are not always transparent and because of it the simulations that the editors make, frequently had thrown results that are incomparable.

5. The impact factor does not compile cites that are in different knowledge products, that could inform about other impacts.

Due to these reasons, diverse declarations have stated the need to even abolish from the assessment the measures of impact derived from the use of knowledge related to citations (Seglen, 1997a, 1997b). Nonetheless, there are authors that consider that the impact factor could be improved based on the pointed critics and it could be taken as part of a wider system of scientometrics measures (Lee, Hah, & Kim, 2018). Measures should not be taken by the global indicators of the journal in a period of time, which usually is random and not standardized according to the dynamics of communication and uses of each community, but introducing measures that inform about each research line, and even, each article and researcher. The development of these journals’ virtual platforms is allowing other uses of knowledge, that overflow citation and should be included, as it should be considered (as, frequently, it has been insisted in the last years) Altmetrics measures. Nevertheless, it is necessary to break the scheme that the citation measures are equivalent to the ones of research and publication quality.

The assessment systems should take over the costs of international expert panels that contribute to this type of valuations, so that with a careful reading of the products and with quality criteria for research, impact, methodological rigor, theoretical scope, pertinence and potential of social impact, depending on the case; they allow to generate information more linked to quality than to the usage story and the characteristics of the indicators of global impact of the journals. These reviews should be transparent and be related to transformations: in the cultural practices of the academic communities, in the management of the institutions responsible for the assessment, and in the decisions about financing a research. What is clear, is that the practices that we currently have are creating, even, corrupt behaviors in the researchers communities as Weingart (2005) and Wilhite and Fong (2012) pointed out.

Without any doubt, we have many dares to improve the processes of assessment of research and the researchers’ production; we should make strict critical analysis of them and create proposals to enhance it.

References

Additional information

How to cite: López-López, W. (2018). Regarding the Assessment of Research and Researchers: Criticism to Metrics and Recommendations. Universitas Psychologica, 17(4). https://doi.org/10.11144/Javeriana.upsy17-4.seii