The assessment of the intellectual production of researchers, groups, centres, institutes, institutions, regions and countries, has so many economic and political implications, that making decisions that may improve assessment processes is essentially related to economic, more than scientific, criteria.

That is to say, a poorly designed system can generate costs it cannot account for later, and then needs to take corrective action that is regularly punished from a political standpoint and have a negative impact on output. It can also create collateral damage, such as academic corruption. An example of this is the unfortunate policy taken by systems such as Colciencias, because of its inability to modify a decree that failed to calculate the true cost of paying for the amount of articles to the lifetime salary of researchers, which would consequently compromise the viability of the already precarious and unfortunate situation of the Colombian public universities. This policy of strengthening criteria for journal coverage also created unfortunate endogamic processes and ended in the death of nearly half of all Colombian journals. To this day, nobody has examined the validity and impacts of these criteria on the national scientific output.

On the other hand, although in a complementary way, a late increase in scientific output in the region and the unfortunate perspectives of the first accreditation systems, prompted universities to create journals. This has generated collateral damage, since it compromises autonomy and peer review processes – funding entities do not always understand that journals are symbols of high quality knowledge management, and that they help position the university’s brand and show how the university produces knowledge. In fact, in today’s academic marketing, an image of excellence, leadership, scientific innovation, knowledge dissemination on the student community, income on patents, consulting jobs, spin-offs and research investment is highly relevant. This is why universities with internationally visible journals should take care of their autonomy and use editorial policies which prevent these from losing their scientific character and turn into institutional channels of communication.

But, how do we make decisions that protect, foster and maintain knowledge production? Scientometrics is a useful tool in cases in which, for example, a country needs to improve knowledge transfer processes in order to overcome dependency, or wants to strengthen international collaboration. In such events, co-author analyses are a suitable tool, or promote megajournals instead of carrying many smaller journals and reward the editorial effort giving weighted scores to the journals that carry more articles and not using the impact factor, which punishes those journals publishing more papers. This has led to simple practices such as lowering output in order to increase IJF, SJR, SNIP or the new Scopus impact indicators.

Perhaps another important element in journal assessment should be a determination of authors that become central citation nodes: this can be summarised by centrality measures. These can be supplemented by cluster coefficient measures in order to establish how the knowledge produced by an author, an article or a journal is being used. This is important if an institution or a country finds it relevant to promote interdisciplinary knowledge. Whichever measure is used, it is clear that evidence-based measures that are put in place and evaluated is better than taking simplistic steps based only on the journal quartile or on article-citation ratios.

In summary, scientometrics is creating a number of analytical tools that will make several processes possible: from a qualification of the procedures for finding and hiring researchers and lecturers, to the assessment of their promotion criteria and the incentive policies that operate on their intellectual production (Brynjolfsson & Mitchell, 2017; Brynjolfsson et al., 2015). Needless to say, these tools can also enhance journal rankings, research groups and even large popular university rankings. These tools can be used to improve decision-making processes and decrease subjectivities and one-sided decisions in the systems that give value to intellectual production – as mentioned, these are, to a large extent, based upon political, economic, or personal interests in every level. As I said in a previous editorial, other systems that give weight and importance to other qualitative variables should be used, and their impacts and predictive value should be evaluated.

References

Descargar