This article is published in Yliopisto, the science magazine of University of Helsinki.
When funding dwindles, the desire to monitor its use increases, and research funding is no exception: each euro invested in science is expected to yield a quick profit for society. What is more difficult to define is what this profit is and how it could be quantified.
Last autumn, Minister of Education and Culture Sanni Grahn-Laasonen sent a letter to Finnish universities, including universities of applied sciences, reprimanding them for “using resources inefficiently”. The letter was met with outrage in the academic world, but is the criticism valid?
It is not, says Professor Osmo Kivinen from the Research Unit for the Sociology of Education at the University of Turku.
Together with his colleagues Juha Hedman and Kalle Artukka, Kivinen recently published three articles delving into the situation of the Finnish research sector and education system in comparison with other countries. According to the articles, Finnish universities are performing perfectly well, if their achievements are considered in relation to the population of Finland and the resources provided to universities.
“We have generated good results in research and education for a reasonable price, while safeguarding equal opportunities in education,” remarks Kivinen.”That’s not bad at all.”
Kivinen feels that common sense is lost completely when Finnish universities are compared to American universities which consistently place at the top of international rankings. The budgets of Finnish universities are fractions of those of their American counterparts. If the funding keeps dwindling at its current rate, Finland may find itself struggling to even keep the University of Helsinki in the global top 100.
“Being the best university in the world is not cheap.”
THE SYSTEM WORKS
According to Minister Grahn-Laasonen, Finnish universities are not cost-effective. Kivinen and his colleagues found no proof of this, in fact they found the contrary.
For their comparison, they selected a total of 33 universities from the Nordic countries and the Netherlands, all of which produce an annual minimum of 1,000 publications in peer-reviewed Web of Science journals. When the number of publications was compared to the funding received by each university, the universities of Helsinki, Turku and Eastern Finland ranked back to back at 10, 11 and 12. They left all Danish and Swedish universities behind them, with the exception of the Karolinska Institutet. Dutch universities dominated the top positions.
In Kivinen and Hedman’s discipline-specific ranking, the status of Finnish research has enjoyed positive trending across the board during this millennium, if calculated based on publication numbers.
Why is Finland placing the blame at the feet of research and education if there is no reason to do so?
“Maybe it’s easier to justify cuts if the system is seen as not working. But it does work – any other country would be overjoyed to have our education system if they could.”
Kivinen believes we are now fixing a machine that is not broken. However, if improvements must be made, they should be based on facts. This would require appropriate analysis by field of science and education. The investments and results could be calculated by researcher, if desired. These results could then be compared with international examples in the same field.
“Politics is increasingly based on guesstimates. But before budgets are cut, facts should be established.”
PUBLICATION OR CITATION?
Minister Grahn-Laasonen based her accusation primarily on the Academy of Finland’s State of Scientific Research 2014 report, which evaluates fields of science by calculating the number of citations articles receive. According to Kivinen, citations primarily describe the visibility of the articles in the research community.
For this reason, Kivinen selected the number of peer-reviewed articles accepted by the Thomson Reuters Web of Science journals as his focus. He believes that the international peer-review process is sufficient proof of quality, and that developments in the number of publications also reflect the development of the field in general.
Otto Auranen, Science Advisor at the Academy of Finland, describes the difference between number of publications and citations as the familiar quality/quantity distinction, in which the number of publications reflects the growth of the amount of scientific knowledge, and citations describe the development of science.
“Being cited in another article speaks of influence. Some publications are forgotten while others continue to collect citations over time, becoming a part of the fundamental research in their fields,” Auranen explains.
RISING CHINA
Finnish publications have been slipping in citation figures despite a growth in the number of publications in esteemed journals.
This doesn’t necessarily mean that we are doing anything wrong. Other countries have just done even better in terms of cited publications. The visibility of Chinese research, for example, has surged during the past few years.
Regardless of whether the focus is on publication numbers or their citation figures, there is little sense in comparing different disciplines with each other. The rate of publications and the accrual of citations vary so dramatically between disciplines that the comparison should be international, but within the same discipline.
IMMEDIATE OR LASTING BENEFIT
What, then, is this highly touted “research impact”? Calculating the numbers and citations of publications mainly describes the scientific impact. But the social and economic impacts are more difficult to trace.
Which is not to say people are not trying. Universities and universities of applied sciences as well as the Ministry of Employment and the Economy are all chasing reliable impact indicators. The Academy of Finland has also launched a project aiming to determine what is known of the social impacts of projects funded by the Academy of Finland.
“In the Netherlands and the UK, research funders have been evaluating the social impact of research for some time,” says Auranen.
Osmo Kivinen’s group at the University of Turku has also begun to look into the social impacts of research.
“There is an expectation that research should yield an almost immediate profit to society. But the true significance of research is often only discovered over the course of several decades," Kivinen points out.
According to an article in April’s edition of Suomen Kuvalehti magazine, when university funding is reviewed next year, the plan is to increase the significance of social and economic impact in allocating funding. This is another motivator to determine how to measure research impact.
LEARNING FROM THE DUTCH
Politicians usually rely on financial incentives. But this may not be effective for researchers. Otto Auranen wrote his doctoral dissertation on the topic two years ago at the University of Tampere.
Auranen emphasises that when he talks about his dissertation, he is not speaking as an official of the Academy of Finland and his research results do not represent the opinions of the Academy. A key finding of the dissertation research was that competition does not significantly increase the profitability of universities.
Auranen examined the impact of competitive funding on the production of research in Australia, Denmark, Finland, Germany, the Netherlands, Norway, Sweden and the UK. His research material covered a period of time from the early 1980s to the early 2000s. Towards the end of this period, the UK was the most competitive environment and Finland was near the middle.
The traditional mode of budgeting is to fund a university according to its number of students. The current trend is moving towards a mode in which funding is granted based on what the university produces.
During the 2010s, Finland has moved towards a more competitive approach, as have many other western countries. The result from Auranen’s comparison, however, is that the competition has not increased the production of research in the long term.
In the Netherlands, research efficiency skyrocketed during a ten-year period when competitive funding did not significantly increase. This efficiency was boosted in particular by good international networks. Auranen considers the Netherlands a success story we could learn from.
IMPENDING VALUE CONFLICT
Competition is one way of choosing the research that should receive funding. Its downsides include the time wasted on applications and evaluations, its temporary nature and the financial insecurity.
Sometimes the profitability indicators start to guide operations in the wrong way. Auranen believes this is one of the risks of the current funding model used by the Ministry of Education and Culture, which may encourage universities to churn out degrees and publications.
“The government should trust that the research community can generate good results. This does not mean that we should return to the old times when researchers could do as they pleased with no questions asked. Universities should have stricter recruitment and research management policies.”
But why does competition not increase research quantity and quality? One explanation is the value conflict between the academic community and politicians. Researchers are used to the idea of conducting research for its own sake. External demands are poorly suited to this principle.
The recently deceased ecology expert, Professor Ilkka Hanski, wrote about this phenomenon in a recent issue of Kanava magazine. According to Hanski, one of the pitfalls of the current research policy is that it puts at risk researchers’ passion for research as well as the trust between them and politicians. This means weaker results and less success.
References in Finnish:
Kivinen, Hedman, Artukka: Suomalaisyliopistot, tutkimus ja maailman kärki?, Tiedepolitiikka 1/2016
Kivinen, Hedman: Suomalaisen korkeakoulutuksen kansainvälinen taso on väitettyä parempi, Yhteiskuntapolitiikka 2/2016
Kivinen, Hedman: Näkökulmia Suomen tieteen kansainväliseen tasoon, Yhteiskuntapolitiikka 1/2015
Auranen: Parantaako rahoituskilpailu tuloksellisuutta? Tampereen yliopiston verkkolehti Alusta! 2014