Bibliometrics, or publication metrics, is a field that studies research and science using scholarly publications and the information about them as research data. Information on authors and their organisations, publication dates and channels, descriptive keywords and lists of references are examples of information that can be used in bibliometric analyses.
Publication metrics can be used to determine, for example, which researchers collaborate, which topics are investigated in faculties and how frequently publications are cited. Changes and trends can also be monitored, for example, whether general or author-specific publication numbers have changed over the years.
A range of indicators aimed at describing the general characteristics of publications can also be constructed from publication data. The most typical indicators are those describing the citation impact of publications, such as the related impact factor, the number of citations and the h-index. Also used are citation impact indicators that take into consideration the different publication practices of various disciplines. These include Top10% indicators and citation indexes normalised by discipline.
Bibliometric methods and indicators have also been utilised in a range of research and researcher assessments as well as in knowledge-based management. In fact, they can provide a great deal of useful information, especially when looking at the publishing activities of major entities, such as entire universities or their faculties.
The use of bibliometric methods and indicators can be problematic, as bibliometrics requires interpretation. If bibliometric indicators are used as the sole or primary tool of assessment, not taking into consideration, for example, the goals of the researcher or unit being assessed, or the resources available and the discipline, it is easy to draw erroneous conclusions. Using indicators designed for a specific purpose outside their original scope also poses a problem. A typical example is assessing scholarly article quality using indicators designed for assessing scholarly journal quality, such as the publication rating of the Finnish Publication forum or journal-specific impact factors. Individual articles can be of a high quality regardless of the reputation of the journals in which they have been published.
The academic community has awakened to these issues. In the past decade, several recommendations for responsible assessment have been published, commenting also on the use of bibliometrics. These include the Recommendation for the responsible evaluation of a researcher in Finland (2021), the Declaration on Research Assessment (DORA, 2021) and the Agreement on Reforming Research Assessment (2023). The University of Helsinki is committed to these guidelines. In terms of publication metrics, the key message of these recommendations is that qualitative assessment is the primary method of assessing research, while quantitative methods, including bibliometrics, are used to support the conduct of qualitative assessment.
The bibliometrics specialists of Helsinki University Library have contributed to the recommendations on the responsible use of publication metrics included in the ‘Recommendation for the responsible evaluation of a researcher in Finland’. The core message of the recommendations is that all bibliometric analyses must consider the specific characteristics of the object of assessment, such as the publication practices of the relevant discipline. In addition, careful consideration must be given to the questions to be answered in specific assessments. The data and indicators used in analyses must be suited to specific situations.
The recommendations also highlight the importance of openness in responsible publication metrics. For each analysis, the library’s bibliometrics specialists consider how to take the various aspects of the recommendations into consideration as effectively as possible. Openness in analyses can be realised by justifying, for example, the selection of specific data and indicators. Any deficiencies in the data and indicators must be highlighted.
The recommendations also take a stance on bibliometric expertise and training. Responsible use of publication metrics requires familiarity with discipline-specific publication practices, bibliometric indicator properties and the data used in bibliometric analyses. This information is needed by people who conduct such analyses, as well as those who use them.
The Finnish national guide to publication metrics (2022) makes it possible for all users of publication metrics to familiarise themselves with the relevant key concepts as well as the properties of indicators and data sources. The guide is maintained by the Finn-ARMA publication metrics network, which brings together Finnish experts in the field. The bibliometrics specialists of Helsinki University Library contributed to writing the guide and remain involved in updating its cont[LV1] ent. The content of the national guide to publication metrics is also at the core of the bibliometrics training provided to doctoral researchers at the University of Helsinki and the materials tailored to the University community as a whole.
Investigating the applicability of available bibliometric methods to analyses conducted at the University of Helsinki is a significant means for promoting the realisation of responsible publication metrics. A practical example of such investigation is a survey, conducted in 2023 by the library’s bibliometrics group, describing how the publication classifications of various databases linking research to the Sustainable Development Goals of the UN function when applied to the publications of the University of Helsinki. The survey was presented as a poster at the Nordic Workshop on Bibliometrics and Research Policy 2023, a conference for broad discussion on new methods in bibliometrics, including the perspective of responsibility (link to poster).
The chief benefit of bibliometric methods is that they enable the analysis of extensive datasets. Responsibly conducted bibliometric analyses can yield a lot of useful additional information or confirmation in support of qualitative assessment. In fact, the best practices in publication metrics should be promoted with regard to traditional indicators and analyses. New factors to be taken into consideration await us in the near future. Openly available bibliometric databases are in development. How well are these new sources of data suited to analysing publications produced at the University of Helsinki? And what about artificial intelligence? AI-based applications are being developed in many fields, not least publication metrics. To enable the continued use of bibliometrics without compromising on openness and responsibility, this development should be carefully monitored and the ensuing changes taken into consideration responsibly.
Text
Terhi Sandgren
Information Specialist