Altmetrics for research evaluation

Posted by Misha Kidambi on 29th August 2023

In a recent paper, researchers have unveiled a new framework that uses altmetrics to strengthen university research evaluation. Altmetrics, or alternative metrics, provide a means to measure the attention garnered by scholarly work beyond traditional citation counts. The paper, titled “Evaluative altmetrics: is there evidence for its application to research evaluation?,” introduces an approach that applies altmetrics to identify unique audiences, local attention, engagement metrics, and mentions and the profile of the actors involved in the mentions. 

By integrating tools like Botometer and Site Traffic API, which are not commonly used in altmetric studies, the researchers demonstrate how the quality and context of altmetrics data can be used for evaluating research. “One of the most relevant contributions [of the study] is that it illustrates in a practical way how indicators can be combined with unique sources of information to allow contextualization of attention,” they write. The team proposes the creation of dashboards at both publication and author levels, offering comprehensive insights into measured attention while segmenting different instances of social engagement. According to the team, the proposed framework can be effectively employed by institutions and evaluation agencies for informed decision-making.

However, the paper does acknowledge certain limitations. Data distribution discrepancies across social media platforms, potential biases in news mentions, and inaccuracies in identifying audiences using machine learning techniques are some challenges. Moreover, it highlights the need for open access to data sources to facilitate wider implementation.  

The paper concludes with a call to utilize altmetrics within evaluation contexts. The team states that while altmetrics is a considerable improvement over traditional bibliometrics, it should complement and not replace, established peer-review processes. This approach echoes the European Union’s stance and underscores the need to integrate alternative metrics into the evaluation landscape seamlessly.

Curious to learn more about Altmetric? Find out more about our data or contact the Altmetric team!