In the first in a two-part blog post, Juergen Wastl, Director of Academic Relations and Consultancy at Digital Science, explains how Altmetric data can support REF 2021 REF submission preparations.
Following the final consultation on the Research Excellence Framework (REF) in mid October 2018, UK higher education institutions (HEIs) are now ramping up preparations for REF 2021 – with the deadline for submissions rapidly approaching in November 2020. In this blog post we will look back at REF 2014 and provide a view on how altmetrics could have assisted in REF 2014 submission preparations and how they can support the preparations for REF 2021. In a second blog post which will be published later this week, we will look in more detail at how the Altmetric Explorer can be used to find relevant input for your submission.
Impact: reach and significance beyond academia
Impact was introduced for the first time in REF 2014 and its definitions (reach and significance beyond academic impact) were previously uncharted territory in the UK research evaluation landscape. Many HEIs found it difficult to bridge the gap between underpinning research and the narrative case for their impact. Gathering evidence of impact is of crucial importance, so any assistance in compiling evidence for case studies that minimise time and effort will be vital.
Would altmetrics have helped in preparing for the REF 2014 submission?
The endpoint of the research lifecycle has more to offer than just citations. Citations as a means of publication-to-publication relationship are well captured, understood and a matter of research in themselves, which brings to light differences of scope and even definition; what is a citation, and are their differences in quality? Altmetrics haven’t been around that long, so there is no long-term analysis showcasing their value. Unsurprisingly, with the breadth and variety of underpinning sources, altmetrics offer a new and innovative look into a much broader picture of the impact of a research output, fulfilling the aim of the impact agenda by demonstrating influence beyond traditional metrics.
Many sources that Altmetric tracks, in particular policy documents and patent citations, were added after REF 2014. At the time of the last REF exercise, the Altmetric Explorer for Institutions, with crucial functionality such as HR/CRIS system synchronisation and reporting facilities, was not available. Also, in recent years and months we have added additional functionality to the Altmetric Explorer which allows for better evaluation and reporting such as Field of Research (FOR) categorization of outputs; analysis by institution for benchmarking, and filters for Open Access and Funders.
For REF 2021, the capabilities of Altmetric Explorer for Institutions makes the task of identifying potential impact beyond academic credentials much easier to carry out. Altmetric can be used to gather evidence to underpin the impact and add to the narrative, which will be beneficial to institutions, particularly in the light of the revised and updated guidance on the next REF which introduced additionality in the context of impact assessment.
Additionality and altmetrics
New on the horizon for the Impact element in REF 2021 is the aspect of additionality, i.e. the potential ‘recycling’ of previous impact case studies in terms of new, additional impact according to the definitions set out in Research England’s working methods and guidance. Additionality of impact can and will form part of the forthcoming REF exercise.
How can Altmetric help in cases that may qualify for recycling in the event of additionality? REF 2014 impact case studies can be re-analysed to determine attention and engagement gathered since the last round of REF, whether they are in the form of news coverage, blogs, policy documents or patent citations. Searching for Altmetric mentions using a case study’s DOI (or indeed set of DOIs) and superimposing the results onto the same search carried out at the time of REF 2014 gives an indication of the altmetrics gathered since the last REF exercise.
Qualitative aspects of altmetrics
Consistent with the view that the Altmetric Attention Score is not a measure for, or indicator of, the quality of the mentioned scholarly output, the same is true for the individual underpinning sources. However, Altmetric does provide a diverse, robust and timely tool to look at impact beyond academic citations via a range of channels.
Looking at an individual researcher’s publication(s) and analysing the portfolio with the Altmetrics Explorer yields interesting insights that may help impact coordinators and administrators to narrow down or pinpoint new leads with respect to additionality. This is one way to gain an emerging picture of impact beyond academic citations.
Mentions and citations later in the life cycle of a scholarly output may lead to an investigation into why a paper published years ago now features in the news. Is it because of a patent or policy document? Why is a movie clip citing the DOI now? These questions can be answered with the Altmetric Explorer to find leads and track down further evidence to underpin impact case studies. Read our next blog post which will be published tomorrow to learn more or contact us to schedule some time for a chat.