National Physical Laboratory

About the organization

The National Physical Laboratory (NPL) is the UK’s national measurement institute, and an applied research laboratory. With over 680 researchers supported by a team of around 200 people, the NPL’s mission is to support the UK economy by making measurement better for industry, and to increase quality of life for the general public.

We spoke to Jenny Wooldridge, a member of the NPL’s Impact Evaluation team, to learn how they’ve been using metrics to look at the performance of their publications.

Exploring the data

Jenny’s team used the Explorer for Institutions platform to run reports on the altmetrics relating to publications submitted as evidence in the REF2014 exercise, either as research outputs or as cited references within impact case studies.

Having noticed a correlation between NPL authored publications with high Altmetric attention scores, and research considered to be “high impact” through internal research evaluations, they were keen to expand the study to a much larger scale investigation to see how well altmetric scores aligned with peer review evaluations of research impact.

What they found

A total of 41,409 DOIs of publications submitted to main panel B (relating to mathematics, engineering and the physical sciences) were assessed in the exercise. The departmental scores for the percentage of 3* and 4* judged research outputs and impact case studies were evaluated with respect to Altmetric attention scores, with controls applied for citation rates (the average number of traditional bibliometric citations accrued per year post publication), the age of the publications, and the research subject areas. Altmetric scores were found to be strongly and positively correlated with the judgement of quality of research impact (the case study scores), whereas a null result was obtained for the judgement of research quality alone (from the research output assessment).

What’s more, altmetrics are able to provide a potential early indicator of research impact as compared with bibliometrics, with the vast majority of mentions occurring within the same year the paper was published.

Where next?

NPL’s research on the REF2014 data provides direct evidence for how the presence of online activity discussing publications is related to the wider societal impact of research, and Jenny plans to publish the results of this work. With respect to the online mentions of NPL’s publications, Jenny is mostly interested to see where work has been picked up by news outlets and within public policy documents, and so they will be further analysing the REF2014 publications to focus on these aspects.