The institutional dilemma: Assessing impact vs. mitigating risk

For academic research administrators and institutional leadership, the core mandate is twofold: drive high-value, real-world impact while ensuring that institutional resources are allocated to reputable, trustworthy science.

The current ecosystem, however, often creates a misalignment. The pressure to demonstrate “impact” via metrics, whether traditional citations or altmetrics, can inadvertently incentivize, or at least reward, quantity over quality. Traditional bibliometric indicators (e.g., Journal Impact Factor, h-index) are well-established, but they are proving insufficient for catching sophisticated integrity issues that can ultimately damage an institution’s reputation and waste scarce funding.

The question we must ask our assessment frameworks is simple: Are we rewarding genuine societal contribution, or just the appearance of it?

When high impact hides low integrity

A recent in-depth analysis, undertaken by Federica Silvi, Product Manager at Altmetric, Digital Science, underscores this urgency. Silvi detailed her work in the piece Questionable Research with High-Profile Attention. The analysis found that questionable research networks are successfully gaining traction in the ultimate impact arena: policy documents and clinical practice guidelines. This demonstrates that the system designed to celebrate impact is simultaneously creating an avenue for low-quality or fraudulent work to achieve a veneer of legitimacy that can be leveraged for career advancement and further grant acquisition.

Altmetric’s forensic capabilities

Institutions must move beyond simply focusing on high scores, and proactively address this risk. They must leverage data that allows for forensic due diligence to ensure integrity. While no single tool offers a complete solution to fraud, comprehensive attention tracking provides crucial data to help assessment teams rapidly identify and investigate suspicious patterns.

The Altmetric data set allows teams to detect the signals of artificial legitimacy by enabling a deeper, more contextual dive:

• Source Filtering: Instead of relying on a high score, assessment teams can filter attention sources to detect risk. Is the impact narrative inflated by a suspicious cluster of attention?
• Integrity Signal Detection: Altmetric data helps flag potential issues by allowing cross-referencing of impact against integrity signals. For example, institutions can track whether a highly cited paper has also been the subject of scrutiny on platforms known for research ethics reporting.
• Contextualizing Attention: Altmetric data reveals the narrative behind the numbers, helping to distinguish organic influence from an impact record that appears disproportionately focused on high-profile, potentially misleading attention.

Integrity in assessment

Using this granular data empowers institutions to operationalize the principles of Responsible Research Assessment (RRA). When reviewing promotion and tenure cases, or assessing funding applications, Altmetric data provides the evidence to:

1. Vet Attention Quality: Hold high-impact claims to a higher standard by investigating the source of the attention.
2. De-emphasize Volume: Focus evaluation away from a sheer publication count toward the quality and context of the outputs that do gain traction.

The integrity of the scholarly record is a shared responsibility. By integrating forensic attention analysis, institutions can build robust internal safeguards, ensure funding goes to the most trustworthy actors, and protect their reputation against the risks posed by research manipulation.

Ready to move beyond simple celebration to data-driven due diligence?

Request a demo of the Altmetric Explorer today, focusing specifically on how to utilize attention source filtering for research integrity and risk indicators.