Understanding attention with sentiment in Medical Affairs

For Medical Affairs teams, “impact” has often been measured by volume: how many times a paper is mentioned, how often it appears in discussions, and where it sits within Share of Scientific Voice (SSoV). Useful as these metrics are, they capture only the surface. They reveal how much attention a publication receives, but fail to distinguish whether that attention is praise, debate, or criticism.

A single social media post can spark a global debate about a clinical trial, and so analysis by volume needs to be matched by a qualitative understanding of what is being said (and by who).

A new study published in Frontiers in Research Metrics and Analytics details a breakthrough AI-driven framework designed to decode the qualitative reality of research impact. As described in the paper, Sentiment analysis of research attention: the Altmetric proof of concept, this innovation moves beyond simple mention counts to reveal the tone and intent behind the conversations surrounding scholarly work.

Traditional sentiment tools frequently fail in scientific or medical contexts. Specialized terminology, nuanced debate, and the relationship between a post and the linked research often lead to misclassification.

Consider this scenario: A Key Opinion Leader (KOL) posts: “What you are saying is wrong, and this paper proves it”.

• Standard sentiment model/AI: Likely flags the language as negative due to the confrontational tone.
• Altmetric’s New Model: Identifies this as positive toward the research. The model is trained to recognize that the paper is being used to support an argument, and how it relates to the conversation.

The new proof-of-concept study demonstrates that by using Large Language Models (LLMs) specifically trained on this distinction, mentions can be classified on a nuanced 7-point scale – ranging from “strong negative” to “strong positive”.

Why this changes the game for medical affairs

This advancement represents a strategic upgrade for three core Medical Affairs pillars:

Forensic KOL Mapping 

Volume indicates visibility; sentiment indicates alignment. The new model captures general sentiment polarity with high reliability, reaching an accuracy of 87.2% when grouping sentiment into negative, neutral, and positive categories. This level of insight allows teams to differentiate between a neutral share and a strong positive recommendation, helping to identify true advocates for the work.

Supporting the Scientific Narrative 

The study found that “Strong Negative” posts often involve explicit disapproval or accusations against the integrity of the research. By filtering mentions by sentiment score, Medical Affairs teams can establish an early warning system. This capability enables teams to quickly assess whether new research is being met with enthusiasm or concern, allowing for proactive communication management.

Contextualized Share of Scientific Voice

This AI-driven approach allows for a move from monitoring to true strategic intelligence. The model was developed using a dataset with a noted prevalence of clinical and medical sciences publications. While this was a limitation for generalizability in the academic study, it suggests the model is well-tuned to the specific nuances of medical discourse rather than generic social media chatter. It is capable of understanding subtle complexities, including sarcasm, irony, and implicit criticism.

From monitoring to strategy

This functionality is already available in Altmetric Explorer. The vision is to provide intuitive visualizations that show sentiment distribution at a glance, displaying the percentage of mentions that are positive, negative, or neutral.

In a crowded digital landscape, the loudest voice doesn’t always win. The most trusted voice does. This evolution in metrics provides the Medical Affairs community with an invaluable resource for navigating the complex landscape of modern scientific communication. Early experience highlights that users pay particular attention to strong negative comments, as these signals reveal emerging points of resistance or misunderstanding. By identifying negative sentiment early, teams can anticipate where the scientific dialogue may diverge from their results and prepare targeted mitigation strategies that reinforce trust and credibility.

Our research into sentiment analysis continues: usage data accumulated in the months since sentiment analysis was added to Altmetric Explorer indicates that users are most interested in Strong Negative messages, potentially using this data to understand any perceived weaknesses in data.

Ready to see what true sentiment impact looks like? Contact the Altmetric team today!