Altmetric Blog

Interactions: An Uncontrolled Proliferation of Hype

Jean Liu, 6th December 2012

Spinning until we’re dizzy

For years, news media outlets have been pushing countless claims about cancer-causing agents in our environment. Many of these claims signal real danger, but others, like visiting the toilet at night, are complete hogwash. But after being bombarded by an constant stream of bad news, one could be forgiven for developing mild hypochondriasis. When combined with Wikipedia, medical advice forums, and various unreliable medical websites, it isn’t hard to imagine why the word “cyberchondria” now exists.

When it comes to cancer reporting, the misrepresentation of research and inflation of risk have become such large problems that improving cancer risk communication in the media is now a research topic in its own right. A subject that seems to be a media favourite is that of potential carcinogens in our food, and as such, the incessant reporting of cancer risk creates the impression that nothing is safe to eat. Unfortunately, the overblown fears of potential food carcinogens are unintentionally fed by the way in which information is disseminated within the field of nutritional epidemiology. Inherent to the scientific method is the quality that studies don’t yield firm, set-in-stone answers: instead, statistical tests are used to determine the likelihood that a given result occurred by chance. However, referring to probabilities can lead to confusion in the general public when the results of different studies appear to be in conflict with each other. Moreover, when newly-published, single studies are viewed outside the broader research context (i.e., without comparison to other studies), reported cancer risks are in danger of being inflated by the media and other science communicators.


Food for (rational) thought

Last week, a refreshing, clarifying perspective on the research area of food carcinogens appeared in a new paper entitled “Is everything we eat associated with cancer? A systematic cookbook review”. The paper, written by Jonathan Schoenfeld and John Ioannidis and published in the American Journal of Clinical Nutrition, randomly selected 50 ingredients from recipes in The Boston Cooking-School Cook Book (which, amusingly, was published in 1911) and searched for clinical studies which looked at the individual ingredients’ cancer risk. Of the 50 chosen ingredients, 40 of these had been examined within studies in the scholarly literature. The researchers found that individual clinical studies were far more likely to report an increase or decrease in cancer risk, but that these results were typically only weakly statistically significant. Unsurprisingly, positive results received much more emphasis than negative ones. Furthermore, individual studies showed greater effect sizes compared to those determined from the results of many studies (meta-analyses). This is consistent with the idea that news reports which only pull information from a single study are in danger of exaggerating the actual cancer risk.

Within several days after its publication online, the Schoenfeld and Ioannidis paper had received a lot of attention, primarily on Twitter. Most people praised the article, citing its importance in debunking the idea that “everything causes cancer”:

Tweeters hailing from Great Britain made cheeky references to the UK-based paper The Daily Mail, which in recent years has gained a bit of an online reputation for obsessively reporting about cancer risk:

Scepticism about the entire field of nutritional epidemiology also fueled a lot of the Twitter activity. Ed Yong, who blogs at Not Exactly Rocket Science, referenced the Schoenfeld and Ioannidis paper while making a blunt remark about the field as a whole:


Barriers to unbiased reporting

What makes it so difficult to obtain a balanced view of cancer risk? Arguably, the problem arises from both the scientific reporting (e.g., publication bias towards positive, significant results) and in the subsequent news reporting. Journalists are not necessarily experts in the details of the scientific method or statistics, and therefore single studies with seemingly profound findings might receive a disproportionate amount of attention. Meta-analyses might show more dilution in the reported effects (thus appearing less glamorous when presented in popular media), but provide a more comprehensive overall view – as such, these types of studies are generally more useful for looking at broad trends.

Another problem lies in the inaccessibility of research to members of the general public. Instances in which news stories link to research articles are rare, and consequently, readers usually can’t find the research papers and abstracts in question. Bloggers tend to be better at linking to research articles, and those with extensive science backgrounds are thus better positioned to provide balanced views on new findings. But for all science communication outlets, whether they are news sources or blogs, the audience should be able to find the actual paper in order to be able to personally assess the research findings if so desired. Tangential to this is the issue of publisher paywalls (and open access), but the important point here is still transparency. Providing a link to the research paper may help to prevent the public’s reliance on media hype for assessing cancer risk. (And, as a side note, links to research papers also improve the accuracy of alt-metrics, which in turn could help readers to filter out key studies from large volumes of research.)


An alt-metric food analysis

It probably comes as no surprise that diet and health are hugely popular topics in social media mentions of scholarly articles. With respect to cancer-related scholarly articles, we wondered which food ingredients (specifically, which among the 20 ingredients listed in Figure 1 of the Schoenfeld and Ioannidis study) attract more scholarly attention online. While exploring the Altmetric database, we kept the criteria broad and counted all papers with titles indicating that the effects of a food ingredient (including derivatives or chemical components) on a cancer-related endpoint were examined. Meta-analyses and animal studies were also included. The following search string was used for all database queries (with bacon as an example):

bacon AND (cancer OR carcinogenesis OR oncology OR tumour OR tumor)

The number of relevant scholarly articles that matched our criteria was counted for each of the 20 ingredients. (Note that we didn’t analyse the type of association (positive or negative) that an ingredient had with cancer. Below is a graph that ranks the ingredients by the number of associated cancer-related papers. (You can view the dataset on figshare.)

Histogram showing the frequencies of cancer-related papers about 20 ingredients, sampled from the Altmetric database.

Despite the Internet community’s love for bacon, oddly enough, bacon papers didn’t show up frequently in the titles of cancer-related papers picked up by Altmetric. Instead, the world’s most popular caffeine-containing beverages (tea and coffee) appeared in the highest number of cancer-related article titles. Tea dominated the set, appearing in 103 article titles. Coffee came in second, and appeared in 61 titles. Milk was third on the list, and appeared in 28 titles. Keep in mind that these numbers refer only to the quantity of relevant papers being discussed online, not to the Altmetric scores for each ingredient. Individual papers that were particularly popular, such as this coffee one, sometimes comprised a large portion of an ingredient’s cumulative Altmetric score.

Overall, these data indicate which food ingredients are more likely to be mentioned online in the context of cancer. It seems that not every ingredient is being hyped, in terms of alt-metrics. Still, whatever the actual associated risks for these ingredients are, it isn’t very surprising at all that the drinks that start the days of many people also tend to start their conversations online.

5 Responses to “Interactions: An Uncontrolled Proliferation of Hype”

[…] for ways to justify their high consumption of coffee and tea (also suggested by my previous “food altmetrics analysis”), so the fact that this new study in Nature Neuroscience provoked a veritable blizzard of press […]

May 6, 2014 at 12:00 am

I love it when people come together and share ideas. Great blog, keep it up!

cartier love bracelet pink gol
October 13, 2014 at 12:00 am

Wonderful blog! Do you have any tips and hints for aspiring writers?

I'm planning to start my own blog soon but I'm a little lost on everything.

Would you advise starting with a free platform like Wordpress
or go for a paid option? There are so many choices out there that I'm totally confused ..

Any recommendations? Kudos!

Marty Cantua
February 25, 2015 at 12:00 am

Great post!

Lela Hagedorn
November 24, 2015 at 12:00 am

some cheap softwares does not offer good online technical support so i would caution about using them`

Leave a Reply

Your email address will not be published. Required fields are marked *