This post is authored by Kalmer Lauk, a Bibliometrics Specialist at the University of Tartu in Estonia. Kalmer presented research similar to this post at the 3:AM Conference in late September 2016. To view the related poster, visit [goo.gl/dHEKco].
A few months ago I was walking home from an altmetrics seminar, when I remembered an article in Scientometrics that analyzed publications in the field of dance (Ho et al. 2015). The authors analyzed the distribution of words in article titles to see how the field had changed through the years. I also remembered a similar analysis I did for an oil shale journal (Lauk, 2016). This was especially interesting to me at that time since a researcher at the my altmetrics seminar said that altmetrics are more or less pointless because of the bias towards of “cool” and “hip” titles. And to prove this he said that he will put the word “Kardashian” in the title of his next article. This was all in good humor, but the point was made and it was clear. A good, catchy, cliché or clickbait title is something that will presumably draw attention and a higher Altmetrics Attention Score (Lockwood, 2016).
Soon after I remembered the dance article with its title words analysis, I learned that there is a list of the top 100 academic articles that got the most attention in 2015. Instantly, I put together these two pieces together and came up with an idea to conduct a word in title analysis on the top 100 altmetrics articles.
I knew that the data sample (n=100) is too small for a full academic analysis and for a certain conclusion that non-research related words are used, but the type was just ideal. If catchy words draw more attention than there is not a better sample to test this hypothesis on then the articles with the most attention. The idea seemed even better when I found out that I can repeat the experiment on the top 100 academic articles from 2014 and 2013.
I quickly realized that I must do more than just create a simple and dull list of the most used words in titles. Since a lot of my research is related to citations, I felt the primal call to have citations somewhere somehow. [https://figshare.com/authors/_/3186606]
To have a larger and more comprehensive picture of both alternative attention and academic attention, I went to Web of Science and found the citations to these top 100 articles. To get an even more accurate view to the academic impact of these publications, I looked up the category normalized citation impact (an indicator that normalizes citation counts based upon the “expected citation rate for documents with the same document type, year of publication and subject area”). Results are in Table 1 below and the analysis was done on the last day of June 2016.
Table 1. Bibliometric impact of the Top articles by Attention Score, 2013-2015
|Top 100 articles indexed in Web of Science||Number of top 100 articles with no cites, of those indexed in Web of Science||Average number of citations to the top 100 articles||Average category normalized citation impact of the top 100 articles||Average number of citations to all articles in Web of Science|
It is clear from the bibliometric overview that the top articles by alternative attention have also a notably higher citation per publication rate. Even in their own fields, the impact of these articles is much higher than expected. Naturally, there are articles with a few or no citations which does not immediately mean they don’t have academic value, but maybe have not had the chance to be used or have not been able to exceed their first expected potential.
Altogether, there were 4828 words total in the titles from 2013-2015. You can see the full list of the ten most often used words for each year in Table 2.
Surprisingly, none of the top ten words used in titles were actually cliché! Of course it is up for debate what are the “cliché” words or subjects. Maybe for some they are all in the top ten, but for me they are not there.
I added three more words to my analysis, words which in my opinion are more attractive to readers– anything starting with sex, anything to do with diet or obesity and vaccine. These three have a much lower frequency in the Top 100 titles then the other top ten words. This hints that maybe sex does not sell! I must say, it is amazing how awesomely the 2013-2015 top three words came together – human, mortality, and study.
Table 2. Word usage in the titles of the Top 100 article with the highest Attention Score, by year
|4||diet- (obesity)||4||diet- (obesity)||5||diet- (obesity)||13||diet- (obesity)|
Author-defined “cliché” words in italics.
Though the sample size is too small to draw a substantial conclusion, the results give a hint of the reality. It seems as if the academic articles that have the highest Altmetrics Attention Scores also have a strong academic impact and that they don’t rely very much on the catchiness of the title. But I wanted to know more.
In my opinion, the three most used words in the titles of the top 300 articles with the highest Attention Score from 2013-2015 (human, mortality, and study) are not cliché. In a way, these three and most of the top ten most used words show that we (both academics and non-academic persons) have a healthy curiosity towards research that probes into things that threaten our lives and our environment.
When it comes to academic research, sex does not sell. Yes, the usage of clickbait on the Internet has long ago passed the thin line of good taste and some of this can be seen in research communication (mainly news) (Sumner et al., 2014), but as my small experiment proved, the titles of the most cited articles are similar to those that are tweeted and discussed online the most. Based on this analysis alone, I cannot say for certain that the words that appear in the Top 100 titles are any more likely to occur in that sample than in all articles. That’s an area for further study.
A similar analysis on the titles of the top 1000 publications with the highest Altmetrics Attention Score was displayed at the Poster session at the Altmetrics Conference 3AM on 28th and 29th of September 2016. To see the poster, visit [https://figshare.com/authors/_/3186606].
Ho, H.-C. and Y.-S. Ho (2015) “Publications in dance field in Arts & Humanities Citation Index: a bibliometric analysis”. Scientometrics 105, 1031–1040. DOI: 10.1007/s11192-015-1716-1
Lauk, K. (2016). A bibliometrical analysis of reserch published in Oil Shale. Oil Shale, 33(3), 290-297 DOI: 10.3176/oil.2016.3.07
Lockwood, G. (2016). Academic clickbait: articles with positively-framed titles, interesting phrasing, and no wordplay get more attention online., The Winnower. 3:e146723.36330 DOI: 10.15200/winn.146723.36330
Sumner, P.; Vivian-Griffiths, S.; Boivin, J. et al. (2014) The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ. 349: g7015 DOI: 10.1136/bmj.g7015