↓ Skip to main content

The heterogeneity statistic I2 can be biased in small meta-analyses

Overview of attention for article published in BMC Medical Research Methodology, April 2015
Altmetric Badge

About this Attention Score

  • In the top 25% of all research outputs scored by Altmetric
  • High Attention Score compared to outputs of the same age (91st percentile)
  • High Attention Score compared to outputs of the same age and source (99th percentile)

Mentioned by

news
1 news outlet
blogs
1 blog
twitter
11 X users

Citations

dimensions_citation
742 Dimensions

Readers on

mendeley
491 Mendeley
You are seeing a free-to-access but limited selection of the activity Altmetric has collected about this research output. Click here to find out more.
Title
The heterogeneity statistic I2 can be biased in small meta-analyses
Published in
BMC Medical Research Methodology, April 2015
DOI 10.1186/s12874-015-0024-z
Pubmed ID
Authors

Paul T von Hippel

Abstract

Estimated effects vary across studies, partly because of random sampling error and partly because of heterogeneity. In meta-analysis, the fraction of variance that is due to heterogeneity is estimated by the statistic I (2). We calculate the bias of I (2), focusing on the situation where the number of studies in the meta-analysis is small. Small meta-analyses are common; in the Cochrane Library, the median number of studies per meta-analysis is 7 or fewer. We use Mathematica software to calculate the expectation and bias of I (2). I (2) has a substantial bias when the number of studies is small. The bias is positive when the true fraction of heterogeneity is small, but the bias is typically negative when the true fraction of heterogeneity is large. For example, with 7 studies and no true heterogeneity, I (2) will overestimate heterogeneity by an average of 12 percentage points, but with 7 studies and 80 percent true heterogeneity, I (2) can underestimate heterogeneity by an average of 28 percentage points. Biases of 12-28 percentage points are not trivial when one considers that, in the Cochrane Library, the median I (2) estimate is 21 percent. The point estimate I (2) should be interpreted cautiously when a meta-analysis has few studies. In small meta-analyses, confidence intervals should supplement or replace the biased point estimate I (2).

X Demographics

X Demographics

The data shown below were collected from the profiles of 11 X users who shared this research output. Click here to find out more about how the information was compiled.
Mendeley readers

Mendeley readers

The data shown below were compiled from readership statistics for 491 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
United States 1 <1%
Germany 1 <1%
Brazil 1 <1%
Unknown 488 99%

Demographic breakdown

Readers by professional status Count As %
Student > Master 74 15%
Student > Ph. D. Student 71 14%
Student > Bachelor 60 12%
Researcher 53 11%
Other 22 4%
Other 69 14%
Unknown 142 29%
Readers by discipline Count As %
Medicine and Dentistry 114 23%
Psychology 42 9%
Nursing and Health Professions 21 4%
Agricultural and Biological Sciences 20 4%
Biochemistry, Genetics and Molecular Biology 20 4%
Other 108 22%
Unknown 166 34%
Attention Score in Context

Attention Score in Context

This research output has an Altmetric Attention Score of 21. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 28 November 2023.
All research outputs
#1,740,618
of 24,885,505 outputs
Outputs from BMC Medical Research Methodology
#223
of 2,218 outputs
Outputs of similar age
#22,149
of 269,648 outputs
Outputs of similar age from BMC Medical Research Methodology
#1
of 23 outputs
Altmetric has tracked 24,885,505 research outputs across all sources so far. Compared to these this one has done particularly well and is in the 92nd percentile: it's in the top 10% of all research outputs ever tracked by Altmetric.
So far Altmetric has tracked 2,218 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 10.4. This one has done well, scoring higher than 89% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 269,648 tracked outputs that were published within six weeks on either side of this one in any source. This one has done particularly well, scoring higher than 91% of its contemporaries.
We're also able to compare this research output to 23 others from the same source and published within six weeks on either side of this one. This one has done particularly well, scoring higher than 99% of its contemporaries.