To characterize reporting of P values, confidence intervals (CIs), and statistical power in health professions education research (HPER) through manual and computerized analysis of published research reports.
The authors searched PubMed, Embase, and CINAHL on May 7, 2016, for comparative research studies. For manual analysis of abstracts and main texts, they randomly sampled 250 HPER reports published in 1985, 1995, 2005, and 2015, and 100 biomedical research reports published in 1985 and 2015. Automated computerized analysis of abstracts included all HPER reports published 1970-2015.
In the 2015 HPER sample, P values were reported in 69/100 abstracts and 94 main texts. CIs were reported in 6 abstracts and 22 main texts. Most P values (≥ 77%) were ≤ .05. Across all years, 60/164 two-group HPER studies had ≥ 80% power to detect a between-group difference of 0.5 standard deviations. From 1985 to 2015, the proportion of HPER abstracts reporting a CI did not change significantly (odds ratio [OR] 2.87; 95% CI 1.04, 7.88) whereas that of main texts reporting a CI increased (OR 1.96; 95% CI 1.39, 2.78). Comparison with biomedical studies revealed similar reporting of P values, but more frequent use of CIs in biomedicine. Automated analysis of 56,440 HPER abstracts found 14,867 (26.3%) reporting a P value, 3,024 (5.4%) reporting a CI, and increased reporting of P values and CIs from 1970 to 2015.
P values are ubiquitous in HPER, CIs are rarely reported, and most studies are underpowered. Most reported P values would be considered statistically significant.
This research output has an Altmetric Attention Score of 5. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 31 May 2019.
All research outputs
#9,607,477
of 34,400,738 outputs
Outputs from Academic medicine
#3,208
of 7,711 outputs
Outputs of similar age
#136,681
of 492,476 outputs
Outputs of similar age from Academic medicine
#73
of 108 outputs
Altmetric has tracked 34,400,738 research outputs across all sources so far. This one has received more attention than most of these and is in the 70th percentile.
So far Altmetric has tracked 7,711 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 11.8. This one has gotten more attention than average, scoring higher than 56% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 492,476 tracked outputs that were published within six weeks on either side of this one in any source. This one has gotten more attention than average, scoring higher than 70% of its contemporaries.
We're also able to compare this research output to 108 others from the same source and published within six weeks on either side of this one. This one is in the 32nd percentile – i.e., 32% of its contemporaries scored the same or lower than it.
