Assessing residents by direct observation is the preferred assessment method for infrequently encountered subspecialty topics, but this is logistically challenging.
We developed an assessment framework for internal medicine (IM) residents in subspecialty topics, using tuberculosis diagnosis for proof of concept.
We used a 4-step process at 8 academic medical centers that entailed (1) creating a 10-item knowledge assessment tool; (2) pilot testing on a sample of 129 IM residents and infectious disease fellow volunteers to evaluate validity evidence; (3) implementing the final tool among 886 resident volunteers; and (4) assessing outcomes via retrospective chart review. Outcomes included tool score, item performance, and rates of obtaining recommended diagnostics.
Following tool development, 10 infectious disease experts provided content validity. Pilot testing showed higher mean scores for fellows compared with residents (7 [SD = 1.8] versus 3.8 [SD = 1.7], respectively, P < .001) and a satisfactory Kuder-Richardson Formula 20 (0.72). Implementation of the tool revealed a 14-minute (SD = 2.0) mean completion time, 61% (541 of 886) response rate, 4.4 (SD = 1.6) mean score, and ≤ 57% correct response rate for 9 of 10 items. On chart review (n = 343), the rate of obtaining each recommended test was ≤ 43% (113 of 261), except for chest x-rays (96%, 328 of 343).
Our assessment framework revealed knowledge and practice gaps in tuberculosis diagnosis in IM residents. Adopting this approach may help ensure assessment is not limited to frequently encountered topics.
This research output has an Altmetric Attention Score of 7. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 01 August 2021.
All research outputs
#6,149,022
of 29,477,053 outputs
Outputs from Journal of Graduate Medical Education
#696
of 2,054 outputs
Outputs of similar age
#94,793
of 348,703 outputs
Outputs of similar age from Journal of Graduate Medical Education
#25
of 47 outputs
Altmetric has tracked 29,477,053 research outputs across all sources so far. Compared to these this one has done well and is in the 79th percentile: it's in the top 25% of all research outputs ever tracked by Altmetric.
So far Altmetric has tracked 2,054 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 12.1. This one has gotten more attention than average, scoring higher than 65% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 348,703 tracked outputs that were published within six weeks on either side of this one in any source. This one has gotten more attention than average, scoring higher than 72% of its contemporaries.
We're also able to compare this research output to 47 others from the same source and published within six weeks on either side of this one. This one is in the 46th percentile – i.e., 46% of its contemporaries scored the same or lower than it.
