↓ Skip to main content

A Faculty Development Program to Reduce Rater Error on Milestone-Based Assessments

Overview of attention for article published in Journal of Graduate Medical Education, December 2014
Altmetric Badge

Mentioned by

twitter
1 X user

Readers on

mendeley
27 Mendeley
You are seeing a free-to-access but limited selection of the activity Altmetric has collected about this research output. Click here to find out more.
Article details
Title
A Faculty Development Program to Reduce Rater Error on Milestone-Based Assessments
Published in
Journal of Graduate Medical Education, December 2014
DOI 10.4300/jgme-d-14-00161.1
Pubmed ID
Authors
Abstract

Rater errors, such as halo/reverse halo, range restriction, and leniency errors, are frequently cited as threats to the validity of resident assessment by faculty. We studied whether participation in faculty development on the use of a new Milestone-based assessment tool reduced rater error for participants compared to individuals who did not participate. We reviewed evaluations of resident Milestones completed by faculty at the end of rotations between July 2012 and June 2013. The 2 Milestones in each competency with the greatest number of ratings were selected for analysis. A total of 412 evaluations were analyzed, including 217 completed by faculty who participated in the development activity, and 240 completed by nonparticipant faculty. All evaluations that contained identical scores for all Milestones (16%) were completed by nonparticipant faculty (χ(2)  =  37.498, P < .001). Faculty who had participated in development assigned a wider range of scores and lower minimum scores to residents, and provided the highest ratings for residents less frequently (P < .001) than nonparticipants. Faculty who participated in education about the Milestones demonstrated significantly less halo, range restriction, and leniency errors than faculty members who did not participate. These findings support a recommendation to develop a cadre of "core faculty" by training them in the use of Milestone assessment tools, and making them responsible for a significant portion of resident assessments.

Login to access the Attention Digest and the Sentiment Analysis related to this output.

Timeline Attention over time Attention Score history
Login to access the full charts related to this output.
Activity
Login to access the full charts related to this output.
X Demographics

X Demographics

The data shown below were collected from the profile of 1 X user who shared this research output. Click here to find out more about how the information was compiled.
Mendeley demographics

Mendeley demographics

The data shown below were compiled from readership statistics for 27 Mendeley readers of this research output. Click here to see the associated Mendeley record.
Login to view Mendeley reader trends over time.

Geographical breakdown

Geographical breakdown
Country Count As %
Unknown 27 100%

Demographic breakdown

Readers by professional status
Readers by professional status Count As %
Student > Postgraduate 4 15%
Other 3 11%
Researcher 3 11%
Student > Doctoral Student 2 7%
Professor 2 7%
Other 8 30%
Unknown 5 19%
Readers by discipline
Readers by discipline Count As %
Medicine and Dentistry 16 59%
Nursing and Health Professions 2 7%
Social Sciences 2 7%
Psychology 1 4%
Physics and Astronomy 1 4%
Other 0 0%
Unknown 5 19%
Attention Score in Context

Attention Score in Context

This research output has an Altmetric Attention Score of 1. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 04 December 2014.
All research outputs
#20,655,488
of 25,373,627 outputs
Outputs from Journal of Graduate Medical Education
#1,736
of 1,909 outputs
Outputs of similar age
#274,275
of 369,122 outputs
Outputs of similar age from Journal of Graduate Medical Education
#29
of 35 outputs
Altmetric has tracked 25,373,627 research outputs across all sources so far. This one is in the 10th percentile – i.e., 10% of other outputs scored the same or lower than it.
So far Altmetric has tracked 1,909 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 13.1. This one is in the 1st percentile – i.e., 1% of its peers scored the same or lower than it.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 369,122 tracked outputs that were published within six weeks on either side of this one in any source. This one is in the 14th percentile – i.e., 14% of its contemporaries scored the same or lower than it.
We're also able to compare this research output to 35 others from the same source and published within six weeks on either side of this one. This one is in the 1st percentile – i.e., 1% of its contemporaries scored the same or lower than it.