| Title |
Workplace-based assessment: raters’ performance theories and constructs
|
|---|---|
| Published in |
Advances in Health Sciences Education, May 2012
|
| DOI | 10.1007/s10459-012-9376-x |
| Pubmed ID | |
| Authors |
M. J. B. Govaerts, M. W. J. Van de Wiel, L. W. T. Schuwirth, C. P. M. Van der Vleuten, A. M. M. Muijtjens |
| Abstract |
Weaknesses in the nature of rater judgments are generally considered to compromise the utility of workplace-based assessment (WBA). In order to gain insight into the underpinnings of rater behaviours, we investigated how raters form impressions of and make judgments on trainee performance. Using theoretical frameworks of social cognition and person perception, we explored raters' implicit performance theories, use of task-specific performance schemas and the formation of person schemas during WBA. We used think-aloud procedures and verbal protocol analysis to investigate schema-based processing by experienced (N = 18) and inexperienced (N = 16) raters (supervisor-raters in general practice residency training). Qualitative data analysis was used to explore schema content and usage. We quantitatively assessed rater idiosyncrasy in the use of performance schemas and we investigated effects of rater expertise on the use of (task-specific) performance schemas. Raters used different schemas in judging trainee performance. We developed a normative performance theory comprising seventeen inter-related performance dimensions. Levels of rater idiosyncrasy were substantial and unrelated to rater expertise. Experienced raters made significantly more use of task-specific performance schemas compared to inexperienced raters, suggesting more differentiated performance schemas in experienced raters. Most raters started to develop person schemas the moment they began to observe trainee performance. The findings further our understanding of processes underpinning judgment and decision making in WBA. Raters make and justify judgments based on personal theories and performance constructs. Raters' information processing seems to be affected by differences in rater expertise. The results of this study can help to improve rater training, the design of assessment instruments and decision making in WBA. |
X Demographics
The data shown below were collected from the profiles of 3 X users who shared this research output. Click here to find out more about how the information was compiled.
Geographical breakdown
| Country | Count | As % |
|---|---|---|
| Japan | 1 | 33% |
| Australia | 1 | 33% |
| Unknown | 1 | 33% |
Demographic breakdown
| Type | Count | As % |
|---|---|---|
| Members of the public | 2 | 67% |
| Scientists | 1 | 33% |
Mendeley readers
The data shown below were compiled from readership statistics for 235 Mendeley readers of this research output. Click here to see the associated Mendeley record.
Geographical breakdown
| Country | Count | As % |
|---|---|---|
| Netherlands | 3 | 1% |
| Spain | 2 | <1% |
| Germany | 1 | <1% |
| Bangladesh | 1 | <1% |
| Argentina | 1 | <1% |
| United States | 1 | <1% |
| Unknown | 226 | 96% |
Demographic breakdown
| Readers by professional status | Count | As % |
|---|---|---|
| Student > Master | 31 | 13% |
| Student > Ph. D. Student | 29 | 12% |
| Professor > Associate Professor | 22 | 9% |
| Researcher | 21 | 9% |
| Other | 17 | 7% |
| Other | 75 | 32% |
| Unknown | 40 | 17% |
| Readers by discipline | Count | As % |
|---|---|---|
| Medicine and Dentistry | 100 | 43% |
| Social Sciences | 37 | 16% |
| Psychology | 13 | 6% |
| Nursing and Health Professions | 12 | 5% |
| Business, Management and Accounting | 7 | 3% |
| Other | 23 | 10% |
| Unknown | 43 | 18% |
Attention Score in Context
This research output has an Altmetric Attention Score of 3. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 15 September 2023.
All research outputs
#14,003,371
of 24,447,003 outputs
Outputs from Advances in Health Sciences Education
#511
of 919 outputs
Outputs of similar age
#92,583
of 167,906 outputs
Outputs of similar age from Advances in Health Sciences Education
#6
of 12 outputs
Altmetric has tracked 24,447,003 research outputs across all sources so far. This one is in the 41st percentile – i.e., 41% of other outputs scored the same or lower than it.
So far Altmetric has tracked 919 research outputs from this source. They typically receive a little more attention than average, with a mean Attention Score of 5.7. This one is in the 40th percentile – i.e., 40% of its peers scored the same or lower than it.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 167,906 tracked outputs that were published within six weeks on either side of this one in any source. This one is in the 44th percentile – i.e., 44% of its contemporaries scored the same or lower than it.
We're also able to compare this research output to 12 others from the same source and published within six weeks on either side of this one. This one has gotten more attention than average, scoring higher than 50% of its contemporaries.