| Title |
Assessing and overcoming participant dishonesty in online data collection
|
|---|---|
| Published in |
Behavior Research Methods, December 2017
|
| DOI | 10.3758/s13428-017-0984-5 |
| Pubmed ID | |
| Authors |
Chris Hydock |
| Abstract |
Crowdsourcing services, such as MTurk, have opened a large pool of participants to researchers. Unfortunately, it can be difficult to confidently acquire a sample that matches a given demographic, psychographic, or behavioral dimension. This problem exists because little information is known about individual participants and because some participants are motivated to misrepresent their identity with the goal of financial reward. Despite the fact that online workers do not typically display a greater than average level of dishonesty, when researchers overtly request that only a certain population take part in an online study, a nontrivial portion misrepresent their identity. In this study, a proposed system is tested that researchers can use to quickly, fairly, and easily screen participants on any dimension. In contrast to an overt request, the reported system results in significantly fewer (near zero) instances of participant misrepresentation. Tests for misrepresentations were conducted by using a large database of past participant records (~45,000 unique workers). This research presents and tests an important tool for the increasingly prevalent practice of online data collection. |
X Demographics
The data shown below were collected from the profiles of 3 X users who shared this research output. Click here to find out more about how the information was compiled.
Geographical breakdown
| Country | Count | As % |
|---|---|---|
| United States | 1 | 33% |
| Unknown | 2 | 67% |
Demographic breakdown
| Type | Count | As % |
|---|---|---|
| Members of the public | 2 | 67% |
| Science communicators (journalists, bloggers, editors) | 1 | 33% |
Mendeley readers
The data shown below were compiled from readership statistics for 67 Mendeley readers of this research output. Click here to see the associated Mendeley record.
Geographical breakdown
| Country | Count | As % |
|---|---|---|
| Unknown | 67 | 100% |
Demographic breakdown
| Readers by professional status | Count | As % |
|---|---|---|
| Student > Ph. D. Student | 14 | 21% |
| Researcher | 8 | 12% |
| Student > Bachelor | 7 | 10% |
| Professor | 3 | 4% |
| Lecturer | 3 | 4% |
| Other | 10 | 15% |
| Unknown | 22 | 33% |
| Readers by discipline | Count | As % |
|---|---|---|
| Psychology | 15 | 22% |
| Business, Management and Accounting | 5 | 7% |
| Medicine and Dentistry | 3 | 4% |
| Social Sciences | 3 | 4% |
| Sports and Recreations | 2 | 3% |
| Other | 10 | 15% |
| Unknown | 29 | 43% |
Attention Score in Context
This research output has an Altmetric Attention Score of 2. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 19 October 2022.
All research outputs
#15,926,695
of 25,653,515 outputs
Outputs from Behavior Research Methods
#1,424
of 2,564 outputs
Outputs of similar age
#247,915
of 449,698 outputs
Outputs of similar age from Behavior Research Methods
#17
of 31 outputs
Altmetric has tracked 25,653,515 research outputs across all sources so far. This one is in the 36th percentile – i.e., 36% of other outputs scored the same or lower than it.
So far Altmetric has tracked 2,564 research outputs from this source. They typically receive more attention than average, with a mean Attention Score of 8.0. This one is in the 42nd percentile – i.e., 42% of its peers scored the same or lower than it.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 449,698 tracked outputs that were published within six weeks on either side of this one in any source. This one is in the 43rd percentile – i.e., 43% of its contemporaries scored the same or lower than it.
We're also able to compare this research output to 31 others from the same source and published within six weeks on either side of this one. This one is in the 45th percentile – i.e., 45% of its contemporaries scored the same or lower than it.