| Title |
Collecting response times using Amazon Mechanical Turk and Adobe Flash
|
|---|---|
| Published in |
Behavior Research Methods, May 2013
|
| DOI | 10.3758/s13428-013-0345-y |
| Pubmed ID | |
| Authors |
Travis Simcox, Julie A. Fiez |
| Abstract |
Crowdsourcing systems like Amazon's Mechanical Turk (AMT) allow data to be collected from a large sample of people in a short amount of time. This use has garnered considerable interest from behavioral scientists. So far, most experiments conducted on AMT have focused on survey-type instruments because of difficulties inherent in running many experimental paradigms over the Internet. This study investigated the viability of presenting stimuli and collecting response times using Adobe Flash to run ActionScript 3 code in conjunction with AMT. First, the timing properties of Adobe Flash were investigated using a phototransistor and two desktop computers running under several conditions mimicking those that may be present in research using AMT. This experiment revealed some strengths and weaknesses of the timing capabilities of this method. Next, a flanker task and a lexical decision task implemented in Adobe Flash were administered to participants recruited with AMT. The expected effects in these tasks were replicated. Power analyses were conducted to describe the number of participants needed to replicate these effects. A questionnaire was used to investigate previously undescribed computer use habits of 100 participants on AMT. We conclude that a Flash program in conjunction with AMT can be successfully used for running many experimental paradigms that rely on response times, although experimenters must understand the limitations of the method. |
X Demographics
The data shown below were collected from the profile of 1 X user who shared this research output. Click here to find out more about how the information was compiled.
Geographical breakdown
| Country | Count | As % |
|---|---|---|
| United Kingdom | 1 | 100% |
Demographic breakdown
| Type | Count | As % |
|---|---|---|
| Scientists | 1 | 100% |
Mendeley readers
The data shown below were compiled from readership statistics for 138 Mendeley readers of this research output. Click here to see the associated Mendeley record.
Geographical breakdown
| Country | Count | As % |
|---|---|---|
| United States | 11 | 8% |
| Germany | 2 | 1% |
| United Kingdom | 2 | 1% |
| France | 1 | <1% |
| Brazil | 1 | <1% |
| Argentina | 1 | <1% |
| Iceland | 1 | <1% |
| Unknown | 119 | 86% |
Demographic breakdown
| Readers by professional status | Count | As % |
|---|---|---|
| Student > Ph. D. Student | 44 | 32% |
| Researcher | 25 | 18% |
| Student > Master | 14 | 10% |
| Student > Doctoral Student | 10 | 7% |
| Student > Bachelor | 10 | 7% |
| Other | 21 | 15% |
| Unknown | 14 | 10% |
| Readers by discipline | Count | As % |
|---|---|---|
| Psychology | 62 | 45% |
| Social Sciences | 9 | 7% |
| Business, Management and Accounting | 7 | 5% |
| Computer Science | 6 | 4% |
| Medicine and Dentistry | 6 | 4% |
| Other | 22 | 16% |
| Unknown | 26 | 19% |
Attention Score in Context
This research output has an Altmetric Attention Score of 2. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 26 September 2016.
All research outputs
#16,721,208
of 25,373,627 outputs
Outputs from Behavior Research Methods
#1,539
of 2,525 outputs
Outputs of similar age
#125,854
of 205,982 outputs
Outputs of similar age from Behavior Research Methods
#10
of 18 outputs
Altmetric has tracked 25,373,627 research outputs across all sources so far. This one is in the 32nd percentile – i.e., 32% of other outputs scored the same or lower than it.
So far Altmetric has tracked 2,525 research outputs from this source. They typically receive more attention than average, with a mean Attention Score of 8.1. This one is in the 35th percentile – i.e., 35% of its peers scored the same or lower than it.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 205,982 tracked outputs that were published within six weeks on either side of this one in any source. This one is in the 36th percentile – i.e., 36% of its contemporaries scored the same or lower than it.
We're also able to compare this research output to 18 others from the same source and published within six weeks on either side of this one. This one is in the 44th percentile – i.e., 44% of its contemporaries scored the same or lower than it.