PageFocus: Using paradata to detect and prevent cheating on online achievement tests

Birk Diedenhofen, Jochen Musch

Cheating threatens the validity of unproctored online achievement tests. To address this problem, we developed PageFocus, a JavaScript that detects when participants abandon test pages by switching to another window or browser tab. In a first study, we aimed at testing whether PageFocus could detect and prevent cheating. We asked 115 lab and 186 online participants to complete a knowledge test comprising items that were difficult to answer but easy to look up on the Internet. Half of the participants were invited to look up the solutions, which significantly increased their test scores. The PageFocus script detected test takers who abandoned the test page with very high sensitivity and specificity, and successfully reduced cheating by generating a popup message that asked participants not to cheat. In a second study, 510 online participants completed a knowledge test comprising items that could easily be looked up and a reasoning task involving matrices that were impossible to look up. In a first group, a performance-related monetary reward was promised to the top scorers; in a second group, participants took part in a lottery that provided performance-unrelated rewards; and in a third group, no incentive was offered. PageFocus revealed that participants cheated more when performance-related incentives were offered. As expected, however, this effect was limited to items that could easily be looked up. We recommend that PageFocus be routinely employed to detect and prevent cheating on online achievement tests.