Is the cognitive reflection test a measure of both reflection and intuition?

Gordon Pennycook, James Allan Cheyne, Derek J. Koehler, Jonathan A. Fugelsang

The Cognitive Reflection Test (CRT) is one of the most widely used tools to assess individual differences in intuitive-analytic cognitive styles. The CRT is of broad interest because each of its items reliably cues a highly available and superficially appropriate but incorrect response, conventionally deemed the "intuitive" response. To do well on the CRT, participants must reflect on and question the intuitive responses. The CRT score typically employed is the sum of correct responses, assumed to indicate greater "reflectiveness" (i.e., CRT-Reflective scoring). Some recent researchers have, however, inverted the rationale of the CRT by summing the number of intuitive incorrect responses, creating a putative measure of intuitiveness (i.e., CRT-Intuitive). We address the feasibility and validity of this strategy by considering the problem of the structural dependency of these measures derived from the CRT and by assessing their respective associations with self-report measures of intuitive-analytic cognitive styles: the Faith in Intuition and Need for Cognition scales. Our results indicated that, to the extent that the dependency problem can be addressed, the CRT-Reflective but not the CRT-Intuitive measure predicts intuitive-analytic cognitive styles. These results provide evidence that the CRT is a valid measure of reflective but not of intuitive thinking.