Anticipatory eye fixations reveal tool knowledge for tool interaction

Anna Belardinelli, Marissa Barabas, Marc Himmelbach, Martin V. Butz

Action-oriented eye-tracking studies have shown that eye fixations reveal much about current behavioral intentions. The eyes typically fixate those positions of a tool or an object where the fingers will be placed next, or those positions in a scene, where obstacles need to be avoided to successfully reach or transport a tool or object. Here, we asked to what extent eye fixations can also reveal active cognitive inference processes, which are expected to integrate bottom-up visual information with internal knowledge for planning suitable object interactions task-dependently. In accordance to the available literature, we expected that task-relevant knowledge will include sensorimotor, semantic, and mechanical aspects. To investigate if and in which way this internal knowledge influences eye fixation behavior while planning an object interaction, we presented pictures of familiar and unfamiliar tools and instructed participants to either pantomime 'lifting' or 'using' the respective tool. When confronted with unfamiliar tools, participants fixated the tool's effector part closer and longer in comparison with familiar tools. This difference was particularly prominent during 'using' trials when compared with 'lifting' trials. We suggest that this difference indicates that the brain actively extracts mechanical information about the unknown tool in order to infer its appropriate usage. Moreover, the successive fixations over a trial indicate that a dynamic, task-oriented, active cognitive process unfolds, which integrates available tool knowledge with visually gathered information to plan and determine the currently intended tool interaction.