Advantages of melodic over rhythmic movement sonification in bimanual motor skill learning

J. F. Dyer, P. Stapleton, M. W. M. Rodger

An important question for skill acquisition is whether and how augmented feedback can be designed to improve the learning of complex skills. Auditory information triggered by learners' actions, movement sonification, can enhance learning of a complex bimanual coordination skill, specifically polyrhythmic bimanual shape tracing. However, it is not clear whether the coordination of polyrhythmic sequenced movements is enhanced by auditory-specified timing information alone or whether more complex sound mappings, such as melodic sonification, are necessary. Furthermore, while short-term retention of bimanual coordination performance has been shown with movement sonification training, longer term retention has yet to be demonstrated. In the present experiment, participants learned to trace a diamond shape with one hand while simultaneously tracing a triangle with the other to produce a sequenced 4:3 polyrhythmic timing pattern. Two groups of participants received real-time auditory feedback during training: melodic sonification (individual movements triggered a separate note of a melody) and rhythmic sonification (each movement triggered a percussive sound), while a third control group received no augmented feedback. Task acquisition and performance in immediate retention were superior in the melodic sonification group as compared to the rhythmic sonification and control group. In a 24-h retention phase, a decline in performance in the melodic sonification group was reversed by brief playback of the target pattern melody. These results show that melodic sonification of movement can provide advantages over augmented feedback which only provides timing information by better structuring the sequencing of timed actions, and also allow recovery of complex target patterns of movement after training. These findings have important implications for understanding the role of augmented perceptual information in skill learning, as well as its application to real-world training or rehabilitation scenarios.