Abstract:
Composers working with sounds and moving images are immediately confronted with a paradox. On one hand, audiovisual materials appear to offer the possibility of complementing one another – of forming a highly effective means of communicating artistic ideas – and on the other, they appear to carry the risk of detracting from one another – of deforming the musical language that he or she has worked so hard to create. Durk Talsma and Max Mathews succinctly state the opposing principles. ‘Many behavioural studies have provided evidence for the hypothesis that integrating visual and auditory stimuli serves the purpose of enhancing perceptual clarity y These results suggest that communication between the visual and auditory brain areas is a highly effective and relatively automatic process’ (Talsma, Doty and Waldorff 2007: 679). ‘I personally find most combined music-video art problematic. It seems to me that the sound and images often compete for my attention y If I pay attention to what I am seeing, I often miss what I am hearing, and if I try to concentrate on the music, the images can often be an irritating distractiony’ (Mathews 2007: 94). This article seeks to transcend this paradox through the identification of audiovisual materials that function in different ways. Examples of creative work are offered to illustrate more general points of ‘language’, a model for classifying media pairs is put forward, and practical methods of audiovisual composition are proposed. The narrow findings of the study offer a vocabulary for discussing the functionality of audiovisual materials, detailed methods of media pairing, and techniques of parametric alignment, while the wider findings extend to associated domains such as live electronic music, and hyper-instrument design.
