This is the third experiment in a 1-year AHRC Digital Transformations project run jointly between Birmingham Conservatoire’s Integra Lab and Glasgow School of Art’s Digital Design Studios. The project explores new ways of positioning sound in space using touch-free interaction.

In this experiment we use the Microsoft Kinect as a motion capture device, enabling sounds can be dragged from a palette into a virtual 3D environment and moved around using direct physical manipulation.

Additionally, the volume of a selected sound can be controlled by drawing on a virtual slider with the left-hand index finger. Spatial proximity to existing sounds is indicated by a visual glow.

This initial experiment was developed in Unity 3D using the 3Dception plugin by TwoBigEars for binaural positioning.

Further details about the project can be found here.