Perceiving space through sound: Mapping human movements into MIDI

Gestural expressiveness plays a fundamental role in the interaction with people, environments, animals, things, and so on. Thus, several emerging application domains would exploit the interpretation of movements to support their critical designing processes. To this end, new forms to express the people's perceptions could help their interpretation, like in the case of music. This paper presents a novel algorithm for mapping human movements into MIDI music. The algorithm has been implemented in a system that integrates a module for real-time tracking of movements through a sample-based synthesizer using different types of filters to modulate frequencies. The system has been evaluated through a user study, in which several users have participated to a room experience, yielding significant results about their perceptions with respect to the environment they were immersed.