Body Movement Sonification using the Web Audio API

Christian Baumann, Jan-Torsten Milde and Johanna Friederike Baarlink

In this paper we describe the ongoing research on the development of a body movement sonification system. High precision, high resolution wireless sensors are used to track the body movement and record muscle excitation. We are currently using 6 sensors. In the final version of the system full body tracking can be achieved. The recording system provides a web server including a simple REST API, which streams the recorded data in JSON format. An intermediate proxy server pre-processes the data and transmits it to the final sonification system. The sonification system is implemented using the web audio api. We are experimenting with a set of different sonification strategies and algorithms. Currently we are testing the system as part of an interactive, guided therapy, establishing additional acoustic feedback channels for the patient. In a second stage of the research we are going to use the system in a more musical and artistic way. More specifically we plan to use the system in cooperation with a violist, where the acoustic feedback channel will be integrated into the performance.