Combining Web Audio Streaming, Motion Capture, and Binaural Audio in a Telepresence System

Lawrence Fyfe, Olivier Gladin, Cédric Fleury and Michel Beaudouin-Lafon

This paper describes the use of web audio streaming, motion capture cameras, and binaural audio in a telepresence system. The telepresence system is part of a larger project for facilitating research collaboration. The goal of the project is to create hardware and software infrastructure that enables researchers to collaborate on visualizing large data sets or other complex collaboration tasks. Telepresence is a significant part of that infrastructure, including both audio and video streaming. Binaural audio, with the ability to spatialize sound sources for listeners, shows promise for use in telepresence applications. When combined with a motion capture system, the location of collaborators in physical spaces can be sent to remote locations, putting the collaborator into remote audio spaces. The audio spaces can be configured in a variety of ways from single shared audio spaces to adjacent, contained, or split audio spaces, facilitating explorations in the possibilities of binaural telepresence.