Klangkörper - Compose sounds with your body
Klangkörper is an interactive installation implemented in Touchdesigner and designed to experience music in a new, inclusive way. The project invites participants to become part of a digital choir, represented as avatars on a projection screen. Through gestures and facial expressions, users can generate and modify sounds, which then harmonize with the tones of other participants, creating a shared musical experience.
Klangkörper was developed in 2024/2025 as part of my joint master project and thesis with Julia Herzog at the Technical University of Augsburg. While the concept of the project was developed together, Julia focused on the full design process and film documentation, and I was responsible for the technical realization.
Making off
From movement to music - how you control the sound
As soon as users step on one of the stages, they get recognized by the system and create a unique avatar that mirrors the user simultaneously. As they open their mouth, their avatar open it’s mouth too and starts to sing like a choir singer. Users can shape the sound with simple movements. Raising or lowering their hands changes the pitch, while moving them closer or apart controls the volume. By switching microphones, they can choose between a bass, tenor, and soprano voice. Mouth shapes like A, E, or O adjust the vowel sound. With more participants, a digital choir forms, inspiring interaction and shared harmonies.
To help users understand how to interact with the installation, an information video next to the installation explains the different interaction possibilities in detail.
Implementation
After the initial concept, our focus shifted to defining the interaction parameters. We collected a wide range of movements and refined them to feel natural in combination, easy to perform, and meaningfully connected to the resulting sound changes.
After each milestone, we conducted user tests to gather outside feedback and continuously improve the experience.
The technical implementation was built in TouchDesigner. A Kinect camera captures users’ full-body movements, which are processed directly in TouchDesigner. In addition, a webcam in front of each of the three stages tracks facial expressions using MediaPipe. Both body and face data are filtered according to our needs and mapped to the visual avatars as well as to the corresponding audio output.
Set Up
klangkörper goes Modular
As part of the university-wide call “Pitch Dich auf’s Modular” in March 2025, the Technical University of Augsburg invited submissions for innovative and participatory projects. Our master’s project klangkörper was selected to take part in the Modular Festival 2025 and became part of the official program.
This gave us the opportunity to present the installation on a large scale and let several thousand people experience and test it. We are very thankful for this adventure and exerience. The following video shows how it all unfolded.
Thank you!
Klangkörper was a true passion project and showed me how powerful and inspiring interdisciplinary collaboration can be. My project partner Julia Herzog will always hold a special place for me. In the project and beyond, we always have each other’s backs.
Special thanks to Clara Eckhardt, who was part of the team during the first four months, as well to Francisco García Días and Mathias Hradecsni for technical support, Olivia Nigl and Julia Hess for support in film and photography, and our supervisors Michael Kipp and Andreas Muxel from the Technical University of Augsburg for conceptual guidance and all-around support.
