*falcon heavy is a multi-user experience and stereoscopic audiovisual sculpture for Deep Space 8K, generated in real-time. An ever-shifting entity rules the ritualistic semi-virtual arena; populated by people participating together to find synchronous signals in all the noise.
*falcon heavy is based on SHARESPACE, a large European R&D project on using avatars in social situations. The goal of *falcon heavy is to bind the entity represented as spatial audio-visual chaos into concrete forms in a group effort. To achieve this, up to four participants work together in Deep Space 8K of the Ars Electronica Center in Linz. Led by pulsating rhythms, they need to reach certain spatial locations and achieve synchronicity in their hand movement patterns.
Presentation of *falcon heavy @ Futurelab Night; photo: Bettina Gangl
Presentation of *falcon heavy @ Deep Space 8K; photo: flap
Presentation of *falcon heavy @ Futurelab Night; photo: Bettina Gangl
Presentation of *falcon heavy; photo: flap
SHARESPACE is a European research and development consortium focused on the ethically sound future use of digital avatars for social interactions in mixed human and automated settings. The project with 14 partner institutions across eight countries is running from January 2023 to December 2025. The Futurelab is mainly responsible for developing art scenarios in Deep Space 8K of the Ars Electronica Center.
Sync up in all the noise
In *falcon heavy, the Futurelab lasertracking system pharus determines the player’s positions while the movement of their hands is monitored via inertial measurement unit (imu) sensors. Humans thus occupies physical space, facing an audiovisual entity in virtual space – an extended movement area with an audio-reactive particle system that is displayed on the wall projection in stereoscopic 3D.
The floor projection in physical space serves as a visual guiding system for the participants to find where they are supposed to move. Meanwhile, particle flow from their imu sensors to virtual space showcases the correct hand movements to synchronize with each other. The entity reacts visually to improved synchronicity and forms a gestalt as imu sensor tracking stays synchronous. After a brief moment, the entity sheds this layer and enters a new stage – the cycle repeats with new objectives.
Presentation of *falcon heavy @ Futurelab Night; photo: Bettina Gangl
Presentation of *falcon heavy; photo: Bettina Gangl
*falcon heavy presentation @ Deep Space 8K; photo: Bettina Gangl
This idea of “the other” within a dramatic structure is used throughout *falcon heavy for the overall shaping of experience as a sequential series of interactive events and sounds in a shared space. The animation and actions of the entity are controlled by a behavior tree. It can sense people via pharus tracking and decide if it just wants to look at them, move into their direction, or change its color.
An external application monitors the sensor data and calculates a quality of synchronicity value called qSyn. For each stage, qSyn is measured and accumulated within the sequence’s timeframe. Only when the threshold is overcome, the participants move onto the resolution – if not, the entity reverts back to the initiation phase of the stage. When the threshold is met, the sound design indicates progress with a positive crescendo motif. In this way, qSyn forms noise to structure.
Credits
Ars Electronica Futurelab: Johannes Pöll, Daniel Rammer, Raphael Schaumburg-Lippe, Arno Deutschbauer, Cyntha Wieringa
PARTNER: SHARESPACE Consortium
This work has been developed in the framework of the SHARESPACE project.
SHARESPACE has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 10192889.