While four racing drones were darting above the audience at speeds of up to 140 km/h, an LED wall displayed a mix of live shots taken by the drones and a 3-D model of the future stainless steel mill. Highly focused drone pilots steered their custom-made unmanned aerial vehicles right above the heads of the assembled dignitaries as a virtual mockup of the new plant emerged on screen.
The spectacular performance at voestalpine’s groundbreaking ceremony in early April 2018 was produced and staged by Ars Electronica Solutions in collaboration with the drone crew of MOHIO Gaming. In this interview, project director Chris Bruckmayr and Andreas Neubauer, founder and CEO of MOHIO GmbH Sports, tell us more about their mixed reality drone race.
Ars Electronica Solutions recently staged a mixed reality drone race at the voestalpine stainless steel mill being built in Kapfenberg. How did this come about?
Chris Bruckmayr: It’s a long story! At the 2016 Ars Electronica Festival, we produced a drone performance in the Gleishalle [PostCity’s railroad loading dock] at which we quickly realized that more dramaturgy would make for a more entertaining show. Back then, the concept was still in its infancy—we worked with projections and also supported the Big Concert Night with drones. Ever since, we’ve been brainstorming how to expand on this. When voestalpine approached us because they wanted something interesting for the groundbreaking ceremony at their new stainless steel mill in Kapfenberg, we recognized our big opportunity—the construction site is a superb setting for a performance by racing drones! Then we came up with the idea of using racing drones to delineate the dimensions of the future mill and simultaneously depict the digitization & automation processes that will be going on inside it. With racing drones, we can fly at high speed and also have the possibility of displaying first-person views from the cockpit’s perspective. These elements were dovetailed with a three-dimensional model of the future plant, which was provided by voestalpine. So, during this virtual fly-through, the audience saw a blend of 3-D data and drone footage mixed live on a jumbo LED wall. Thus, spectators got the feeling of actually gliding through a facility that’s only in the process of being built. But at the same time, there’s also this racing experience!
Credit: Ars Electronica Solutions
What makes such a performance more exciting than a conventional drone race?
Andreas Neubauer: Following up on our joint production in the Gleishalle at the 2016 Ars Electronica Festival, we thought it would be really interesting to stage a high-speed drone racing show with virtual reality elements mixed in. Plus, I have to mention that we get a lot of requests for show races, and we had already gotten started putting them on, but we quickly realized that just a racing show is totally boring for the audience. The drones aren’t very big and they’re hard to see when the race is staged in broad daylight. The combinations with 3-D animation or interspersing footage shot from the drones are also really interesting for us as pilots because the demands are different than in a racing situation. The point of a race is to fly around the course fastest and win, whereas a show race is more a matter of coordination. You have to fly fast, as well as coordinate your departure and arrival with the video. The challenges are more fun, and this provides a considerably more entertaining experience for the spectators.
Credit: Ars Electronica Solutions
In a race, each pilot wears a headset and takes up position on the edge of the racecourse…
Andreas Neubauer: They use so-called goggles [data glasses]. A camera in the drone sends a video signal via a transmitter to a receiver mounted on the goggles, which enables the pilot to see the image from the camera right before his/her eyes.
Chris Bruckmayr: That’s why it’s called first-person view—you see what the drone sees, and you feel as if you’re right in the cockpit.
Andreas Neubauer: The fascinating thing about this is the trick it plays on the brain. You’re seated and immobile, but in your head you think you’re inside the drone and actually flying around.
Chris Bruckmayr: The speed is very high, up to 140 km/h. Screening these images in jumbo-format for the audience and then mixing virtual reality and mixed reality elements into them works really well and looks terrific!
Credit: Ars Electronica Solutions
What are the regulatory restrictions that have to be observed in staging such a show?
Andreas Neubauer: We work within a framework that doesn’t entail violating laws. We don’t fly very high—only up to five meters above ground. There are also rules for video signal strength, which we comply with. But keep in mind that there’s a big difference between our drones and the conventional unmanned aerial vehicles available in stores. We custom-built all of our racing drones ourselves, so they’re not suited to hovering around and shooting videos.
Give us a more detailed account of a show race like the one you produced for the voestalpine groundbreaking ceremony?
Chris Bruckmayr: We pre-produced a video showing the drone flight in first-person view, and the flight is timed so that it’s precisely coordinated with the virtual fly-through of the 3-D model. We achieved this with a simple form of tracking. In the performance, a formation of four drones flew around the construction site, and the actual footage showing the first-person view of the lead drone was mixed with the pre-produced video. This imagery produced by the racing pilots and the fly-through of the virtual steel mill thus constantly provided the spectators with a sort of futuristic mélange that really made for an exciting experience. For one thing, these images are viewed on a huge LED wall, and the drones racing through the future work are truly impressive; at the same time, you can watch the real-life drones racing by in formation.
We want to develop this, since there are many more variants to interweaving racing drones and media technology. For instance, we’re giving thought to an enhanced, upgraded tracking system that still retains simplicity so that we can register the positions of several drones and introduce lighting effects, projections and other media techniques. The drone itself is becoming a media tool.
Credit: Ars Electronica Solutions
What sort of technical effort goes into a performance like this?
Chris Bruckmayr: As I mentioned—first, we have to pre-produce the video. Maybe we’ll eventually get to the point at which we don’t have to pre-produce the video; instead, we’ll use the live footage from the flying drones and simply insert it in real time into the 3-D model. Be that as it may, it’s still necessary to pre-produce the timed video. And as far as the drone technology is concerned, the principal source of problems is the radio communication system. The setting of this performance was a construction site, so, at our rehearsal, there were still a lot of bulldozers moving around, which meant all sorts of disturbances that can slightly influence signal transmission.
Andreas Neubauer: But you have to say that all the problems that arose could be solved relatively quickly. Everybody knew exactly what they had to do. The level of professionalism was very high.
Chris Bruckmayr: The situation was demanding because the setting was a construction site where work was going on during the rehearsal! The only break they took was during the actual groundbreaking ceremony; otherwise, the air was always full of smoke and dust. But fortunately there was a downpour the night before the groundbreaking, which considerably improved the situation on the day of the event [LAUGHS]. In going about stuff like this, you simply wing it with prototype ideas, and the problems don’t crop up until you’re on site. But they were relatively minor.
Andreas Neubauer: But that also somehow makes it even more fun, and makes the achievement more satisfying! When everything finally comes off as planned, you think: Well Done Indeed!
Chris Bruckmayr has been director of products at Ars Electronica Solutions since 2018; 2013-17, business manager and creative producer on the Ars Electronica SPAXELS® crew; sound artist who, under the name raum.null, produces dark techno for the vinyl-only label Belgrade dubs / Belgrade; 2015-17, performances at the Ars Electronica Festival.
Andreas Neubauer has been CEO and proprietor of MOHIO GmbH since 2009; from 2016, FAI delegate, technical expert & judge, and AEROCLUB Bundesfachreferent Drone Racing F3U & FPV; from 2017, chairman of FPV Racing Club OÖ.
To learn more about Ars Electronica, follow us on Facebook, Twitter, Instagram et al., subscribe to our newsletter, and check us out online at https://ars.electronica.art/news/en/.