From Las Vegas to Umea, Sweden to Dubai, the Ars Electronica Futurelab’s Spaxels have flown some pretty exotic air space and fascinated thousands of people with their spectacular light shows. Recently, Intel™ Corporation commissioned the Futurelab crew to undertake a quantum technical leap that ultimately landed in the Guinness Book of World Records. For the first time ever, 100 synchronized drones were airborne simultaneously. Since that feat in November 2015, the DRONE 100 show has been performed only once—in Sydney, Australia. But that’s going to change very soon. On September 10, 2016, 100 drones will ascend over the Danube in Linz and provide the lead-in to this year’s Visualisierte Klangwolke, “River of Knowledge.”
To get a behind-the-scenes look, we went to the Ars Electronica Futurelab for a chat with Spaxels crew members Chris Bruckmayr and Andreas Jalsovec. They clued us in on all the work that putting 100 quadcopters aloft actually entails.
First of all, what actually are Spaxels?
Chris Bruckmayr: Simply put, they’re quadcopters equipped with LED modules that can be programmed—for example, they can be synchronized to music, and you can regulate how fast the lights blink. In more abstract terms, Spaxels—short for space pixels—are points of light that can be freely positioned in space. In this case, they’re quadcopters but, of course, you could use any flying objects you choose.
Andreas Jalsovec: The idea that led to the Spaxels was actually to conjure up an enchanting display in the night sky, one that’s as large and seamless as possible. The concept behind it is that of a video display that fills an entire space, and pixels can be positioned anywhere on it and turned on and off. What we ultimately foresee is a lot more than 100 pixels, so that we can form all sorts of three-dimensional shapes and symbols, almost like a hologram.
Credit: Vanessa Graf
But what’s so special about the Spaxels in comparison to other drones?
Chris Bruckmayr: Individual drones are already being employed to do all sorts of jobs, in agriculture, for instance, or logistics. What we do is to assemble a sort of swarm at a single location. We have software that can simultaneously navigate many drones. It is not possible to just set up 100 pilots who then go about steering 100 quadcopters via remote control. The software we’ve developed is capable of coordinating the flights of 100 drones via a central processor. Ideally, they behave almost like a flock of birds, jointly maneuvering and thereby forming three-dimensional objects that are constantly changing. But without the lights, they wouldn’t be Spaxels, After all, this is actually a light show, and the original approach was a spinoff of media art.
Andreas Jalsovec: But keep in mind that, technically speaking, the idea of 100 pilots each with a remote control unit wouldn’t even work. In fact, this was the very first problem we had to solve. The ground control software that we’ve written is highly advanced at this point, very well-developed. The next aspect that was important to us was the question of how we could actually come up with a completely new display.
Chris Bruckmayr: Now, we’re capable of conjuring up a visual composition in the heavens with this drone swarm. One configuration can freely morph into another one; we can achieve fluid transitions that remain synchronized to the music. Needless to say, there are limitations as to how the maneuvers actually have to be done and how to avoid collisions, but, in principle, we compose a sort of light symphony in the evening sky.
Andreas Jalsovec demonstrates how the LED module works. Credit: Vanessa Graf
So, actually, it only takes one person to supervise the software, and not 100 pilots for 100 drones. Who steers the Spaxels during an appearance?
Andreas Jalsovec: At appearances, we have four pilots, whereby only one of them is supervising all the drones on a screen. The other three pilots each have a third of the swarm on screen, since that’s approximately the maximum number that you can maintain an overview of and really concentrate on. The pilots’ job is actually to conduct the individual tests prior to the show. Each quadcopter gets a check-up and is put through its paces aloft, and with 100 drones, that takes quite a while, which is why we have four pilots. During a show, each one concentrates on a small part of the swarm, checks to see that all parameters are A-OK, and intervenes if there’s an emergency.
Chris Bruckmayr: In going about this, the airborne drones aren’t tracked visually; this is performed via software—how’s the radio communication, how many satellites see the drones, how’s the battery performing?
Credit: Vanessa Graf
So you’re saying that the flight paths aren’t adjusted during a performance.
Chris Bruckmayr: The flight path is predetermined. Nevertheless, if, for example, the software reports that a collision is imminent, the pilots intervene. We’re able to withdraw drones from the swarm and even force them to land. The fact that we have this capability is also important for the government agencies responsible for public safety.
Andreas Jalsovec: If a drone actually does deviate from its flight path—for instance, because it’s buffeted by a gust of wind or the GPS signal is poor—then, of course, we see that in the software. If one drone comes too close to another one, we receive a collision warning and we’re able to respond. Often this means assessing the situation and performing an evaluation with the ground control software—where’s the quadcopter currently and where should it be? And then, of course, you can do a visual check and then decide whether to step in or not.
What creative, technical and administrative challenges are associated with a show like this?
Chris Bruckmayr: The biggest hurdles are the regulations imposed by the air traffic safety authorities. This swarm control system we’ve developed has gotten us into an area that’s uncharted territory for these agencies. So, severe constraints are always imposed on us—how large our security radius has to be; that we’re never permitted to overfly people; and that certain areas are simply no-fly zones if, for example, they’re too heavily built-up or densely populated. In Linz, there are restrictions whereby we’re not allowed to fly at night, and thus no later than dusk. The reason is that after dark, you lose visual contact, and that’s a violation of Austria’s Drone Law. So, this first step—getting permission to fly at all—is the most complex part. Only then can the design process begin.
Andreas Jalsovec: First of all, the security radius is marked off and the position of the audience is stipulated. After that, we have to lay out the airfield—how do we position the drones on the ground, and what do we do to prevent collisions? When they’re airborne, they have to be at least six meters apart, and that minimum distance is even slightly greater when they’re stacked vertically in the air. Aloft, this spacing must be maintained. But on the ground, we can’t keep them six meters apart; instead, we bunch them closer together and then take off sequentially in several waves.
Andreas Jalsovec at work. Credit: Vanessa Graf
What are the difficulties you have to overcome in the design process?
Andreas Jalsovec: In the early days of the Spaxels, our design process was relatively straightforward; we simply took three-dimensional objects and animated them. This was still feasible with 50 drones, though this too entailed considerable effort—we had to prevent collisions, maintain speed, stay oriented on the physical characteristics of the flight model. How long does a drone need to get from Point A to Point B? How quickly can one configuration morph into the next one in the routine? Meanwhile, you have to keep in mind how long the battery’s charge lasts and how much time is required from takeoff to landing. These are all aspects that flow into the show design. With 50 drones, that could be done manually, but as the number of drones increased, the design process got more complicated. In the meantime, we’ve developed a new approach that combines several different animation methodologies. On one hand, the drones are treated like particles; on the other hand, it works just like the animation of characters or physical shapes. As soon as the animation is finished, it’s exported to our ground control software, which is a module we developed in-house to simulate test flights. The simulated drones behave just like real ones, which means that they don’t fly any faster and a certain degree of inertia and imprecision is built in. Throughout the design process, a simulation is always run in conjunction with the audio track or the musical piece that’s to be played so that the LED animation is coordinated with it.
Chris Bruckmayr: Once the simulation is successful to a fairly high degree, it’s converted into GPS data. Thus, to what precise point in space does the respective drone have to fly in order to depict the shape that I’ve animated on the software?
The drones are already flying over the Danube in Linz – at least on screen. Credit: Vanessa Graf
And this last step is always customized to the specific location at which the drones are flying?
Andreas Jalsovec: Precisely. It makes a difference if we’re flying here in Linz above the trees along the Danube and, thus, whether the drones are actually far enough away from the audience, or, as was the case in Sydney, whether we can ascend directly from the airfield, and all we have to do is make sure that we maintain a safe distance from our own pilots.
Chris Bruckmayr: Actually, our most impressive achievement is that we’re able to coordinate the movements of 100 drones in an animated program and to communicate the information to them in the form of radio commands. This is the interface that we have totally mastered. And that’s why we’re now evidently the only ones capable of flying 100 drones in synchronized animation.
In the next installment in our Spaxels interview series, Andreas Jalsovec and Chris Bruckmayr reminisce about the early days of the drone shows and talk about the many spectacles staged at venues worldwide. That interview will be appearing shortly here on the Ars Electronica Blog.
If you’d like to experience—right up close—what it looks like to synchronize the animation of 100 drones, then stop by Donaupark in Linz on September 10, 2016 by 7:45 PM. And, following this spectacular European premiere, anyone who’d like to learn more about unmanned aerial vehicles is invited to the Drone Lab at the 2016 Ars Electronica Festival September 8-12 in POSTCITY, where exhibits, presentations and demonstrations will show what this technology is capable of.