In the first installment of this blog post, Chris Bruckmayr and Andreas Jalsovec told us what Spaxels actually are, and all the preparations that are necessary for them to fly—getting authorized to do so, and then bringing off the flight without a hitch. In Part 2, they reminisce a bit about the history of the Spaxels project and the interesting assignments that have taken them around the world.
Credit: Christopher Sonnleitner
When you came up with the idea of synchronized formation flight of so many drones, this project was said by some to be too ambitious …
Chris Bruckmayr: Yeah, a lot of people thought the telecommunications weren’t up to the task.
Andreas Jalsovec: And not only that. We didn’t even have any drones yet. And a lot more time would actually have been needed for the development. Our approach was an outgrowth of the experience we had gained in the area of virtual reality. Florian Berger [key researcher and artist at the Ars Electronica Futurelab] wrote the first version of the ground control software, which was nothing more than his own VR system. Due to work we had previously done in the Futurelab on 3-D processes, this was, as we saw it, the only way to go—to stick with the same approach and utilize it this time to animate real objects. And obviously this turned out to be the right way to go about it. After that, the only problem was to communicate with 50 drones simultaneously and to get them aloft. By now, a whole team has formed around the head of Spaxels development Martin Mörtz, who did not only just completely rewrite the ground control software, but is continuing to expand the program and add new functionalities for simulation and analysis of drone swarms.
Chris Bruckmayr: Of course, we had to make some changes to the drone software—after all, you can’t just buy a drone and expect it to fly in a swarm. And many drones don’t even permit modifications to their control program, to the so-called autopilot, which is actually what makes the drone fly. The model we ultimately decided on—from Ascending Technologies, a German firm—was a research drone, and the only one that allowed us to supplement its code.
What exactly had to be modified on the Spaxel drones for them to fly in a swarm?
Andreas Jalsovec: Not considering the fact that we wanted to communicate with the drones via our ground control software, we also had to think about the LED modules that we set up on the drones. The drones of Ascending Technologies have two processors, a high-level processor, which gives us control over the drone, and a low-level processor. This last one is responsible for the flight control. This processor was partially adapted to our specifications, but otherwise we didn’t make a lot of modifications to it. The second processor is responsible for controlling the lighting as well as for the safety features that we’ve additionally built in on the basis of the experience we subsequently gained. So, we programmed one processor, and the second one does the flight control.
Adapting the LED module was a major challenge for Michael Platz, who is responsible for the fact that you can see the Spaxels from afar and that the weight of our hummingbird is not heavier than the maximum of 200 grams. A commercially available drone comes with a couple of lights used for positioning, but what we needed was a really powerful LED module that’s visible from 500 meters away. At present, we’re using the third version of an LED module we developed in-house. We had a circuit board custom-made to our specifications, and mounted four extremely strong LEDs onto it. Plus, we built our own 3-D housing so it retains its shape and doesn’t violate the rules of aerodynamics.
Credit: Vanessa Graf
Where does the technical know-how come from that’s necessary for all the tinkering you do?
Andreas Jalsovec: Actually, none of us had the slightest idea. Nobody here had experience flying model airplanes or anything like that. First of all, we had to get a feel for the drones.
Chris Bruckmayr: I guess that was just the right approach because, as a result, we weren’t in awe of these drones. The point was simply to get them to do exactly what we wanted.
Andreas Jalsovec: Needless to say, we now know quite a bit about drones and, in retrospect, we’re kind of amazed how self-evidently the first show came off at all!
This first show with 50 drones was staged in conjunction with the 2012 Klangwolke in Linz.
Chris Bruckmayr: That was our very first world record, but due to cost considerations, we didn’t have it certified by Guinness.
But you put on quite a few shows until you finally went for the world record …
Chris Bruckmayr: Yeah, we did a lot of flying in the meantime. We were in London, where we formed the Star Trek logo. We opened the European Capital of Culture in Sweden, and then the Islamic Capital of Culture in Sharjah. We were in Dubai for the National Day celebration of the United Arab Emirates, and in Hannover for the Day of German Unity festivities.
Since the Klangwolke back then, the Spaxels have gotten very commercial. Ars Electronica began to get booked for these flights. The 100-drone flight for Intel™ in November 2015 was the highpoint of this development. At the time of their inquiry, we owned 70 drones; Intel wanted 110: 100 in the air; 10 in reserve. This was the biggest commercial commission that our drone division has ever received.
When you’re commissioned to put on a show, what are the subsequent steps? What has to happen for a show to take place?
Chris Bruckmayr: Notwithstanding the official permits you always have to obtain, the first question is always: Where precisely can the drones take off and land? How big is the flight area? How widely will the drone swarm be dispersed—over the Danube, for instance? What’s the minimum permissible security zone? Where will the audience be positioned, and how can we create figures they’ll be able to recognize? What follows is an elaborate logistical process, since, after all, we have to set up a complete airfield. We need containers in which we can store the drones; we have to set up antennas; then we have to mark off the area and secure it. And, of course, we have lay out the drones and test each one. The preparations usually start a couple of days before the actual show.
What are some of the causes of difficulties at the various locations?
Andreas Jalsovec: In Sydney, for example, we took off from two barges—simple floating platforms that aren’t self-propelled. We lined up the two biggest barges that were available at the time on the east coast of Australia. Now, before we start preparing for a show, there’s always a so-called recce, a sort of local reconnoitering tour during which we send up one or two drones for a test flight on site. A barge was brought in especially for the recce in Sydney, and set up exactly the way it would be at the performance—it was towed into position and anchored, so it would be easy for us to determine if a show would even be possible there. After the recce, we flew back to Linz with what we had recorded during the test flight and discussed how the show might work. Now, since we were working with 100 drones at this point, a second barge was necessary for the performance, but it couldn’t be delivered due to a storm on the east coast, so a replacement barge was provided. But this one was longer and narrower, and there were some obstacles on the deck. So, suddenly we were dealing with a totally different airfield than we had planned for. That storm cost us a day of rehearsal time, and our airfield had completely different dimensions. So we had to come to terms with this new barge, which meant that, instead of five rows of drones, there would be only three and they’d be much longer. This complicated the take-offs and landings. Plus, we had to protect the obstacles on deck from the drones and vice-versa. What we finally did was cover them with nets so a drone wouldn’t be totally demolished in case it veered off during a takeoff or landing and smashed into one of them.
Chris Bruckmayr: Sometimes it can happen that you arrive at a location—like in Dubai, where it was agreed that the main road next to the show venue was to be closed—and it turns out that they just forgot about that provision. But it’s out of the question; you can’t fly over a six-lane highway. Then, you have to somehow work out a deal whereby they shut it down for 10 minutes during the show. There are lots of problems—you arrive at the location and the equipment isn’t there yet, or it shows up a day later. We often have space problems, or after we’ve done our recce, we arrive at the location and find that, suddenly, there are obstacles on the field—for instance, a loudspeaker tower right in the middle of it.
Andreas Jalsovec: But at this point, we have the planning & animation software so down pat that we can quickly deal with changes like this. If everything had to be done manually, then we’d never be able to adjust on such short notice.
Chris Bruckmayr: In Sharjah, a bulldozer suddenly showed up and the driver wanted to cross our airfield because it would have been a shortcut for him. Really, the weirdest things can happen. An air show like this is very sensitive, and the agreement has to include detailed provisions covering everything—how much space we need and which areas are off-limits to the public. But this rarely comes off without a hitch; you can pretty much count on the fact that, no sooner do you turn your back, somebody is going to come along and put a crimp in your plans.
Credit: Vanessa Graf
How long does the design process itself take?
Chris Bruckmayr: The whole process from show design to delineation of the precise points along the route that each drone has to negotiate takes at least four weeks.
Andreas Jalsovec: In most instances, far too little time is available for this. The design process differs every time. In Bergen, Norway, for instance, we had the luxury of working with a director who had already caught our act. He had seen the show and was delighted. And then we agreed that he would come by with the musical director who would be composing the soundtrack for the entire opening event in Bergen, and he could see for himself what we’re capable of. We could talk to him and tell him exactly what we wanted—sustained pieces, up-tempo numbers, transitional passages while we’re segueing from one figure to another. The musician also viewed videos of how we animate and how quickly the drones move. So, of course, this makes for successful collaboration.
Chris Bruckmayr: The client, on the other hand, isn’t always on your side. He has his own timing—sometimes, there’s little advance notice with respect to the music, or last-minute revisions. You know, that’s show business—you never have the luxury of knowing three months in advance what piece of music you’ll be working with. Things happen on short notice and that makes it difficult.
Andreas Jalsovec: In Sharjah we were commissioned to fly the drones in conjunction with a stage-based performance—an opera, two hours in length, and we appeared in three scenes. The choreographies of the first two were prescribed, and the director left the third one completely up to us. And, of course, that’s great when you just have the music, and the events on stage are sketched out in general terms, so you can give free rein to your imagination. That’s the ultimate—when you’re given creative latitude. On the other hand, the more freedom you have, the more difficult it is, you know, since then you have to live up to your own expectations.
It’s a creative process. For example, at first, Intel just wanted their signature music and for us to fly their logo, but that was much too brief. First we needed a piece of music as a lead-in, and we created that ourselves. As for the design, my highest priority at that point was three-dimensionality.
Credit: Christopher Sonnleitner
At the 2016 Ars Electronica Festival in Linz, the 100 drones will be making their European debut. What can you tell us about the show?
Chris Bruckmayr: At this stage, we’ve developed our own vocabulary, a sort of aerial visual language. We’ll demonstrate a diverse array of show elements that are really spectacular. We’ll enchant you!
Andreas Jalsovec: Our big problem in Linz is that it’s a long flight from the airfield to where we’ll perform and back again, so we have to make sure to comply with the legal constraints and also stick to our time budget. The musical accompaniment is by Sam Auinger, whose work we’re familiar with, but we haven’t received the piece yet. I think, as soon as we get it, the initial images will start to emerge. Then we’ll brainstorm our ideas and take it from there. There’s no symbol or motif that we definitely have to use for Linz. That gives us creative freedom, but it’s a challenge at the same time.
Chris Bruckmayr: You can also say that the show in Linz will have a lot more to do with the Danube. When we fly over water, as you can imagine, we tend to think a little differently. Of course it’s going to influence the show design when you’re flying over an open body of water. There are no world-famous architectural landmarks as the show’s backdrop, like the 7-star luxury hotel in Dubai or the Sydney Opera House that everyone’s familiar with. In Linz, we’re also working with the specific facts & circumstances on site—that we’re flying over the Danube at dusk. This will be a magnificent setting, with the drone swarm reflected by the water.
You can get a sneak preview of the spectacle in the heavens over Linz shortly before the “River of Knowledge,” this year’s Klangwolke on September 10, 2016, and also find out more after the big show by visiting the Drone Lab at POSTCITY during the Ars Electronica Festival, September 8-12, 2016. There’ll be exhibits, presentations and demonstrations showing the amazing stuff drones can do.