During the development of the Spaxels and the Fluxels, Futurelab researchers faced challenge after challenge in organizing, controlling and experimenting with swarms. Being the first in something means that it’s up to you to invent purposes, languages and, of course, tools to work with it.
swarmOS was born from a simple idea: An operating system for swarms. The task of an OS is to abstract away the complexity of different hardware that needs to talk to each other, and to offer interfaces that make it easy to focus on creating and using applications for a variety of scenarios. The core parts of SwarmOS are Ground Control, a cross-platform application to monitor and control swarms, and the Implant, a component that is attached to a drone or robot (either as a physical device or a software component) to make it “talk” to Ground Control. Off-the-shelf drones and bots can be compatible with swarmOS if they support specific protocols. At the same time, the Futurelab’s own spx protocol is also available, and can be implemented on vehicle firmware if it is a system that has special requirements or does not support the standard protocols.
The first version of swarmOS was developed in 2017 as a step-by-step remodeling of the existing Ground Control system (that was only compatible with the original Spaxels hardware). Around the point of the first mixed-swarm performance (Swarm Arena 2018), Ground Control and swarmOS were released as version 2 – incorporating all the significant characteristics described above, and verified in a performance with drones and robots controlled in real time by a single system.
Ground Control
Ground Control is a cross-platform application that offers a complete set of features to monitor and control a swarm of compatible vehicles. Its modular UI allows the user to configure panes and views exactly how it suits them best for their use case. The UI centerpieces are the 3D view and the QuickStateUi: In the 3D view, a full visual representation of the swarm and the environment is rendered, and the user can select vehicles and send waypoints. QuickStateUi is the compact representation of all the most important information for each drone, and highlights everything that most likely requires the operator’s attention in different colours according to the level of urgency. Many other UI components provide more detailed information and configuration options.
Other central features:
- The application can be configured to use “live” drones, simulated drones or a combination. Simulating drones is essential for trying out new scenarios and choreographies.
- Fleet composition, drone capabilities and communication systems can be freely configured at startup.
- Ground Control can run in a distributed configuration, making it possible for multiple operators to focus their attention on a subset of the swarm.
- Any session can be recorded, yielding a complete representation of a swarm mission that can be replayed in the 3D view and analyzed with tables and highly customizable graphs after the fact.
- Choreographies can be loaded directly into GC and played back (i.e. sending waypoints to drones)
- GC provides TCP and UDP network endpoints to which a separate application can connect via LAN to receive the swarm state and send commands (such as waypoints). Using this, it is easy to create complex scenarios of multiple choreographies and interactive scenes in, for example, the Unity game engine (for which we provide templates).
Implant
The attaches to the vehicle and connects it to swarmOS. Through version 2 of swarmOS, the implant is a special software on a pixracer board. Its core tasks are:
- Managing wireless bidirectional communication between Ground Control and the drone’s “autopilot” (low-level software that deals, among other things with motor control) – in particular, waypoints are forwarded to the autopilot and status information such as GNSS location or battery voltage is reported back.
- Making autonomous safety decisions: When a certain perimeter (“geofence”) is left, when communication fails or other exceptional situations occur, the implant can (and must) decide how to act, and in many cases (especially on communication loss) enter a mode different from the last valid instruction it received: It may decide to return to its home position, for example, or land/stop where it is.
The core philosophy and particular distinction of the implant concept is that the drone firmware (autopilot) does not have to be manipulated to allow integration into a swarm control system.
Other components
Apart from components specific to the special 2.4GHz Laird system that can be used for communication, there are several other software components that have been created to facilitate swarm management: fleet-utils and swarm-sync are utilities that make it easier to keep a swarm of vehicles updated with software versions and files, and keep track of all their configurations. These are specific to linux-based platforms and will integrate with into the new implant – see below.
swarmOS v3.0
Since late 2021, we have been working on the next version of swarmOS, with a complete refactoring of the Implant and major other components, with the following main goals:
Modular fail-safe communication
Indoors or outdoors, Europe or Asia, quadcopters or ground bots – the environments and communications parameters can vary greatly between swarm performances, and there is no one communication channel that is the best solution in all cases. We are equipping swarmOS with a modular communication system: The user can pick any number of platforms, for example combining specialized 2.4 ghz radio, wifi and LTE. The system will always select the most recent information and in case of an issue with one of the channels, the others can continue to stay connected to the swarm. For all IP-based communication methods, an additional intermediary server called swarm-relay is used. This process may run on a cloud server or in a LAN via a fully portable docker setup.
Multi-platform implant
The implant used to be a special firmware running on one particular platform (the Pixracer board). Now, the implant software can also run on generic ARM linux platforms such as NVIDIA Jetson and Raspberry Pi. This means that, if you have such a platform on a drone or bot, you can run the implant directly on it, and need nothing more than a laptop and any kind of wireless IP-based network to get started with your swarm – no special swarmOS hardware. With this, swarmOS will become much more accessible to our partners and the wider swarm robotics community.
The Future
The origin of swarmOS lies with choreographies, and its philosophy is one of centralization. Swarms will be designed in evermore autonomous ways and the true potential of their seamless, intelligent integration into human space will unfold with such increased individual agency and decentralization. swarmOS will keep evolving as it must be able to drive our own research into making swarms smarter and more independent, without sacrificing the ability for humans to clearly instruct and always intervene when necessary.
Swarm Arena / NTT (JP), Ars Electronica Futurelab (AT)
Foto: Tom Mesic
Related Projects
