Interactive Art + 2020

Interactive Art + in the Age of Uncertainty

Jury Statement by DooEun Choi, Sabine Himmelsbach, Haytham Nawar, Vladan Joler, Stefan Tiefengraber

In 2016, Prix Ars Electronica expanded their definition of the “Interactive Art” category, broadening the concept of interaction and adding the + sign. Ironically, the 2020 jury found themselves in extraordinary circumstances while judging the submitted works, as the coronavirus pandemic affected transportation globally and imposed social distancing. It is on this shifting ground that the jury decided on the prize-winners. The 2016 category, “Interactive Art +,” is concerned with interactivity, and now, during the pandemic, the jury needed to use social media, and other forms of interactive technology, to view and judge the submitted works. In 2020, there was actually NOTHING outside the interactive.

In the midst of the global pandemic caused by the coronavirus, a jury meeting on site in Linz was no longer possible, borders were closed, and travel opportunities across the entire planet were restricted, coming to an almost complete standstill. The jury meetings shifted to online platforms and took place on video conferencing platforms, overcoming the different challenges such as time differences, physical distance, and technical challenges. It actually worked surprisingly well. Each jury member had to spend time viewing the projects alone, instead of the usual process of viewing the projects with the group in a shared space. Nevertheless, the mutual exchange was intense and constructive, and included animated discussions. All jury members were well prepared, making for a smooth process. Our only concern was the question of interactivity as embodied by each individual artwork, since jury members could not experience the individual works physically. However, the project documentation and the discussions between jury members facilitated the process.

This year’s submissions were very distinct from previous years and were highly influenced by socio-cultural aspects, trying to react to international phenomena. With over 1,300 submissions, the abundance of artistic production in the Interactive Art + category was once again evident. The submissions included interactive installations and videos, biotechnological experimental set-ups, architectural studies, robotics, and artificial intelligence. It is undeniable that technology has become an indispensable part of the artistic canon today, especially in the case of AI projects. It was impressive to see many socio-political interventions within the submitted projects. Artists are reflecting on the topics of labor, surveillance, and censorship that are questioned by current technological advancement. Another important field of reflection is ecology and the issues related to climate change and the steps we need to make regarding a sustainable future. Last but not least, emotions, which are often called the core of human experience, are addressed in many works, as our emotions are extensively manipulated and targeted by algorithmic systems and regimes.
Furthermore, an increase in the outstanding achievements of women artists was welcomed as a positive development, and the jury is very pleased that the Golden Nica will go to a female artist this year, while eight female artists will receive Awards of Distinctions and Honorary Mentions. In addition, the jury was careful to achieve a general parity and to provide a broad and diverse range of the submitted artworks in the category of Interactive Art+. Nevertheless, we hope to reach more diverse applications in terms of geography and ethnicity in future editions. On the other hand, a sensitive approach in the selection of the submitted works and the values that these projects convey was as important as innovative technological aspects.

The quantity and diversity of the submitted works also reflect the views of the individual artists on the category Interactive Art +. It is extremely exciting to see in which direction this art form is developing. The selected works not only show the origins of the definition of interactive art but also indicate the very open approach that this category carries as “+.” As the artworks continue to evolve, new developments are taken into account and a current state of this art form is represented. The new developments are different from classical approaches of human-machine-interaction. In 2020, our ideas about interactivity are no longer linked to specific mediums. As a result of the global pandemic, the meaning of interactivity has been challenged and our understanding of the relationship between humanity, nature, and the machine has deepened, facilitating more profound discussions about humanity in an age of uncertainty.

Golden Nica

Lauren Lee McCarthy (US): SOMEONE

SOMEONE is an interactive installation that uses our smart homes as an opportunity to shed light on the tension between intimacy and privacy as well as convenience and independent action and to ask about the role of the human workforce in a future of automation.


Upon entering the gallery, visitors encounter a command center that resembles a cross between a call center and WeWork coworking space, featuring four computer stations. Each computer looks into one of four participant homes, installed with custom-designed smart devices, including cameras, microphones, switches, lights, and appliances. Visitors are invited to take on the role of a human Amazon Alexa. They may hear smart home occupants call out for “Someone”—prompting them to step in as their home automation assistant and respond to their needs. They can peek into the four homes via the laptops, watch over them, and remotely control the devices in their homes. The first version of this installation was a live remote intelligence portal into four homes across the United States that took place over a two-month period.
We’re sold smart devices that outfit our homes with surveillance cameras, sensors, and automated control, offering us convenience at the cost of loss of privacy and control over our lives and homes. We’re meant to think these slick plastic pieces of technology are about utility, but the space they invade is personal. The home is the place where we are first watched over, first socialized, first cared for. How does it feel to have this role assumed by artificial intelligence? Our home is the first site of cultural education—it’s where we learn to be a person. By allowing these devices in, we outsource the formation of our identity to a virtual assistant whose values are programmed by a small, homogenous group of developers. It is hard to imagine any living person with a personality like Alexa or Siri. AI assistants lack the flaws and inconsistencies of human personalities. Right now, virtual assistants are designed to accommodate the most common and universal needs. Imagine if instead they attended to our particular, obscure needs and desires. They could probe beyond what we expect of these technologies, into the types of help we might feel able and comfortable to ask only of technology. This points to possibilities that are both exciting and worrying. This installation does not impose a judgment but creates space for the viewers and inhabitants to form their own opinions.

While designing the project, I spent a lot of time thinking about the question, “If I were an AI, what would I be like?” I aimed to create SOMEONE as an entity that felt human but could also function like a system. Rather than speaking to people directly, visitors used a text-to-speech interface to invoke a synthesized voice. Rather than feeling like a guest to be accommodated in the home, they could more easily integrate into the infrastructure of the environment.
The devices were based on research into existing smart home and IoT devices, while exploring more human and imaginative forms and functionalities they may take on. The custom device objects incorporate cameras, microphones, and sensors, and address the needs of the inhabitants by functioning as smart switches, lights, locks, environment adjusters, small appliances, and more surreal mechanisms of control. Custom software, built with node.js, runs on a Raspberry Pi computer that serves as the central hub for the system, connecting all the devices and enabling them to be interactive and remotely controlled.

By substituting humans for AI, the role of virtual assistant is re-contextualized. As inhabitants call out for “Someone,” invoking visitors as intelligence, the dynamic between audience and performer is complicated. Installed simultaneously in multiple homes across the country, we’re challenged to consider the scale of the work, and the even more expansive, networked systems that structure society. SOMEONE is a meditation on the smart home, the tensions between intimacy vs privacy, convenience vs agency, and the role of human labor in the future of automation.

Credits:
Artist: Lauren Lee McCarthy
Software and hardware development: Harvey Moon, Josh Billions
Interface Software: Lauren Lee McCarthy
Furniture design collaboration and fabrication: Lela Barclay de Tolly
Smart home participant collaborators: Valeria Haedo, Adelle Lin, Amanda McDonald Crowley, Ksenya Samarskaya
Photo: Lauren Lee McCarthy

Jury Statement:

The artist Lauren Lee McCarthy questions the role of the human in the age of intelligent machines by examining communication media and re-imagining them. For McCarthy, the SOMEONE is the human version of Amazon’s Alexa, staging an interaction between the public audience in the gallery and people at home. The human-to-machine interaction is reversed to become a human-to-human interaction as filtered through a machine. During the global pandemic, physical interaction between humans has been challenged and virtual interaction between humans mediated through the machines has been tested in our everyday life in an unprecedented way. To navigate an uncertain future, this work raises awareness of the new, emerging challenges of the 21st century. It suggests relevant ideas about what might constitute the definition of intelligence itself by revisiting the social and emotional intelligence of humans as they were, but now couched within a machine-intelligent system.

Award of Distinction

Simon Weckert (DE): Google Maps Hacks

Google Maps Hacks

99 secondhand smartphones are transported in a handcart to generate a virtual traffic jam in Google Maps. Through this activity, it is possible to turn a green street red which has an impact in the physical world by navigating cars on another route to avoid being stuck in traffic. The advent of Google’s Geo Tools began in 2005 with Maps and Earth, followed by Street View in 2007. Google’s map service has fundamentally changed our understanding of what a map is, how we interact with maps, their technological limitations, and how they look aesthetically.
In this fashion, Google Maps makes virtual changes to the real city. Applications such as Airbnb and Carsharing or Uber have an immense impact on cities: on their housing market and mobility culture, for instance. There is also a major impact on how we find a romantic partner, thanks to dating platforms such as Tinder, and on our self-quantifying behavior, thanks to the Nike jogging app. An additional mapping market is provided by self-driving cars; again, Google has already established a position for itself.
All of these apps function via interfaces with Google Maps and create new forms of digital capitalism and commodification. With its Geo Tools, Google has created a platform that allows users and businesses to interact with maps in a novel way. This means that questions relating to power in the discourse of cartography have to be reformulated. What is the relationship between the art of enabling and techniques of supervision, control, and regulation in Google’s maps? Do these maps function as dispositive nets that determine the behavior, opinions, and images of living beings, exercising power and controlling knowledge?

Credits:
Foto: Simon Weckert

Jury Statement:

Google Maps Hacks by Simon Weckert is a great example of the contemporary situationist practice combining performance, social activism, obfuscation, and hacking. In this action, 99 second- hand smartphones are transported in a handcart to generate a virtual traffic jam in Google Maps. Through this activity, he is turning a green street red, which has an impact in the physical world by navigating cars along another route to avoid being stuck in traffic. With this intervention Simon Weckert is basically giving us a simple DIY recipe on how to interact and influence invisible algorithms behind the Google Map, a proprietary platform that is shaping our movements and behavior. We believe that actions like this, that are experimenting with the strategies on how to obfuscate and intervene in data flows, is of great importance for our present and the future in the age of surveillance capitalism and digital totalitarianism.

Award of Distinction

Lynn Hershman Leeson (US): Shadow Stalker

Shadow Stalker

Lynn Hershman Leeson (US): Shadow Stalker

Shadow Stalker is a “live” interactive installation that uses original algorithms, data mining, live performances, and projections in order to make visible normally private internet information. This project directly reveals an individual’s “digital shadow,” providing access to online internet information, which is crucial as data has surpassed oil as the world’s most valuable commodity. Shadow Stalker directly exposes hidden surveillance systems that now dominate police arrests and highlights the consequences of geographic and economic limitations these systems cause. Audiences will be educated about options for understanding how this software functions and have access to identify some of the limitations that are surreptitiously being placed on their freedom. Increasingly, law enforcement promotes racial profiling and employs flawed artificial intelligent logic to target low income or non-white individuals as possessing criminal intent, often leading to their arrest before a crime occurs.
This is how it works: participants stand inside a “red square” and enter their email on an iPad, a surveillance camera captures their shadow and exposes a chilling sense of their own vulnerability to this kind of data-mining, revealing, in complete sentences, online information inside their shadow, which tracks progressive data in real time as they move through the installation where a video projection warns about invisible surveillance systems and algorithmic violence that robs people of their civil rights. The video advises how to take control and own their online digital profiles and independence. Government issued and tracked red squares instigate a coded prison. Yet the red square has historically been a place of revolution.
Specially designed algorithms and data mining systems define people by race, economics, or gender. There is an implicit urgency now to expose the existence of these tactics and develop a means of revealing and correcting the global damage these dangerous and violent algorithms are causing.

Credits:
Shadow Stalker by Lynn Hershman Leeson
Commissioned by The Shed with additional funding from VIA Art Fund
Programming:
Original algorithms programming: Mark Hellar
Interactive Developer: Rachel Rose Ulgado
Algorithmic support: Alister Ashe
Conversion of zip code algorithms for White Collar Crimes: Francis Tseng, Sam Lavigne, Brian Clifton
Foto: Dan Bradica. Courtesy The Shed

Video:
Director, writer, editor, producer: Lynn Hershman Leeson
Performer: Tessa Thompson
Shadow of the Deep Web: January Steward
Camera: Hiro Narita
Sound: Javid Soriano
Costumes: Nina Hollein
Compositing and color corrections: Gary Coates
Additional editing: Erica Jordan and Collin Kriner
Production team: Marine Macerot Bahet, Emma Scully
Background score: Mathew Young
Production assistants: Alexandra Phelps, Theadora Walsh, Lor O’Connor
Set photographer: Pamela Gentile, Attorney George Rush

With support from: VIA Art Foundation, The Shed, Hotwire Productions, Fine Arts Museum San Francisco.

Jury Statement:

Shadow Stalker is an interactive installation by American media art pioneer Lynn Hershman Leeson, based on personal data that allows visitors to experience their data shadows in real time. Based solely on the use of just their email address, personal information about the participants retrieved from internet databases is displayed within a generated digital shadow. Further elements of the installation include a film that focuses on Data Mining, predictive policing, and identity theft, and a website that presents the percentage of predicted financial crimes by an algorithm, which is presented by zip code. The jury was convinced by the conceptual strength of the work, the highly political topic, and the awareness it raises for surveillance issues and racial profiling. The jury also recognizes the formal quality of the work, how visually compelling the whole installation is and how it enables the audience to engage with the important subject matter by becoming an active part of it.

Honorary Mentions

999 years, 13 sqm (the future belongs to ghosts) / Cécile B. Evans (BE), in collaboration with biologist-architect Rachel Armstrong (GB)
999 years, 13 sqm (the future belongs to ghosts) is an installation by the British artist Cécile B. Evans and Rachel Armstrong, professor of experimental architecture. The installation reflects on the topic of housing and is composed of two parts: a ‘living’ wall that houses several elements that create an organic battery system which is powered by microbes and provides small amounts of electricity that run the electronic devices, and an animation of a dying bird, projected onto a misty curtain of fog. The jury was convinced by the strength of the artistic concept, which reflects the severe challenges in terms of the housing situation, future living scenarios, and the need for co-habitation with other entities. The piece confronts us with a dying life form and a living entity, to whose care we must contribute by active engagement. It enables critical reflection as well as a poetic encounter.

Algorithmic Perfumery / Frederik Duerinck (NL)
Algorithmic Perfumery is an interface that allows the user to create their own personal scent combination based on the data they input through a questionnaire. The machine processes this data and generates an individual code that is processed to create a particular fragrance. This work by Dutch artist Frederik Duerinck offers a new narrative to the future of perfumery and allows the visitor to take away a physical souvenir of the experience. After testing the generated scent, the visitor can communicate their feedback to help the algorithm learn and refine itself.
The jury was persuaded by the multisensory nature of this work and how the experience is both conceptual and physical, making this project particularly attractive. The user is at the center of this project and each experience, although following the same meticulous process, results in a different and unique product. This artwork gives a glimpse into the future of perfumery and challenges the concept of mass production. By interacting with the machine, the user possesses the power to be an active part of the creation process.

Appropriate Response / Mario Klingemann (DE)
Appropriate Response is an interactive installation that is concerned with meaning, expectation, and relationship with AI. The visitor is presented with a traditional wooden kneeler facing a split flat display that generates random sentences. The AI system is triggered when the visitor uses the kneeler and displays a unique sequence of words. Through his pertinent work, German artist Mario Klingemann explores the significance attached to written words.
The jury found it witty and particularly relevant that the entire experience is transformed into a ritual that is very similar to religious experience. Although this interaction between Human and AI results in unique coherent aphorisms, it is how the human understands it and makes sense of it that gives this interaction meaning. The artwork is built on expectation, hope, and fear—feelings that associate the generated words with feelings, giving them a new life.

Center for Technological Pain / Dasha Ilina (RU)
Center for Technological Pain offers, as the artist states: “multiple practices for eliminating tech-pain.” Wrapped in a diverse range of self-built objects and solutions, and a DIY approach, an archive around workshops and tutorial videos like “Yoga for Healthy Phone Use” is created, to start a critical discussion about the negative effects that technology can have on our bodies. The artist’s approach is not to condemn new technologies and to ban them from our lives, but to investigate how we can live in symbiosis with digital technologies without being harmed.
The jury was convinced that this humorous approach to a—in a certain context—serious topic, plays with a contemporary problem that affects the rapidly growing space digital technologies take in society. In addition, the topics of open commons and DIY culture are addressed, which are becoming increasingly important, especially in the current situation.

Compression Cradle / Lucy McRae (GB)
Compression Cradle is a machine that gently squeezes the body and holds it tight. The Australian-British artist Lucy McRae has developed a bed or ’cradle’ that the visitors can step into. Slowly the air is let out and the participant is softly held by a blanket, as if in a tender embrace. The immersive artwork reflects on a touch-deficit status of our mediated society, where people increasingly seek closeness through machines, but in the pandemic that currently shakes the whole world, becomes even more an attempt to prepare the self for a future without human touch. The jury was convinced by the urgency of the work, its critical reflection on our relationship towards technology, and how it addresses our needs for human connectivity. The work offers the audience new perceptual experiences that—especially in our troubled and challenging times—open space for reflection and thought.

khipu / electrotextile prehispanic computer / Constanza Piña Pardo (CL)
The artist Constanza Piña Pardo, also known as Corazón de Robota (Robot Heart), is a member and co-founder of different feminist collectives and labs and a very active performer in the field of sound art who shares her knowledge of open hardware in workshops and on her website. In her project khipu / electrotextile prehispanic computer she relates to the ancient conclusion that beyond the visible sky lies a calculating power that organizes everything. The result is an open-source textile computer based on an astronomical Inca khipu, an early computing system. Produced in an experimental creation laboratory together with five other women, the artist uses her artistic approach to interpret technology combined with her research about the resemblances of this system of knots and contemporary binary coding. The jury acknowledged that the interaction of this project does not only take place in the exhibition space, but also in the creation of the artwork, in the artist’s community, where old and almost forgotten knowledge is preserved by transferring it to our time by following a DIY and open commons approach.

Machine Auguries / Alexandra Daisy Ginsberg (GB)
With this work the artist shows the easily overlooked man-made influences and their effects on the environment, especially in densely populated areas. Light and noise pollution influences the behavior of the birds necessary for a balanced ecosystem. They have to adapt their behavior to survive alongside humans in this environment. Alexandra Daisy Ginsberg created with Machine Auguries an immersive environment in which the visitor can experience the singing of birds accompanied by changing light moods according to times of the day. Parts of the audible bird sounds are artificially created and mixed with the singing of real birds—communication between birds and the machine that tries to hear and respond to the voices with the help of machine learning and is thereby creating a new language. The crescendo that is audible through a multi-channel sound installation is also reflected by the trained neural networks that are responding to the real birds and slowly taking over at the end with pure artificial bird songs not distinguishable from real bird songs.

Mind / Shinseungback Kimyonghun (KR)
Emotions which were thought to exist only in the realm of the human, are captured here in Mind by the artists Shinseungback Kimyonghun. In contemporary culture, collective emotions are instrumentalized, classified, analyzed, and manipulated by machines. Mind examines captured facial expressions and data collected by machine intelligences such as FER+ Emotion Recognition and ONNX (Open Neural Network Exchange) Runtime. A collective “meta-mind” generated by the last one-hundred faces of visitors to the exhibition site, is generating algorithmic patterns. These algorithmic patterns are used to create sound, which acts upon a series of moving plates filled with small metal beads. The sounds and beads move in tandem, all generated by the emotional expressions of exhibition visitors. Shinseungback Kimyonghun describe this work as “a sea of emotions.” The jury felt that the piece expressed the power of art to help people with the emotional challenges of the current global pandemic, suggesting that, collectively, humanity can navigate this difficult time.

The Intimate Earthquake Archive / Sissel Marie Tonn (DK) in collaboration with Jonathan Reus (US/NL)
The Intimate Earthquake Archive by Sissel Marie Tonn is an ongoing research project and interactive installation dealing with the phenomenon of man-made earthquakes. In her work she is offering us the chance to experience “deep listening” of the effects of fossil fuels exploitation. Even this project has a strong local dimension because it deals with the site-specific problem that is a result of the gas drilling in one of the largest gas fields in Europe. It is also provoking dialog in a wider ecological and socio-political context of the human role in the environment undergoing changes. By creating tactile sensory experiences from the collected seismic recordings she is offering us the chance to feel those changes and effects with our bodies.

The Net Wanderer – A tour of suspended handshakes / Cheng Guo (CN)
The Net Wanderer by Guo Cheng is an artistic research project that explores the connection between the critical network gateways in China, known as the Great Firewall of China (GFW), and the infrastructure behind these gateways. This project investigates the materiality of the invisible but one of the most influential technological borders that shape territoriality of the internet nowadays. In this project, which combines artistic and technical research, his personal investigative journey into the physical locations determined by the network diagnostic tools are transformed into the installation that lets us become virtual firewall tourists. Even as this project reveals some of the aspects of this complex and invisible infrastructure, it also reveals to us many obstacles and limitations of our capacity to investigate those systems from the position of the artist and independent researchers.

Trickle Down, A New Vertical Sovereignty / Helen Knowles (GB)
Trickle Down, A New Vertical Sovereignty is an installation composed of different elements: a tokenized four-screen video installation, a generative soundscape attached to the blockchain, and a machine. The experience starts when the user puts a coin in a machine that converts it into crypto-currency and distributes it on the Trickle Down community (prisoners, Ethereal Summit attendees, employees at a blockchain company, Mancunians at Openshaw market, Manchester, and the Russian community in central London) who participated to the project. The space is equipped with sensors that are triggered by the visitor’s location, making each experience individual.
The jury agreed that what’s intriguing in this work by British artist Helen Knowles is that every element of the installation is visible to the visitor, making their experience transparent and revealing what is usually hidden in real life. The artwork denounces the technological and financial power structures that support the disparity between a wealthy elite and everyday working people and attempts to imagine a more horizontal approach.

Warriors / James Coupe (GB/US)
In the 2020 Prix Ars Electronica, there were many submissions related to AI, and *Warriors* was selected due to its thematic strength and the level of interaction elicited from viewers whose faces were captured and then assigned to cinematic, filmed actors, composited in real time into the cult film *Warriors*. Coupe’s piece blurred the boundaries between the real and the fake by using the deepfakes algorithm, which is based on a Generative Adversarial Network. The work questions the biases built into the current profiling systems used to construct AI. Viewers are invited to experience the face swap, and the impact of technology is no longer abstract and objective but something intimate and psychologically disturbing.

The Golden Nicas of “Interactive Art” since 1990