Honorary Mention
VFRAME is a computer vision project that develops open source image processing software and neural network models for human rights related research and conflict zone monitoring. Started in 2017 with the goal of bridging the gap between industry between commercially aligned AI and the needs of investigative research, VFRAME is now a pioneer in the development and application of new techniques that combine 3D-photogrammetry, 3D-rendering, and 3D-printing to generate synthetic data for training neural networks.
Instead of following industry trends to scrape data from online sources, which can inherit problematic biases, VFRAME uses an artist-first approach that combines digital fabrication, sculpture, photography, and 3D-artistry to create a virtually unlimited source of training data. The result is powerful computer vision models that can automate the detection of illegal cluster munitions in million scale video datasets from conflict zones. Since the beginning, VFRAME has partnered with Mnemonic to apply this technology to their large archives.
Our first successful pilot project during 2022 resulted in the detection of over 1,000 videos containing the RBK-250 cluster munition bomb in their Syrian Archive, with over 3 million videos and approximately 10 billion image frames, a task that would otherwise be impossible. Last year also marked the beginning of a new collaboration with the NGO Tech 4 Tracing to gain direct access to real munitions for 3D scanning. This important new step allows the VFRAME project to finally scale up development efforts to build more computer vision detection models for application to conflict zone monitoring, which are all available open-source.
Credits
Director, founder, computer vision: Adam Harvey
3D design and emerging 3D technologies: Josh Evans
Information architecture and front-end development: Jules LaPlace
With support from: Prototype Fund (Bundesministerium für Bildung und Forschung); NLNet Foundation and Next Generation Internet (NGI0); NESTA; SIDA; Tech 4 Tracing
Biographies
Adam Harvey (US/DE) is an artist, software engineer, and applied researcher based in Berlin, focused on computer vision, privacy, and surveillance technologies. He is a graduate of the Interactive Telecommunications Program at New York University (2010) and Pennsylvania State University (2001). Harvey’s research and artwork has been featured in prominent media publications including the New York Times, Wall Street Journal, Nature, New Yorker, Frankfurter Allgemeine Zeitung, Süddeutsche Zeitung, Washington Post, Le Monde, the Guardian, BBC, Economist, and the Financial Times; and shown at internationally acclaimed institutions and events including the V&A museum (UK), Seoul Mediacity Biennale (KR), Istanbul Design Biennale (TK), Frankfurter Kunstverein (DE), Zeppelin Museum (DE), Utah Museum of Contemporary Art (US), and Kemper Museum of Contemporary Art (US).
Josh Evans (US/DE) Experienced digital imaging technologist working with real-time and raytraced graphics pipelines, DSLR photogrammetry, 3D printing, digital asset modeling and scene reconstruction for machine learning tasks including object detection, augmented reality, and emergent computer vision applications.
Jules LaPlace (US/DE) is a programmer who makes websites and creative interfaces. Their software has produced artworks shown at the Venice Biennale (IT) and the Centre Pompidou (FR).
Jury Statement
Human rights and conflict zone monitoring is a domain of research in itself—and even one of commercial industries. To establish monitoring of conflicts beyond the bias of industry and policy interests, VFRAME pioneers an artistic approach to 3D-photogrammetry, 3D-rendering, and 3D-printing, to generate synthetic data for training neural networks. With 3D-artistry monitoring software can be trained to be unbiased on unlimited data – and then it can monitor conflicts in a less biased or even an unbiased way. Art for Peace is a long-standing mission of creators of all times. VFRAME now re-frames the use of art, software, and reality to create a computer vision model to monitor harsh conflict realities. In 2022 VFRAME helped detect over 1,000 videos in the Syrian Archive that contained the RBK-250 cluster munition bomb. Artist-driven research on conflicts, using software and 3D printing, creates novel chances for peace in a world, where the digital and real dimensions of war (e)merge more and more.