Gigapixel Editions: Noonday Sun / Afra Eisma, Catalyst Lab, Deep Vision; photo: Christian Kain

Gigapixel Editions

Catalyst lab – The Making of Tomorrow

In celebration of the 50th anniversary of the European Patent Convention, the European Patent Office (EPO) invited the Ars Electronica to co-curate an exhibit space in its headquarters in Munich, Germany: the Catalyst lab. The EPO holds an extensive collection of artworks, many of which have been digitized and archived using ultra-high-resolution photography and digital image mosaics. The Ars Electronica was asked to select three artworks, digitize them, and present them as a video installation on a 12-meter-wide projection screen called “Deep Vision”. The technical challenge of the project was to find a way to animate ultra-high-resolution images with up to 90 billion pixels as a video. 

The space in the basement of the EPO’s headquarters is almost 1.000 m2, originally housing the registry for patent applications. With the digitalization of patent granting procedures, the space was no longer needed for this purpose. In collaboration with the Ars Electronica, it was transformed into Catalyst lab – an enjoyable space for staff and visitors with a forward-looking concept that also preserves the EPO’s DNA as a patent archive. It addresses today’s global challenges ranging from AI to sustainability and explores the transformative impact of technology on society.  

Ars Electronica collaborated on content for the Deep Vision projection environment. Under the name Gigapixel Editions, selected artworks from the EPO’s collection were digitally archived using ultra-high-resolution photography and digital image mosaics. This technique combines several high-resolution images into a single digital representation of each artwork. This reveals details that would otherwise not be visible to the naked eye. The first three artworks from the collection that were opened to the public in this way are Paul Leitner’s Sucrologist (2016), Afra Eisma’s Noonday Sun (2019) and Ivan Šuletić’s CFRP Cityscape XI (2019). 

The technology behind Gigapixel Editions 

The animation and rendering process of a gigapixel video is particularly interesting. Most animation software has a certain limit on the resolution of images it can handle. This is intended to not overload the PC’s resources and ensure at least a decent framerate when playing the animation back. In our case we were using Adobe After Effects which can handle files of up to 30,000 x 30,000 pixels. That’s 900 million (900,000,000) Pixels in total. However, the largest image we received which needed to be animated had 90,000 x 100,000 pixels, which is 90 billion (90,000,000,000) pixels.  

Without preparation, After Effects is unable to handle such a large resolution. If you try, you simply receive an error. But what if we can trick the software into thinking it is handling a lot of smaller images that just happen to be arranged in a neat grid to make up the larger original image? This way it would only need to keep in memory what is shown on the canvas and not have to worry about the rest. This meant splitting up the image into smaller parts. 

Prep work

Luckily Photoshop can handle resolutions of up to 300,000 x 300,000 pixels, meaning it was able to open the gigapixel stills we received for animating. This created the possibility to divide the originally huge file into individual tiles with a resolution of 10,000 x 10,000 pixels, which could be imported and stitched back together in After Effects. 

After doing so, the performance was good, but not great. Another workaround was necessary. By creating a single low-resolution version of the original file (1/8th the size), importing that file to After Effects and scaling it back up by 800%, we receive a proxy image that looks a bit blurry but performs well because it is much smaller. Finally, the grid of large resolution tiles was attached to the proxy file and set to invisible, so After Effects would not load them into memory for each frame while animating. 

Animation

By animating the proxy files’ position and zoom level, we effectively also animate the large resolution grid since it is attached to the proxy. The animations were created according to the content of the individual gigapixel images, drawing special attention to more interesting areas. Special attention was paid to keeping the movement rather slow since the final presentation output was a very large projection surface where the audience would stand directly in front of, to not induce motion sickness. 

Rendering

Rendering such large videos is not very common, so it came with its own challenges. Since the playback format was not clear to the client for a rather long duration of the project, multiple output formats were made for testing. The target resolution was 9480 x 2160 pixels at 60 frames per second. Among other formats, image sequences were rendered to preserve as much quality as possible. These ended up being 16,200 frames long, which added up to around 260GB per sequence and didn’t playback very well. Finally, the h264 code was chosen and rendered at a high data rate as an mp4. This produced visually satisfying results and performed well during playback – resulting in a great experience for visitors that can dive deeply into the artworks. 

Credits

Ars Electronica Futurelab: Patrick Berger (video production and editing)
Ars Electronica: Laura Welzenbach (coordination), Karl Schmidinger (sound and music)
PARTNER: Florian Voggeneder (Gigapixel photography)
Original artworks: Paul Leitner – Sucrologist (2016), Afra Eisma – Noonday Sun (2019), Ivan Šuletić – CFRP Cityscape XI (2019)
Commissioned by the European Patent Organisation in the framework of the Ars Electronica exhibition Catalyst Lab 2023