They almost make us forget the world around us: Virtual Reality (VR) applications let us dive deep into images and videos, promising interactive experiences. To lead immersive media from a niche phenomenon to a more widespread practice in the consumer market, the three-year EU-funded project Immersify took to the task of research and development with five European partners, including the Ars Electronica Futurelab.
The team worked on improving every aspect involved – from video technology, codecs, displays and media rooms to specially created content and artistic visions. Immersify also fostered open innovation, promoting media content and tools for the creative and media industry in Europe. The declared goal: to enable and create media more immersive than ever before.
In addition to the Ars Electronica Futurelab from Austria, Poznan Supercomputing and Networking Center (PSNC) from Poland, Spin Digital Video Technologies GmbH from Germany, Marché du Film – Festival de Cannes from France and the Visualization Center C from Sweden were part of Immersify. The program was funded by the EU program “Horizon 2020”, going from October 2017 to June 2020. The challenges were manifold: The ever increasing amounts of data for compelling immersive content called for improved data compression. Media Players and formats for as many different technical environments and devices as possible were necessary. And techniques such as ultra-high resolution or CGI in 2D and 3D needed to be freely combinable with each other.
Apart from the mainly technological R&D work, the Immersify team also explored participation and interaction for VR applications, complicating matters: What may still be feasible for an individual becomes tricky as soon as the audience is larger. Immersive media rooms like the Deep Space 8K at the Ars Electronica Center in Austria or Visualization Center C in Sweden already give a taste of what could be possible in the future.
Immersify: The translucent St. Stephen’s Cathedral photo: Robert Bauernhansl
Singing Sand 2.0 / Tadej Droljc (SI) Credit: Ars Electronica / Robert Bauernhansl
IMMERSIFY: Next Generation Immersive Media at Deep Space 8K Credit: Magdalena Sick-Leitner / Pano LA 10K – Joe Capra / Scientifantastic
Singing Sand 2.0 / Tadej Droljc (SL) Credit: Ars Electronica – Robert Bauernhansl
The Great Pyramid in 3D photo: Robert Bauernhansl
The outcome
The Immersify team produced around 25 immersive contents to demonstrate technical possibilities as well as to evaluate and improve the developed tools. Demonstrations including 16K media playback and 8K live streams, showcased for example at the Deep Space 8K, made the project tangible for the general public as well as the immersive media market. To strengthen this approach, Immersify was presented not only at conferences and technology fairs, but also at cultural events and festivals like the Ars Electronica Festival and Cannes XR Virtual at the Marché du Film in Cannes. To support content creators and open innovation, several guidelines for imitating the tools and content of Immersify were published.
Several of the projects realized with Immersify gained wide attention and audience by being shown at the Ars Electronica Center’s Deep Space 8K. The translucent St. Stephen’s Cathedral is an artistic representation of the cathedral, based on more than 20 billion laser points. The Great Pyramid in 3D: From the BBC series, Ancient Invisible Cities allows visitors to dive deep into the Great Pyramid of Giza, via an interactive immersive exploration. Several projects included working together with artists on stereoscopic audiovisual journeys like Prima Materia 8K by the Istanbul-based multidisciplinary Studio NOHlab and Singing Sand 2.0 by Tadej Droljc. Another part were professional time-lapse videos in high-resolution, like Atacama Desert and Island in the sky II in 8K resolution as well as Pano LA 10K in a whopping 10K x 4K resolution. Another project was tx-reverse 360° with material from Virgil Widrich Film- und Multimediaproduktions G.m.b.H. – a space time cut through cinema in 10K resolution, rendering 10.000 images with a width of 10.000 pixels 10.000 times.
Impressive demonstrations
In June 2020, the project team concluded Immersify with a live demonstration of highly immersive and interactive media. Newly created content was shown, like a 16K 360° video combined with spatial audio based on high-order ambisonics. The event also included an 8K live stream from PSNC’s laboratories in Poznan, Poland, to Ars Electronica’s Deep Space 8K in Linz, Austria. A 360° live video stream from PSNC showed the technology behind the stream and ambisonic sound production. Live demonstrations of some of the produced content can be re-watched on the Ars Electronica’s YouTube channel.
To learn more about Immersify, it’s objectives and partners as well as the many different contents that came to life, visit the project website. There you also find a dev kit, guidelines, reports and best practise examples aiming to help with the advancement of high-quality immersive media.
You can also find more information on Immersify on the Ars Electronica Blog.
Research Files
Research Articles
- Brzeźniak M., Wadówka K., Woszuk P., Blicharczyk F., Płaczek M., Głowiak M., Stróżyk M, “Sync&share solution for massive multimedia data”, TNC18 Conference, Trondheim, June 13, 2018, DOI: 10.5281/zenodo.3973099, Permanent link: https://zenodo.org/record/3973099 Download Paper
- Kurowski K., Glowiak M., Ludwiczak B., Stróżyk M., Ciżnicki M., Binczewski A., Alvarez-Mesa M., “PSNC advanced multimedia and visualization infrastructures, services and applications.”, Web3D ’18 Proceedings of the 23rd International ACM Conference on 3D Web Technology, Poznan, June 20-2018, DOI: 10.5281/zenodo.3973105, Permanent link: https://zenodo.org/record/3973105 Download Paper
- Alvarez-Mesa M., Ching Chi C., HEVC Software Media Player for Ultra-High Quality Video: 8K and Beyond, The 25th International Display Workshops. December 12-14, 2018. Nagoya. DOI: 10.5281/zenodo.3973090, Permanent link: https://zenodo.org/record/3973090 Download Paper
- M. Alvarez-Mesa, S. Sanz-Rodriguez, C. C. Chi, M. Głowiak, R. Haring, Advanced Video Compression and Rendering for Highly Immersive 8K+ Applications, IBC Conference 2019, DOI: 10.5281/zenodo.3973079, Permanent link: https://zenodo.org/record/3973079 Download Paper
- M. Dąbrowski, M. Głowiak, E. Skotarczak, M. Stróżyk, Nowe technologie obrazu i dźwięku w praktyce produkcyjnej (New image and sound technologies in practical production), chapter in book B. Kostek: Postępy badań w inżynierii dźwięku i obrazu. Nowe trendy i zastosowania technologii multimedialnych (Progress of research in sound and image engineering. New trends and applications of the multimedia technologies), Warszawa, Akademicka Oficyna Wydawnicza Exit, ISBN 978-83-7837-89-5, 2019, DOI: 10.5281/zenodo.3973107, Permanent link: https://zenodo.org/record/3973107 Download Paper
- M. Głowiak, M. Stróżyk, S. Malewski, M. Alvarez Mesa, S. Sanz-Rodriguez, Advanced compression and streaming tools for ultra-high definition immersive media with live experimentation, NEM Summit 2020 conference, 2020, DOI: 10.5281/zenodo.3973092, Permanent link: https://zenodo.org/record/3973092 Download Paper
Credits
Ars Electronica Futurelab: Roland Aigner, Roland Haring, Anna Kuthan, Christopher Lindinger, Johannes Lugstein, Nicolas Naveau, Ali Nikrang, Johannes Pöll, Clemens F. Scharfen
PARTNER: Poznan Supercomputing and Networking Center (PSNC); Spin Digital Video Technologies GmbH; Marché du Film – Festival de Cannes; Visualization Center C
This project has received funding from the European Union’s “Horizon 2020” research and innovation programme under grant agreement No 762079