Re-imagining XR with (and for) People with Sensorimotor Disabilities
Crip Sensorama is a multisensory, interactive XR experience where audience members move, navigate, and interact in VR/AR environments using a set of mouth gestures as a narrative around disability; art and culture unfolds itself. The chosen set of mouth gestures which are developed to enable people with physical disabilities move and interact in XR using their chin, jaw, and lip movements, thus also activate an encounter with artistic practices of disabled artist and collaborator, Eric Desrosiers and Christian Bayerlein in XR.
Crip Sensorama is an XR artwork that focuses on the following question: How can we re-imagine XR as an “assistive technology” to act as a platform of storytelling for (and with) people with disabilities and, in turn, how can the creation of accessible XR enable us to re-imagine our normative assumptions of disabilities?
Crip Sensorama is an interactive VR/AR experience where the audience navigates, interacts with and immerses themselves in the artistic practices of disabled artist Eric Desrosiers—accessible by adjusting to and using a sequence of mouth gestures mapped and parametrized on Eric’s facial muscles.
These gestures, developed by hacking existing face detection AI algorithms and modifying lip and face trackers, enable Eric to move and interact in XR using his mouth and tongue movements while presenting—how the hacked, re-imagined and built assistive XR technologies can act as platforms of storytelling around disability-art with and by people with disabilities.
Photo: Tom Mesic
Photo: Tom Mesic
Bio
-
Photo: Martin HieslmairPuneet Jain
IN
Puneet Jain is an engineer/artist working at the intersection of Human-Computer Interaction, eXtended Reality, Critical Design, and Disability Studies. Puneet’s artistic-research involves the creation of assistive XR interfaces with (and for) people with disabilities, enabling disabled communities to re-shape their own future imaginaries through assistive XR. Jain is pursuing his PhD at Concordia University, Canada under the guidance of Artist/Researcher Chris Salter.
-
Photo: Max GosselinYesica Duarte
AR
Yesica Duarte is an artist, researcher, developer, and VR Experience Design teacher. She graduated in audiovisual production, specializing in spherical video and VFX. She holds a Master’s degree in Technology and Aesthetics of Electronic Arts and is currently pursuing a Specialization in Design for HCI. She explores the immersive language of XR, using experimental wearable interfaces and through performance. She has presented her work in residencies in Canada and exhibited on Latin American platforms.
Credits
Artist collaboration and consultancy on disability perspectives: Eric Desrosiers and Christian Bayerlein
Mouth Interface and VR/AR Development: Puneet Jain
360-degree Video Filmmaking and Documentation: Yesica Duarte
3D Modeling and Design: Wolfgang Kienreich and Nelson Silva
Technical consultant: Nelson Silva
I want to thank Percepto Collective for the 360-degree video content that was shot during the art-residency at Société des arts technologiques [SAT] in Montreal, Canada sponsored and supported by Conseil des arts de Montréal.
This work was initiated as part of the FOUNDING LAB program at IT:U and Ars Electronica in Linz, Austria (2023-2024). The FOUNDING LAB is realized as a Public Partnership between Ars Electronica GmbH and Co KG and the Institute of Digital Sciences (IT:U), Austria and financed through funds from the Austrian Federal Ministry of Education, Science and Research.
