Virtual Dream Reliving

by

Pinyao Liu

Concept

Contemporary psychologists and neuroscientists often employ the term „dreamwork“ to encompass a spectrum of methodologies that deepen and engage with dreams to gain new understandings and personal insights. Central to many of these approaches is the concept of re-experiencing dreams—immersing oneself in the recollection of dream memories, emotions, and bodily sensations. This project introduces the innovative concept of „dreamwork engineering“ and presents a generative AI powered VR system that facilitates dream re-experiencing within a virtual reality (VR) environment.

Over a two-week period, I conducted an autoethnographic study, recording and documenting my own dreams using a voice recording app each morning. From four memorable nights of dreaming, ten distinct dream images (N = 10) were obtained. At the close of each day, I utilized the VR system to re-live these dream experiences, recording both my behaviors and the on-screen VR interactions. During the VR session, I start to retell my own dream in the present tense, first-person perspective. Whenever it comes to a concrete dream object, I put my palms up together, and a tiny ear grows in my palm. I then whisper the dream object into the tiny ear. After 25 – 30 seconds, the AI generated dream object appears in the air. I can then manipulate the scale and position of the dream object using my hand. After the VR dream reliving session, the Heaton’s Gains from Dream Interpretation (GDI) questionnaire was completed, with each item serving as a prompt for written reflections.

This research is a collaboration with Human-Computer Interaction (HCI) expert Alexandra Kitson, dream scientists Claudia Picard-Deland and Michelle Carr, Founding Lab fellow Ray LC, and data visualization scientist Chen Zhu-Tian.

Our findings suggest the potential of a technology-aided dreamwork framework, where spatiality, movement, interactivity, and abstract anchors could enhance traditional dreamwork methods. We propose a collaborative role for generative AI in facilitating insights by fully re-experiencing dream emotions. Moreover, we posit that the scientific community could benefit from integrating dreaming and dreamwork practices to foster scientific creativity.

The system’s development extends beyond individual exploration, culminating in an audio-visual performance in Deep Space 8K. This performance invites audience participation, encouraging them to share their dream objects, which are then dynamically generated and visualized in real-time projections. The result is a collective dreaming experience that intertwines art and science, offering a unique and immersive exploration of altered states of consciousness.

Process Reflection

Collaboration has been at the heart of this multifaceted project, weaving together diverse fields such as Generative AI systems, dreamwork methodologies, Virtual Reality (VR) experiences, and audio-visual performance. The exploration of generative AI and VR technologies, specifically focusing on text-to-3D diffusion models, underscores the technological foundation of this work.

In fostering interdisciplinary collaboration, I engaged with experts across various domains. Collaborating with dream scientists and neuroscientists Claudia Picard-Deland and Michelle Carr in Montreal provided invaluable insights into the intricacies of dream phenomena. Additionally, partnering with a Human-Computer Interaction (HCI) expert Alexandra Kitson in Vancouver contributed to the research methodology. The collaboration extended to include the data visualization scientist Chen Zhu-Tian from Boston. The guidance and consultation from Ray LC, a Founding Lab fellow, further enriched the research process.

Artistic approaches played a pivotal role in shaping the project’s expressive dimension. Collaborating with the sound artist Keon Ju allowed for the composition of dream-like soundscapes, enhancing the immersive quality of the dream reliving system. Constructive feedback from Founding Lab fellows Edwina Portocarrero, Darsha Hannah Hewitt, and Barbara Lippe added valuable perspectives to the artistic creation process, fostering a dynamic exchange between science and art.

As a conceptual outcome of this collaborative endeavor, I propose the establishment of a global network of experts for universities to tap into. This network would serve as a hub for shared interests, enabling in-depth discussions and fostering interdisciplinary collaborations. Recognizing that no single institution can encompass every expertise, establishing connections with external experts becomes a catalyst for innovative research. For instance, my collaboration with the HCI expert from Vancouver and dream experts from Montreal proved instrumental in advancing scientific research. Similarly, the collaboration with an electronic musician significantly influenced the artistic dimension of the project.

In translating these experiences and learnings into a concrete suggestion for a new university, I advocate for the integration of interdisciplinary collaboration as a foundational principle. Establishing connections with a diverse array of experts, both within and outside the institution, fosters a rich environment for research and creativity. This approach aligns with the dynamic nature of contemporary research, where the convergence of multiple disciplines yields innovative solutions. Embracing this interconnected model ensures that the university becomes a vibrant hub for knowledge exchange, where collaborative efforts transcend disciplinary boundaries, paving the way for novel discoveries.