Welcome
Birgitta Hosea (SE/UK), Juergen Hagler (AT)
SUN 11.9. 14:00 – 14:05
Birgitta Hosea (SE/UK) and Juergen Hagler welcome you to this year’s Synaesthetic Syntax.
Sensate Sculpting – In Praise of The Hand in Digital Sculpting and 3D Modeling
Dr Andy Buchanan (AU)
SUN 11.9. 14:05 – 14:35
The title of this paper is an adaptation of, and homage to Henri Focillon’s In Praise of Hands (1989), wherein the author explains that the hands of the artist are not the mind’s ‘docile slave’, but rather have a life of their own, a ‘sensate presence’ that foregrounds the hands within an epistemology of mind. The sensitivity and sensory ability of the hands working in conjunction with the physicality of the world makes them central in our tacit knowledge of objects. And yet the most advanced ‘object’ making interfaces developed by humans – computer graphics – have arrived only recently and imperfectly to the notion of the critical role of the hand and its knowledge through the development of gestural interfaces. A range of historical 3D modeling conventions have evolved in concert with the evolution of the technologies within which these practices take place, including modeling by defining surfaces as polygons, with vertex positions recorded in 3D coordinate space or modeling through the assembly of surfaces defined by mathematical curve functions (such as NURBS modeling). As described by Aylish Wood (2015), 3D modeling technologies require practitioners to integrate their behaviors with the operational logic of software. The structure of the software operations therefore structures the operations and behaviors of the artist. To execute a project successfully, we begin to unconsciously ‘meet the software half-way’, a process Heidegger referred to as ‘gestell’, or ‘enframing’ (Heidegger, 1954). Displacement sculpting and volumetric sculpting take a different approach to integrating user inputs into the logic structure of the software. Rather than applying careful selections and movements to model components, digital sculpture platforms allow artists to apply gesture based changes to the virtual object and in so doing, distinguish “digital craft from mere mechanical machine operation” (MacCullough, 1998). Tools for these digital sculpting practices enrich the veridical premise of digital objects in the moment of creation, presenting them as real, three-dimensional objects just beyond the screen. These software solutions allow the creator to express changes to the surface through gestural interfaces such as pen/tablet devices or VR input tracking. These gestural interfaces create the possibility for improvisation in 3D object production which is in contrast with the engineering approach typical to polygon and spline modeling. The current status of gestural interface and digital sculpting practice as the emerging standard for the creation of 3D models and digital objects provokes this critical analysis across the entire production chain; creator, tool relations, and object; issues for creators discussed include the industrial categorization of practitioners, their cognitive and physical orientations to these practices and the cultural contexts for creation. The paper includes the artists lamentations on the limitations of tactility, texture and depth, the problematic use of the digital as a proxy for the real and the perils of visual abstraction in this context. I provide analysis of virtual surfaces, which as well as their defining properties have specific behaviors, allowing us to subject digital materials to ergonomic assessment in the context of sculpture. The status of digital objects is considered from the perspective of ontology (after Hui, 2016) and philosophies of digital creation as epistemology.
References:
Focillon, H. (1989). In praise of Hands. In The life of forms in art. Zone Books. Heidegger, M. (1954). The question concerning technology. Technology and Values: Essential Readings, 99–113. Hui, Y. (2016). On the existence of digital objects. University of Minnesota Press. MacCullough, M. (1998). Abstracting craft: The practiced digital hand (1. paperback ed). MIT Press. Wood, A. (2015). Software, animation and the moving image: What’s in the box? Palgrave Macmillan
Investigations into gestural and touch interaction with anisotropic metaphors for audiovisual virtual instruments
Jānis Garančs (LV)
SUN 11.9. 14:35 – 15:05
Having a background in classical and digital audiovisual arts, since some time I have been interested in the potentials of anisotropic metaphors for ‚audiovisual virtual instruments‘. Anisotropy is a concept describing orientation and direction dependent properties of materials and space, used in physics, chemistry, cosmology and is used as a metaphor to develop derived metaphors for interfaces that can use (but also extend the limits of) Cartesian coordinate system. This presentation introduces main concepts and practical investigations using several input devices ranging from graphic tablets, ‘5D touch’ MIDI keyboard, VR controllers and motion-, brain bio-feedback sensors, integrated in spatial concepts of interface, usable in immersive media performance. I wanted to explore the capacities of combined control (audiovisual and haptic/gestural) as dynamic, multi-dimensional relationship models that can be experienced in spatial representation (stereoscopic image and multi-channel audio). This research aims to expand the range of interface implementations, offering expressivity dimensions from expert level of complex and subtle manipulations to first-time encounter and possible accommodation of ludic approach or even brutality. The work is situated in the cognate fields of Immersive Analytics (Skarbez, Polys, Ogle, North, Bowman, 2019) – integrating recent developments in visualization, auditory displays, computing and machine learning and neuroaesthetics. The work aims to build upon various initiatives of integration of audiovisual expressivity of such as ‘AVUI’ (Correia & Tanaka, 2021) and reflect on the meaning of gesture within concepts of ‘mediation of self’ (eg. Stiegler 2021) or within the social construct of ‘dispositif’ (Foucault 1977).
Searching for Gestures of Freedom and Intensities within Interactive Media Art
Jana Horáková (CZ)
SUN 11.9. 15:05 – 15:35
The paper will present The General Theory of Gestures (Flusser) as a productive theoretical framework for the analysis of new media art, which is defined as various scores and records of ephemeral, interactive events involving human and non-human actors (Latour). I propose to switch the focus of the debate on the new media art from the description of the technical parameters of the media to the performative qualities of the interactive media art, and thus towards to search for „movements“ and „gestures“ in the global man-machine network. Flusser’s categories for exploration of interactivity from the dualistic perspective of control vs. freedom will be combined with Bachelards’ aesthetic of elements, specifically poetics of water, which allow to think about interactivity through the category of „intensity“ (Delleuze) as well. The aim of the paper is to show how the performative concept of interactive media art not only contributes to the refinement of the theoretical discourse that surrounds it, but as an effect it also leads to the search for more sustainable strategies for preserving media art as a part of the cultural heritage of 20th and 21st centuries.
References:
BACHELARD, Gaston. Water and Dreams. An Essay on the Imagination of Matter. Dallas: The Pegasus Foundation, 1983. FLUSSER, Vilém; BEC, Louis. Vampyroteuthis Infernalis. A Treatise, with a Report by the Institut Scientifique de Recherche Paranaturaliste. Minneapolis: University of Minnesota Press, 2012. FLUSSER, Vilém. Towards a General Theory of Gestures. In: Gestures. University of Minnesota Press, 2014. LATOUR, Bruno: We Have Never Been Modern. Cambridge, Mass.: Harvard University Press, 1991.
Before the Sixth Sun: A codex for our children
Liliana Conlisk Gallegos (US)
SUN 11.9. 15:50 – 16:20
To showcase the forces of coloniality and the decolonial aestheSis (Walter Mignolo), or the processes of thinking and doing, of sensing and existing beyond the impositions and absurdity of coloniality, I am preparing a children’s book with audio, tactile, and AR components that will tell the story of invasion, injustices, and issues of American Indigenous land stewardship through the perspective of native and settler plants. The book produced in fabric, crochet, and embroidery using cotton, wool, burlap, acrylic, and silk is titled, Before the Sixth Sun: A Codex for Children. This piece of hyper design mixes folk with technology exposing the artisan’s hand as a counter-story to mass production and capitalism in the uniqueness of presence, making the moment and the event more valuable than the object, and bringing materiality back to its connection to ritual and meaning. This work in progress is scheduled to be part of the Mexicali Biennial exhibition The Land of Milk and Honey opening at the Cheech Marin Center for Chicano Art & Culture of the Riverside Art Museum in late January 2023 and will be included in the U.S. Library of Congress Mexicali Biennial archive and catalog. As is with textile tradition and transfronteriza knowledge production, this creates an opportunity for a non-linear morphing genealogy of presentations of fragmentary versions of this piece, because the way the piece is exhibited will continue to change as it is embroidered and being completed (always in a non-linear way). Furthermore, the textile on burlap represents the bounty of Indigenous and femenine forms of data storage and storytelling, alluding to the presence of “hidden” elements found in the augmented reality portion. Here, direct and unfiltered decolonial roots are not shared “underground” but through an invisible wireless technology around everyone. The digital element makes it possible to universalize the function of Chicano and Indigenous pedagogies of the home (Delgado Bernal 2001) as futuristic, “the communication, practices and learning that occur in the home and community…and serve as a cultural knowledge base that helps Chicanas… negotiate the daily experiences of sexist, racist, and classist microaggressions“ (p. 624) This represents the overlapping versions of history I received while I was a child at home, recreating the dissonance I experienced as a Kumeyaay descendant while receiving the legitimized version at school while dressed as a pilgrim and singing about giving thanks. Artisan textile AR animation recreates the sensing of the embodied POC experience in receiving overlapping oral histories in constant movement and the dissonance from receiving a static legitimized version of history. A written part will be presented under each textile page, as well as an audio component, and a tactile 3-D printed version of the text to enhance the sensory experience and accessibility to the artist’s hand, especially to the deaf and blind. The final version of the codex will come with an accompanying subtitled video walkthrough and reading of the book with sound effects and Aztec music composed with Indigenous Mexican instruments courtesy of Mexican/Cuban ethnomusicologist and preservationist, Juan Carlos Portillo.
From Screen to Space: The Haptic Experience of Exhibited Animation
Farzaneh Yazdandoost (UK)
SUN 11.9. 16:20 – 16:50
Wes Anderson’s animations are populated with rough and bold textures. The textures are even more highlighted through the mediation of the camera and animators. The close-ups, the left to right and top to bottom puppets and camera movements simulating caress, the flat images with deep focus and shallow depth of field and the perceivable traces animators left on the puppets-the fur and different types of fabric and textiles-as a result of touching them constantly between the frames, cause the images to be haptic. However, the settings and puppets have been displayed in the physical spaces. For instance, in 2018, the settings and puppets of Isle of Dogs (2018) were displayed at Store X in London, and Mr Fox’s museum office is exhibited at Roald Dahl’s museum. In these exhibitions, viewers have an unmediated haptic experience, even though they cannot touch them due to being rare objects. Also, camera and animators as the primary sources for simulating cutaneous and kinesthesis sensations are deducted from the work in such exhibitions and cause miniature puppets and settings to remain motionless. This presentation discusses such exhibitions as expanded animations, which provide a different haptic experience than the on-screen animation for the visitors by comparing the two modes. It also explains how haptic perception bridges the screen and the physical space. In doing so, this presentation will refer to the attractive characteristics of miniatures as inherently haptic works, the physical 3-D space of exhibitions, and the visitors’ proprioception in perceiving the puppets and the settings, combined with the memories and experiences of both on-screen animations and the real life. The following resources inform this presentation: Laura Marks’ The skin of the film (2000) and her specific definition of “miniaturism”, Jenifer Barker’s The Tactile Eye (2009) and Gaston Bachelard’s The Poetics of Space (1964).
References:
Barker, J., 2009. The Tactile Eye, Touch and the Cinematic Experience. Berkeley: University of California Press. Marks, L., 2000. The Skin of the Film, Intercultural Cinema, Embodiment, and the Senses. Durham and London: Duke University Press. Bachelard, G., 1964. The Poetics of Space. Translated from French by M. Jolas. Boston: Beacon Press.
Interactive Multimedia Generated by Rubbing/Tactile Interfaces – Biofeedback Effects for Wellness Entertainment
Yoichi Nagashima (JP)
SUN 11.9. 16:50 – 17:20
This research aims the Wellness Entertainment with biofeedback especially tactile interfaces. A Japanese company „RT“ developed the small „PAW sensor“ which is covered with Urethane Foam and detects four individual values of rubbing controls by fingers optically, and reacts natural elastic repulsions. I have developed four generations system with this unique sensor, and I will report and discuss the possibility of human tactile interaction with a point of view of wellness entertainment. As is well known, there are so many sensors/interfaces are used in the interactive systems, however, almost these interfaces miss the soft feel and sensation of gentle manipulation. The pressure sensor or strain gauge detects the „maximum“ value like a physical fitness test. Of course the „CV“ sensors have no physical reaction because they are separated with people. I developed a new musical instrument called „Multi Rubbing Tactile Instrument“. This had ten sensors on „Egg“ shaped acrylic sphere to be held in both hands. All PAW sensor information were continuous value, so the were converted to the parameters for realtime „voice-like“ generating system (like the meow of a cat or similar animal). Corresponding to the abstract nature of the auditory output of the system, I employed an algorithm that generated fractal images in real time, drawing beautiful two-dimensional fractal images in grayscale for each of the two modes, and changing their state in real time in response to 32-channel parameters coming from the sensor. The demonstrations as an interactive installation work were surprisingly well received. I had two chances of demonstration: „Sketching“ conference in Arizona and Science Museum in Singapore. The researchers/engineers and children began their experience by touching this unknown toy with trepidation at first. To my surprise, I discovered that they all „smiled“ without realizing it as they enjoyed the strange world of sounds and animations generated and the sensation of their fingers being gently returned to them. This experience led me to to develop a new 8-channel tactile sensor „PAW-eight“ with my collaborator, a rehabilitation specialist. This system has eight PAW sensor for both hands to wrap up, total 32 channels information reach the system. From the beginning, the objective of this system was set as „MCI prevention“ (activating the brain through the sense of touch at the fingertips in conjunction with hearing and vision). This system is as a „Serious Game“, people must keep all eight fingers pushing PAW sensor with the „half“ level and softly (4 channel signals will be almost equal) to move the 3D pointer into the „black hall“ in the center of the space. The pitch of sound become low when the pointer is inside of the „black whole“, so the aim of the game is to make the space will be silent with gentle control of all eight fingers. Care and rehabilitation professionals have evaluated the system and found it to be effective in leading to „wellness“. I would like to pursue multimedia that effectively stimulates the emotions of touch and wellness.
Visualizing the city in handmade film animation: cycles of bodily sensing and perceiving
Aristofanis Soulikias (CA)
SUN 11.9. 17:20 – 17:50
Today, technological advances in audiovisual media are largely driven by a quest to accurately represent reality, a race toward the finito, a finite end, which should see representation and reality coincide. Commercial and utilitarian Film Animation can serve as a paradigm of this purposing of computer technology, with its wide use of CGI for creating photorealistic images and movement. Parallel to the “perfect” three-dimensional deception these virtual images offer, the real built environment of cities is also emulating computer generated imagery – often created in similar software and presented as animation – parametric design being a case in point, whereby a “perfect” end-product is dictated by computer algorithms. This modern-day obsession with machine precision that renders obsolete any technology it supersedes has no room for the unfinished, the decayed, and the slow, neither in processes of making nor in the spaces people inhabit. With the premise that these undervalued qualities are essential to the lived experience of the body, employing traditional techniques in both Film Animation and Architecture that involve bodily ways may not be an act of regression but a revisiting of an unfinished business: that of reinstating the human body in both the built environment and its visualizations. My artistic practice consists of film-animated works that reveal the human imprint which makes buildings living and meaningful objects that evolve ad infinitum. These works also reveal the human imprint of the animation and its “imperfections”. My presentation will highlight two ongoing research-creation projects that are part of the Explorations in Sensory Design research programme, directed by Dr. David Howes of Concordia University and supported by the Social Sciences and Humanities Research Council of Canada. The first is an exploration of Montreal’s Urban Parks and their transformation during the time of pandemic restrictions; a transfer of personal sensory experiences and observed collective movements into drawn and handcrafted animation. The second is a speculation on the regulated sensorial environment of the Mall space and its future within the organic forces of the surrounding city, a contrast expressed by the varying degrees with which the computer is employed in generating the animated image.
References:
Barker, J. M. (2009). The tactile eye: touch and the cinematic experience. Berkeley, CA: University of California Press. Classen, C. (2017). The museum of the senses: experiencing art and collections. New York, NY: Bloomsbury. Hosea, B. (2019). Made by hand. In W. P. Ruddell Caroline (Ed.), The crafty animator. Switzerland: Palgrave Macmillan. Marks, L. U. (2000). The skin of the film: intercultural cinema, embodiment, and the senses. Durham, NC: Duke University Press. Pallasmaa, J. (2009). The thinking hand: existential and embodied wisdom in architecture. Chichester, U.K.: Wiley. Parks, C. F. a. (2016). Fluid frames. London: CRC Press Taylor & Francis Group.
Closing Note
Birgitta Hosea (SE/UK), Juergen Hagler (AT)
SUN 11.9. 17:50 – 18:00
Birgitta Hosea (SE/UK) and Juergen Hagler (AT) conclude this year’s Synaesthetic Syntax and Expanded Animation in a short closing note.
