Fraunhofer MEVIS and Ars Electronica have established an artist-in-residency program in conjunction with the European Digital Art and Science Network. An artist has the opportunity to spend several weeks at Fraunhofer MEVIS in Bremen, Germany working closely together with researchers, and then moves on to Linz for a 2-4-weeks stay at Ars Electronica, where she/he is permitted access to leading-edge technical production infrastructure in an interdisciplinary setting.
The current residency program entitled “STEAM Imaging” was awarded to Taiwanese media & sound artist Yen Tzu Chang in March 2017. In addition to her collaboration with researchers at Fraunhofer MEVIS and technology experts at the Ars Electronica Futurelab, the artist was looking forward to integrating school pupils into her project in the form of a workshop offered as part of her residency.
The workshop brings together the technical procedures of medical imaging and sound art. It’s meant for STEM pupils who, in addition to science, technology, engineering and mathematics, are also interested in art. The workshop was conceived by Sabrina Haase and Bianka Hofmann of Fraunhofer MEVIS and artist-in-residence Yen Tzu Chang. The workshop premiered in March in Bremen and was then held June 8-9, 2017 at the Ars Electronica Center in Linz. On that occasion, we talked with Sabrina Haase, who led the workshop together with Yen Tzu Chang.
Sabrina Haase leads the STEM-Workshop. She explaines the basics of medical image computing. Credit: Martin Hieslmair
Let’s start off by asking what Fraunhofer MEVIS does and what you’re working on there?
Sabrina Haase: Primarily, we develop software to assist physicians in imaging-supported early recognition, diagnosis and therapy of cancer as well as diseases of the circulatory system, brain, chest, liver and lungs. Our aim is to recognize diseases earlier and with greater certainty, to develop treatments customized to individual patients and to make it possible to measure therapeutic success.
I’m a mathematician working in the field of bio-physical modeling and simulation, and I also lead STEM workshops.
Fraunhofer MEVIS’ STEAM Imaging project combines computer-aided medicine with education and art, so a workshop for pupils is a natural fit here. How did you select the school classes that were invited to take part in this workshop experiment?
Sabrina Haase: For the first workshop in Bremen, we approached teachers in the fields of art and mathematics with whom we’d already been associated in prior projects. They then recommended several talented and interested students from their classes, and we subsequently invited them in cooperation with the Fraunhofer Talent School. Thus, in Bremen we didn’t invite a whole class; the pupils were complete strangers or only vaguely acquainted with one another. Here, the Ars Electronica staff sought an entire class, and established contact via art teachers.
The pupils learn to tell the difference between CT and MR imaging. They are also teached to process this images with medical computer software. Credit: Martin Hieslmair
What’s the workshop all about? What can pupils do during the workshop?
Sabrina Haase: In the current workshop, the pupils get to put medical imaging technology to artistic uses. And just in passing, they learn the basic principles of computer science, programming, mathematics and physics.
We began with the subject of medical imaging. Here, we primarily focus on questions such as: What is medical imaging, actually? What are two-, three- or even four-dimensional images? How can you recognize which modality was employed to generate an image—that is, x-ray, computed tomography (CT) or magnetic resonance (MR) imaging?
Then, we took a look at the basic procedures for segmentation. That is, we explained how the various parts of the body can be recognized and classified—that’s a tumor, that’s a blood vessel, that’s the liver—and how the size of the particular object can be determined.
Then we went into image registration, which refers to overlaying two or more images of the same patient scanned at two or more different points in time. Overlaying images is necessary because the patient might have just eaten something or was lying in a different position in the scanner, so his anatomy simply differed slightly from when the previous image was made. So, how do I have to process the information from these two images? In the case of a patient undergoing chemotherapy, of course, we want to examine and compare the same tumor, and not, inadvertently, some other one because the images were overlaid wrong.
Sabrina Hasse of Fraunhofer MEVIS and the media and sound artist Yen Tzu Chang working with pupils at the mixer. Credit: Martin Hielsmair
Yen Tzu designed the workshop segment dealing with sound. It kicked off with an introduction to the subject of sound and sound art. What are the background elements? What is sound art and which artists work in this genre? The pupils viewed several videos offering examples of sounds and then built their own “piezos.” A piezo element serves as a sound amplifier—it transforms mechanical pressure or dynamic deformation (e.g. beating a drum or plucking a string) into electrical tension and amplifies it. The pupils experimented with this and isolated the outputs, and then they could hook up their finished piezo to the mixing console and try it out. They all had a lot of fun!
They we turned our attention to producing sound with a computer—first, using the Pure Data computer program; then, generating sound files manually by clapping their hands or some other way. All of this was recorded using the laptop’s microphone. In the next session, we’ll be doing it more professionally, but this is a great way to get started! This makes it easy for the pupils to load, start and stop their sound files, and then play them back with the software.
Tomorrow, we’ll bring together medical imaging and sound art. So the next session is the artistic part.
Medical imaging and sound art – a great match. Credit: Martin Hieslmair
How did the pupils react to this special educational offering? Did they react intuitively with interest, or did they see it as just another day of math and physics instruction?
Sabrina Haase: The so-called STEM academic disciplines [science, technology, engineering and mathematics] have a long tradition at Fraunhofer MEVIS, so we have a great deal of experience with hands-on projects for pupils in this field. Kids and young people really can live up to very high expectations in this area if you take enough space and time to describe the context, if they’re given the possibility of working independently, and, above all, if curiosity and fun come into play.
None of the pupils complained. Not a bit. So I was really impressed! Especially in light of the fact that, during lunch break, the teacher told me that he hadn’t even asked his class if they wanted to participate; he simply informed them that they will be attending a workshop. So the pupils were totally into it! I think this is because we always offer a good mix of topics—some sound art, exercises on the computer, and then medical imaging.
Were there differences between the workshop groups in Bremen and Linz?
Sabrina Haase: There certainly were! Since we only had two small rooms available to us in Bremen, we had to divide the pupils up into two groups. This had an interesting effect: one group associated medical imaging more strongly with sound art, just like we’re doing here in Linz; and the other group focused more on the image than on sound. The pupils made plaster casts of their own hands and then filled the casts with gelatine. Then they placed these gelatin hands in our Institute’s MR scanner and produced MR images of them. Finally, they processed and artistically recontextualized these images.
This September, Postcity Linz will once again be the venue of the Ars Electronica Festival, where Fraunhofer MEVIS will exhibit the results of these workshops. Such a resounding success speaks in favor of continuing these workshops. Are there any plans in this regard?
Sabrina Haase: There are no concrete plans in this direction at the moment. First off, we’ll evaluate the workshops at our Institute; then, we’ll see where we go from here. But considering the tremendous interest and enthusiasm on the pupils’ part, a continuation would certainly be desirable.
Sabrina Haase is a mathematician and, currently, coordinator of international research & industry projects at the Fraunhofer Institute for Medical Image Computing MEVIS in Bremen, Germany. Her previous research at MEVIS entailed bio-physical procedures for mathematical modeling and simulation of thermal therapies as means of improving the therapies’ success. She is strongly committed to expanding the educational offerings the Institute makes available to up-and-coming young researchers—for example, the STEAM Imaging project.
The STEAM Imaging project was originally developed in 2016 in collaboration with SPACE (London).
To read an interview in which Bianka Hofmann, director of science communication at Fraunhofer MEVIS, talks about the STEAM Imaging project, click here:
Fraunhofer MEVIS on the STEAM Imaging project and collaboration with artist-in-residence Yen Tzu Chang:
Blogpost of the Raw Science Foundation on the first STEAM Imaging Workshop in Bremen:
http://www.rawscience.tv/steam-imaging-art-from-the-mri-scanner/