Two S+T+ARTS AIR Residencies Begin at HLRS

Keyvisual image main
Artist Hiền Hoàng (r) visited HLRS in January to begin developing a new, immersive multimedia artwork.

Media artists will work with HLRS visualization scientists on projects focusing on human-AI interaction, and ecology in cities.

Funded by the European Union, the S+T+ARTS initiative launched in 2016 to promote innovation across Europe at the intersection of science, technology, and the arts. By supporting artistic residency programs that bring together artists with scientists, S+T+ARTS facilitates multidisciplinary collaborations that address key challenges facing the EU.

The S+T+ARTS initiative expanded in 2023 with the launch of a new residency program called S+T+ARTS AIR. S+T+ARTS AIR is coordinated by the Media Solution Center Baden-Württemberg, a nonprofit membership organization cofounded by the High-Performance Computing Center Stuttgart (HLRS) to promote collaboration between the culture and creative industries and experts in high-performance computing and visualization. Eight partner organizations from five European countries, including HLRS, joined together in S+T+ARTS AIR to offer their expertise and resources to support artists in realizing new projects at the boundaries of technology and artistic practice.

Taking the concept of air as a metaphorically rich starting point, the S+T+ARTS AIR open call invited proposals addressing two themes: resilient interspecies urban ecosystems, and human-AI ecosystems. In total, 127 artists and artist groups submitted proposals, with S+T+ARTS AIR awarding 10 applicants with residencies. Two of the winning projects will be conducted at HLRS, while the supercomputing center will also support other S+T+ARTS AIR projects in other ways. In mid-January, artists Hiền Hoàng and the Uncharted Limbo Collective held kickoff meetings with the HLRS Visualization Department, and their collaborations will run until September.

An experiment in human-AI collaboration

Uncharted Limbo Collective is a team of creative coders and visual artists based in Athens and London who share a background in architecture but have expanded to develop creative projects using state-of-the-art digital tools. In their project Monolith, they will explore methods for facilitating communication and collaboration between humans and what they characterize as “digital beings.”

At HLRS, Uncharted Limbo will conduct experiments with dancers and movement artists. An algorithm they are developing will generate an audiovisual environment in HLRS’s CAVE visualization facility to prompt the dancers’ motions. At the same time, they will use sensors to capture data characterizing the dancers’ movements and interior states such as heart rate and neural oscillations. These data will then be analyzed using deep neural networks, and the results will be fed back into the algorithm to train it to recognize and respond to patterns in the dancers’ movements. During the development of this method, Uncharted Limbo also plans to work with S+T+ARTS AIR partner Sony Computer Science Laboratories, who conduct research on augmented creativity. The team expects that by gathering and analyzing many hours of data, the algorithm should learn to anticipate, react to, and encourage human movements and emotions in an increasingly confident and deliberate way.

Photo of a dancer on a dark stage with a projection on a screen.

Uncharted Limbo plans to present a performance involving dance and algorithmically generated visuals. (Concept image courtesy of Uncharted Limbo.)

At the conclusion of the project, the team will stage a performance involving equal cooperation between one or more dancers and the resulting digital being — humans will react to audiovisual impulses while the algorithm responds to their movements, creating a performative dialogue between humans and a machine. In addition to testing new interactive technologies, Uncharted Limbo will use the project as a way to engage the public in thinking about issues related to artificial agency, digital presence, data ownership, and the omnipresence of digital technology. The results will also be available to be exhibited elsewhere in the future.

“S+T+ARTS Air offers us a unique opportunity to develop a project we’ve been discussing for a while — an urge to address the important questions of agency and emergent sentience of digital entities,” commented Eleana Polychronaki on behalf of Uncharted Limbo. “As artists with significant experience in coding and art-directing real-time visuals for dance performances and interactive installations, we found ourselves wondering how we can break the norm of the typical ‘accompanying visuals’ and ‘one-way reactivity’ seen in most New Media performances. We hope that by collaborating with the HLRS team and taking advantage of their expertise in innovative visualization and access to supercomputing power we will be able to contribute to the immense field of research that explores human-machine interaction and the ever-evolving dynamics of that relationship.”

The inner lives of city trees

Based in Hamburg, photographer and multimedia artist Hiền Hoàng is interested in how humans perceive and interact with the world around them. In her project Garden of Entanglement, she will develop an immersive virtual environment that encourages participants to explore and gain a better understanding of how human activities affect trees in cities.

Hoàng has already begun working with scientists at the TU Harburg to begin studying the physiology of trees and their biomechanics, including performing 3D and CT scans of trees to gain a dataset that will be the basis for the project. Working with HLRS and the Association for Culture and Education PiNA in Slovenia, she will develop an immersive multimedia installation that will combine such scanning data with other technologies such as data sonification and visualization in virtual reality.

Matvey Fridman (HLRS, Media Solution Center Baden-Württemberg), Leyla Kern (HLRS), Hiền Hoàng (Artist), and Uwe Wössner (Head, HLRS Visualization Department) held a kickoff meeting in January.

Photo of the HLRS team with the artist.

Once completed, Hoàng intends Garden of Entanglement to offer viewers an emotionally resonant, immersive experience of moving through the inside of a tree as it is buffeted by the urban environment surrounding it. She suggests that this should promote environmental consciousness by providing a more holistic understanding of the relationship between urban development and the natural world.

“In the past, conversations with scientists have helped me to understand the biology of trees, but working as a fellow at HLRS is something special because it offers the chance to collaborate directly with experts in visualization and high-performance computing in realizing a project together,” Hoàng said during her first visit to the center. “I am looking forward to seeing how this develops and how it might help to develop my future artistic practice.”

Christopher Williams