Proteus employs a realtime generated reaction-diffusion algorithm as a surrogate of a ferrofluid material, as one of the core components of the Proteus series . It employs reinforcement learning algorithm to increase visitors’ engagement while interacting with this digital matter displayed in a form of a large monolithic oculus in an exhibition context.

The EMAP residency will focus on developing further the Proteus project as an analogue-digital ecosystem. The artists‘ duo will explore a hybrid interaction between the digitized ferrofluid material and humans through gaze-tracking and emotion recognition, distributed online and onsite.

###################################

In Dialogue: Artist Talk with Tereza Teklić (Kontejner, Zagreb):

KONTEJNER: Both of you are architects by vocation. How did you start working on mixed media installations that use digital and interactive technologies?

Maria Smigielska and CompMonks: Architecture is transdisciplinary by design. It always conveys different streams of scientific and artistic discourses about our societies. The digital became an increasingly important topic of transformation in architecture since the second half of the 20th century. For most historians, it has a strong root in medieval and late renaissance thinkers. But schools and offices started to pick it up more seriously when design and fabrication softwares became more sophisticated. We both have been trained to work in a digital world and to think of the other doors it could open for architecture. Almost naturally, we went on to pursue that track and research on new technologies for architecture at large in the university context. But the language needed another dimension. It felt like a natural move to seek means of self-expression through media art. Because we could not find ourselves elsewhere to develop a more adequate vocabulary about the digital. It is, for both of us, a more open and empowering way to approach built forms of mixed-media objects. A different perspective from which we seek to address architecture.

KONTEJNER: Proteus, the project that you are currently developing within the EMAP residency, is focused on the complex behaviour of the ferrofluid material. What made you start researching this material in the first place? What have you learned about it by placing it in different frameworks, physical and digital?

Maria Smigielska and CompMonks: Ferrofluid is a synthetic and completely artificial material developed in the 1970’s by NASA as a rocket fuel that can be moved by magnetic fields in a weightless environment. The same magnetic forces that are used to move this material in outer space, became the vector we use to build interaction in grounded space. We see both forces and the material itself as one, exhibiting visually complex behaviours. From dots to meandering stripes and coagulated regions. For us it became a material allegory of the Greek god Proteus and its ability to shapeshift to avoid foretelling truth. The material itself has an unpredictable behaviour that emphasises uncertainty, and as such, became prevalent in the process of volatile pattern generations. Exposure to such an abstract and contrasted black and white visual brings out a constant search for signification through human visual intelligence. Not surprisingly, working with such physical material has been challenging to bring up to exhibition standards. First installations started small and we focused on intimate interactions. Progressively, we shifted towards a complete digitization of the ferrofluid. At first, it allowed us to begin exploring larger scales and richer spatial interactions involving collective experiences. But it also brought the idea of a layered aesthetics between the original analogue material and its digital surrogate.

KONTEJNER: Proteus is currently in its fourth iteration. How did the work and research change from its first iteration developed in 2018? What will be the focus of the newest version and how will it differ in terms of technology and set-up to the previous ones?

Maria Smigielska and CompMonks: The Proteus series has iteratively evolved along 3 main vectors: a spatial interaction moving between 2.5D displays, architectural objects and immersive enveloppes; a layered interplay between analogue and digital versions of the ferrofluid material; the diversification of forms of interaction from intimate to collective and distributed within the digital realm. Secondary vectors allowed us to be more playful in the creative process while exploring many different ways to produce a physical display of this material, either discrete or continuous, in various resolutions. We also explored a diversity of technological instruments to capture vision-based information and research on behavioural and physiological responses. As attention and gaze are tightly coupled, Proteus has been focusing on visual perception, including a face tracking system (Proteus 1.0), capturing neural signals through a brain-computer interface to couple machine and human intelligences (Proteus 2.0) or extracting visual regions of interest (ROI) based on skeleton tracking for similar coupling (Proteus 3.0). While former versions of the project always involved an interaction in a single physical space, we now seek to shift to a web-based interaction using any kind of personal mobile devices. On one side, embedded cameras in each one of them gives us higher resolution data of gaze and attention based on facial features. This allows us to refine our interactive and generative system. On the other side it gives us the opportunity to explore a completely new aspect of interaction for us that is scattered, distributed, and unconstrained to a single physical space. Interestingly, we also aim to produce a physical installation that expresses these aspects through the design of spatially distributed displays and reflectors supported by a space frame. Proteus 4.0 is the next milestone in our series as it expands the vocabulary we have been crafting together since 2018.