ARBORA

ARBORA


Objects (3D printing, custom-made electronics, custom-made software, sound)
Project by Tanja Vujinović
Programming and custom-made electronics for Arbora object: Dr. Vid Podpečan, Department of Knowledge Technologies, Institute Jozef Stefan
Custom-made electronics for Arbora Protectors objects: Gregor Krpič
3D printing: RogLab
Consulting: Jan Kušej, Tomo Per
English language editing: Derek Snyder
Production: Ultramono and SciArtLab, Department of Knowledge Technologies, 2019
Project is supported by The Department of Culture of Municipality of Ljubljana


Placed within the MetaGarden Sphere2, Arbora is one of the objects that takes care of our health and the health of our environments.
As a wise, old tree rising from the MetaGarden, Arbora is infused with a neural network that understands and responds to human emotions.
Arbora and three protector objects that accompany it are all inspired by the plants of the Carboniferous era. Outer surfaces of objects are covered with bark that resembles scales, much like the Lepidodendron tree that existed approximately 300 million years ago. Fossils of this plant sparked the imagination of our ancestors and might even be responsible for the imaginary conception of dragons.
Emerging from the cloud of mythology, three protector objects are synthetic young trees grown in software. They resemble sprouts and like three ancient Greek gods of medicine, Telosphoros, Hygieia and Asclepius, monitor and reflect the overall environment of MetaGarden.
Arbora senses the emotions expressed in the voice. Our voices can give clues about both the physiological and emotional state we are in. A specially developed and trained deep neural network deciphers the emotional components encoded in the captured voice in order to model a soothing binaural sound. By doing so in synergy with its environment, Arbora, together with its helpers, works towards improving our well-being.

Technical description

Arbora is constructed as a three-dimensional object and printed using 3D printing technology, with the addition of a hardware metal stand. Custom-made electronics placed inside the object are Raspberry-Pi, a module with four microphones, and with headphones placed on the stand and connected to the Raspberry-Pi.

Arbora is devised as a sentient device, with all of the references already mentioned in their general description: human interacting with the tree, the ancient Greek gods of medicine, biomimetic forms of plants and futuristic medical devices. 

Arbora should react to the voices of people in their vicinity who wish to interact with them.For this purpose, we developed a deep neural network model that analyses sound captured from space. Software is written in Python programming language using several libraries like num.py and others. A neural network is constructed based on the Tensor Flow open code library that enables the development of models for machine learning. Tensor Flow previously trained on the Ravdess database was implemented for this purpose. This database is made of voices of actors pronouncing the same sentences with different emotional input. 

The mechanism of this action is that the sound is captured and stored temporarily in the sound buffer. Another part of this custom-made software filters the sound and eliminates everything that it does not detect as human speech. One-second-long pieces of sound are analysed, and the average value of determined emotion is extracted. After this, the sound properties pre-assigned to each detected emotion are generated and played through the headphones. Since we wanted to create the situation within which the object would subtly react to a human voice, binaural beats were chosen as one possible type of sound interaction. 

The brain is a central powerhouse of the human body, and it is no wonder that the studying of it is of pivotal importance. Brain waves are studied in a variety of academic disciplines, like neurology and its subfield epileptology, within psychology, psychoacoustics, and cognitive neuroscience, among many other fields. While we know that different brain waves are associated with diverse states of mind, for example, delta waves are associated with deep sleep, beta with an alert mind, and alpha waves with a mind that is relaxed, there are numerous research attempts that tend to dive deeper into the fine differences among such waves and associated stimulations that might impact brain functions. Such studies are conducted on both humans and animals. One such potential technique is the implementation of binaural beat (BB) stimulation. 

Binaural beats (defined as dual broadcasting of two sinusoids with a small difference in frequency) were implemented for the development of the second part of the software for the Arbora object. Since it is necessary for this particular type of binaural sound to be experienced as two sound streams simultaneously played into both ears, we needed to employ headphones. 

Sound is continuously captured by the device. When the visitor speaks near the object, the voice is processed. If there is no new speech detected within the captured sound, the software emits the average pre-determined frequency. Binaural beat, usually consisting of two sinusoidal sounds that differ in frequency might, for example, emit 95 HZ to one ear and 105 Hz to another, and the final setup would provide the following constants: carrier 50, difference 2, and duration 30. Each predefined emotion (neutral, calm, happy, sad, angry, fearful, disgust, surprise) is coupled with the beat that could restore the emotional state back to calm.



The Arbora software is based on state-of-the-art deep neural networks. It is implemented as a real time audio capture, analysis and synthesis loop which continuously records and analyzes live audio data, classifies the extracted speech using a deep convolutional neural network and generates high quality binaural sound waves designed to stimulate electrical brain activity to thus amplify, soften or transform the detected prevailing emotion. This is possible because of the brain phenomenon known as the frequency following response, which reflects the activity of brainstem neurons and is characterized by a waveform that follows the auditory stimulus wave, which is in our case generated by the Arbora software as a precisely designed binaural beat. The parameters used for generating the binaural beat are chosen according to the currently available knowledge about brain waves and their relation to mood and emotions. In addition, a small amount of randomness employed during the selection of the carrier and beat frequency of binaural sounds from the allowed ranges ensures both non-repeatability and continuity.

Arbora protectors are devised as objects that subtly monitor the environment and echo the changes within it through the sound they generate. The custom-made device where all processing happens consists of several connected modules with analog circuits. Input signals come from three sensors that monitor light and temperature within the space. These signals are being routed into a voltage controlled oscillator (VCO), followed by the voltage controlled filter (VCF). One part of the signal travels directly into the mixer, and one pipeline goes into a delay/echo module. There is also a low frequency oscillator (LFO) that joins the others within the sound composition. Signals can be changed from sinusoidal, triangular or square within the VCO and LFO modules. Output sound is subtle, presenting an organic texture that occasionally changes, and, although minimal, it never feels repetitive.

**********************************************************************

MetaGarden Sphere2

A garden "is never a garden of merely private concerns into which one escapes from the real; it is that plot of soil on the earth, within the self, or amid the social collective, where the cultural, ethical, and civic virtues that save reality from its own worst impulses are cultivated. Those virtues are always ours."

Robert Pogue Harrison

What will our future gardens, the gardens of the third millennium, look like?
Will they be made of objects, machines, and living beings that synergistically maintain their flexible systems and communicate with their surroundings?
MetaGarden is an ongoing project that reflects upon a complex relationship of humanity and its technologically fortified environment of nature-culture, and focuses on a particular issue within each installation.
Through MetaGarden Sphere2, I attempt to examine not only what exists within our lives, but also what multiple possibilities and changes might emerge in biopolitical, social, and environmental domains. 
Throughout history, the garden as a sheltered environment has been re-emerging as a special location for human contact with nature, recreation, and rethinking of mythologies, social relations, and allegories. 
Gardens have never held unitary functions or forms. Filled with idealised flora and fauna or devised as minimalistic environments, gardens would sometimes induce ecstatic feelings or provoke meditative immersions and reflections. The classical Greek Epicurean school promoted understanding of the world through the tending of gardens and, instead of overcoming, it was all about transfiguring nature and self-cultivation. Epicurus viewed gardens as places in which reality could be reconceived and reimagined. 
Michel Foucault thought of gardens as the perfect heterotopias – the other places, detached from ordinary life. Within gardens, we immerse ourselves in relationships with living and non-living objects or non-human agents, and seek in them the forms of transitional, comfort objects. Gardens infuse us with molecules and affect our senses, but we also infuse gardens with our states of mind and impose forms onto nature. Gardens echo our lost contact with nature brought forth by the rapid development of industry and technology. They are associated with regeneration of human beings, our reconnection with nature, and the notion of care and cultivation of both ourselves and our nature-culture environments. 
Since the very beginnings of civilisation on Earth, humans have turned to plants for food, shelter, and medication. Recreation in nature has always been advised in the form of walks(1), meditation, observation of plants, breathing of the healing air in the woods, and tuning in to the countless signals and chemical communication channels of the surroundings. Gardens might be seen as networks of engineered man-made and natural elements that promote the flow among non-human and human agents.
Jean Luc Nancy's concept of synaestetic touch that underlines the necessity to pay special attention to senses other than vision, like touching and smelling, might pave the way for cultivating a novel attitude towards nature in the post-digital world. 
Gardens might also be microcosms that temporarily separate a person from the rest of the anthropocentric world and enfold one into their special texture. As Michel Foucault would say, “the garden is the smallest parcel of the world and then it is the totality of the world.” (2)
What might our future habitats look like? Are we going to seal ourselves off from the atmosphere due to pollution and live in chambers that look like an Apple parking building or Amazon Spheres? If so, who will be able to afford the type of hi-tech water, air purification, and maintenance of plant growth inside the future farming facilities? Such future chambers may enable us to experience the world of “wilderness” to its fullest in a tamed form, devoid of any danger, disorientation, darkness, and of anything uncontrolled. Aquaponic gardens for industrial production of plants operated entirely by robotic agents offer a glimpse into a potential future scenario. 
Utopian ideas have occasionally sprung up of an idyllic garden spreading around the whole Earth, like the one envisioned by futurist Jacques Fresco with his Venus project. We might be very far from such a scenario, but we could at least work towards curbing environmental pollution and providing everybody with access to clean natural environments. A potential way towards the MetaGardens of the future is co-creation with nature and the engineering of upcoming civilisation informed by bionics and biomimicry. Biomimicry, the term coined by Janine Benyus in the 1990s, is the outlook that strives not to extract from nature and domesticate it, but to create solutions learned from the ideas that appear everywhere in the natural world. As Benyus writes, some of the core principles of nature are that it runs on sunlight, uses only the energy it needs, fits form to function, recycles everything, and rewards cooperation. These principles, i.e. functions of nature, should be embedded in the materials of future design – from apparatuses to buildings and infrastructure.

MetaGarden Sphere2 is an ecosystem inhabited by futuristic machines as ur-forms that take care of humans and our environments. These objects, synthetic being-devices, are inspired by paleobotany and artificial intelligence technologies, plasma physics and nano-structured materials. Through the virtual world and physical installations, I explore the twists between synthetic and natural. The works tie past with future and merge concrete elements from science and history with mythology. As the title suggests, I also draw inspiration from numerous metaphors rooted in gardens and their essential components. The works that comprise Sphere2 are Carboflora, Fontana, Genera, and Arbora. Carboflora is an endless virtual simulation, within which virtual flora grows according to measured readings of air pollutant levels. Fontana cleans water by employing plasma, while a little fountain disperses the plasma treated water into the air. Genera cleans the air by channeling it into an assortment of nanotubes. Arbora analyses emotional states through voice and responds by generating binaural sound.

1) “We should take wandering outdoor walks, so that the mind might be nourished and refreshed by the open air and deep breathing.” Seneca, XVII.
2) Michel Foucault, “Of Other Spaces.” Diacritics, Johns Hopkins University Press, 1986.

References:

Foucault, Michel. “Of Other Spaces.” Diacritics, Johns Hopkins University Press, 1986.

Nancy, Jean-Luc. The Sense of the World. University of Minnesota Press, 1997.

Lowenhaupt Tsing, Anna. The Mushroom at the End of the World: On the Possibility of Life in Capitalist Ruins. Princeton University Press, 2015.

Pogue Harrison, Robert. Gardens, An Essay on the Human Condition. The University of Chicago Press, 2008.

Ponting, Klajv. Ekološka Istorija Sveta Životna Sredina i Propast Velikih Civilizacija. Odiseja, 2009.

Leslie, Esther. Synthetic Worlds Nature, Art and the Chemical Industry. Reaktion Books, 2005.

Haraway, Donna J. Staying with the Trouble Making Kin in the Chthulucene. Duke University Press, 2016.

Benyus, Janine M. Biomimicry Innovation Inspired by Nature. HarperCollins, 1997.

L. Annaeus Seneca, Minor Dialogs Together with the Dialog "On Clemency"; Of Peace of Mind; Translated by Aubrey Stewart, pp. 250-287. Bohn's Classical Library Edition; London, George Bell and Sons, 1900.

**********************************************************************

OPEN TEXT IN ENGLISH (PDF):: 
DESCRIPTION OF THE WHOLE PROJECT METAGARDEN SPHERE2 :: 

**********************************************************************

ODPRI BESEDILO V SLOVENŠČINI (PDF) :: 
OPIS CELOTNEGA PROJEKTA METAVRT SFERA2 :: 

**********************************************************************