Art and Science: Challenges and Possibilities

Talk delivered at

Federal University of Rio de Janeiro (UFRJ), Brazil

Departamento de Bioquimica Medica

August 2002

Vibeke Sorensen, Professor and Founding Chair

Division of Animation and Digital Arts

School of Cinema-Television

University of Southern California

Los Angeles, CA 90089-2211  USA

Dr. Leopoldo de Meis, thank you very much for inviting me here today to speak here with you and your colleagues. I am honored, and very much look forward to any questions or comments you might have. I hope that this is the beginning of a longer dialogue.

I am interested in how the mind works together with the body, and makes connections between sense impressions to make a mental model of the world, that we use to navigate that world and help us to survive by putting us into harmony with it and the cosmos.

I see that technology is a dynamic instrument for reflecting on that, not only for modeling connections between the senses, but for actually making connections. By connecting a range of media and the art forms that have emerged from them, all based on the separate senses, we are creating new media, new artforms, and new associations between the senses. These associations are elements of a new language that illuminate how the mind works to make sense out of our world. That world is virtual, both in the mind of human beings and in computers. But it is also outside man in the physical world. The computer is now also hybrid physical-digital, and a continuum exists between them, connecting the mind and body, the mind’s interior/mental/virtual space with the body, the physical environment, most especially nature in ways we are just now beginning to explore. We need to illuminate this process and identify it, and develop it with ethics, humanity, and beauty. This is the intersection of art and science, the territory I am exploring.

I work with new ways of perceiving the world, of seeing space in time, and creating unique spatial experiences made possible by computers. In Morocco Memory II,  I explored connections between multimodal memory fragments, and through it one can perceive that the mind creates new meanings and insights when disparate elements (smell, touch, body movement, vision, and sound) are juxtaposed. The interactor changes the relationships between them and new meanings arise. My works employ embedded systems and wireless technology to make real-time connections between thousands of elements. The structures, or patterns of connections between them, change all the time.

Moreover, when we transform the data of one sense into another sense, it allows us to see ourselves and each other in new ways. Basically, when we change our perception of data, we understand it in new ways.

For many years, I have worked with sound-image correlations and “visual music”. It has allowed me to perceive aspects of each, and structures for combining them, in new ways. I have extended this to text, touch, and other media.

I want to use technology to help us evolve better, that is more ethical, more humanistic, more ecological, more responsible, -- more esthetic structures, patterns, and approaches to interacting with the world, --nature as well as other people--  that help put us into harmony with them, so that we will be better able to live in and with the world, and help it and us to survive. I consider it urgent, given the rate of destruction of our ecosystems and the move towards war and human suffering. From my point of view, technology is only truly useful if it can do that.

My new work Sanctuary is environmental, an interactive architectural space with 4 screens, plants, and organic objects, and like Morocco Memory II, uses wireless technology and capacitance to detect the touching of those plants and objects. It creates a new space that is part subjective(associative) and part objective(logical), and uses combinatorics to create an emergent system of constantly changing juxtapositions. They will be part chance and partly tuned by what is learned in the process of developing the interaction. Sanctuary is about connections and the language that emerges from these associations.

The esthetics, or the language of that which touches our souls and minds, or nourishes us spiritually and intellectually, is something that evolutionary biologists would probably say is something we “like” because it helps us to survive. We ‘like’ some foods because we ‘need’ them. We ‘think’ something is esthetic because we ‘need’ it physically to survive. Given the crisis of disappearance of species and cultures around the world, affected by technology in dramatic ways, there is therefore an urgent need for more esthetics, most especially if it can help them to survive. There is good reason to put esthetics into the development of media and technology of all kinds, including but not limited to the selection and associative processes that form multimedia language. This is the role of artists at all levels of science and technology. This is thus a call for more humanism and art in science and technology, in general.

That is one reason why I return to human beings and nature as central to my own work, and place people and plants in the middle. I seek to honor the body and life as deeply intelligent, with richness and complexity from which to learn, to integrate with, and be inspired by. The senses and the connections between them are vital to explore and develop together with other people and living things. The art is in the sensitivity to nature and other people, and illuminated by technology. This is what allows technology to develop into humanistic and life affirming media.

In some of the images from Sanctuary, I work with juxtaposition of perspectives as a visual strategy for correlating multiple visual experiences into memory and meaning.  Language is the meaning that arises from associations. New relationships are discovered as language develops.

The brain’s intelligence is measured by the number of connections between neurons and it is known that a huge number of connections are made in the brain of an infant as a direct result of the physical stimulation of the senses of the child. It is real-time interaction with the physical world that makes those connections in the brain, and creates the mental model of the world needed for survival. This is the approach I am interested in exploring.

I am also interested in exploring other ways that the mind works with sense impressions and memory in making mental models and surviving. Thus, I have been interested in dream, dream theory, and using hypermedia to model actual dreams, and analyze them. Dream is memory and there are many forms of memory, and many forms of dreams. Cultural and personal construction of narrative naturally emerges. Multimedia is excellent for exploring this. Collective dream is where I think some of this is leading. But I want to direct it also to greater awareness of collective behavior towards nature and cultures, and use it to create a more idealistic world, to help people realize their potential to protect rather than destroy the world.

I would now like to show examples of my work.

Thank you again for inviting me here today.