Identity, Poetry, Memory, Nature

Vibeke Sorensen, Artist and Professor

Division of Animation and Digital Arts

School of Cinema-Television

University of Southern California

September 23, 2000

Identity

Maybe something in my background will help explain my current work with technology. From early in my education, I was uncomfortable with the idea of a mind-body split, as well as the material-spiritual, and art-science splits. They seemed to be artificial dichotomies that exclude the rich continuum of possible relationships between them. I studied art and science, received International Baccalaureate Degrees in English Literature and Mathematics, and played the violin. I left classical music for improvised jazz, because of the energy of the music and because of its rich oral tradition. I was also very attracted to its democratic structure, and the freedom to improvise. Although it seemed unrelated at the time, I was also a visual artist, primarily drawing, and a gymnast (my event was the trampoline). I wanted to put these things together.

I was a misfit, not only because I was multidisciplinary before it was fashionable, but because I was bicultural. Originally from Denmark, I lived in the USA where I was called a “foreigner” and an “alien.” When I returned to Denmark at 16, I was surprised to be called a “foreign Dane” there, in what was my native country. It seemed that I had become an expatriate everywhere, a nomad, and a person without any real home. I felt that I was a child of the earth, like everyone else. I came to regard the earth as our collective home, and all people our family. I traveled a great deal, and my friends, fellow misfits, did too. They were artists, poets, and musicians.

During this time, I studied architecture because I thought it put things together, that it combined everything. On the first day of architecture school, a professor said, “architecture is frozen music.” So, I immediately thought: “liquid architecture.” Not only is the experience of architecture as we live in it one of spatial movement and change, but architecture is an esthetic expression of dynamic compositional characteristics similar to music and film. The space-time elements of architecture, and their potentials, fascinated me. I thought about space on a global scale, and that computers, networks and satellite telecommunications would connect people -and their spaces- all over the world. Each person would interact like a musician in an ensemble, where each space would be a voice in an ensemble of made up of many spaces. A new language of communication would emerge. Because it would be global, it would be international and multicultural. Like music, it had the potential to transcend national boundaries and foster cooperation and tolerance. Therefore, the social, political, and cultural issues central to art, architecture and music were an important concern. In short, over time I developed a view of architecture as much more than static, physical structures. It is:

organized information; including networks and telecommunications
culture, and multi/trans-culturalism - the product of many cultural and historical influences; global
social spaces and structures, human scale, in support of human life, creativity, and communication
spaces and structures in harmony with nature and the environment
spaces and structures articulated by light and sound, smell and touch, multisensory/multimodal
space-time relationships, space in movement and transformation, plastic and fluid, like music, liquid architecture

There are many ways to put these together:

liquid architecture by organized information, or digital computers
multicultural liquid architecture networked using digital computers
multimodal social spaces in harmony with nature and multiculturalism
others

Poetry: Liquid Architecture and Visual Music

I wanted to explore liquid architecture, performing and interacting with spaces in real-time the way a musician performs a musical instrument. I knew that light and sound, like all phenomena, are vibrations. They are part of the electromagnetic spectrum and share common parameters. So I studied the media that provide access to these parameters as a way of connecting space and sound according to a deeper structuring metaphor, the physics of how the universe actually works. I worked with film, video and computer graphics systems, and focused for some time on image and sound synthesis in order to explore their relationships as waveforms, and made many pieces this way (collected works, VideOcean, 1976). But the senses are different and I found that while temporal events can easily share common waveforms, shapes, colors, and textures associated with sound and images required very different waveforms when perceptually similar. So although the vibrations do exist as a common foundation, they are in fact quite different. Each has its own specificity, and how we associate them is not clear. Somehow we miraculously, magically connect them and turn them into a coherent experience, or mental model of the world that to some degree conforms to it. (By “world” I mean the interior world of the mind, the world of the body, and the world outside the body.) In neuroscience, the linking of different sense modalities in the brain is called “the binding problem.” No-one really knows how it works.

My view is that sense information is transformed into impressions that are associated, and that these impressions and their associations are stored in memory, and through a dynamic process of mapping, layering, filtering, and re-association of new sense information from the world, the mind/brain turns them into symbols and language for communication. Further, the meaning we give to sense impressions over time, is narrative.

Frames of Mind: Memory Matrix, 1998, Vibeke Sorensen. Explorations of memory and narrative.

30" x 40" Ilfochrome transparency on light box.

Although I have spent many years making what I previously called “non-narrative” or “abstract” work, I now believe that there is no such thing as “non-narrative.” Abstract work is non-objective, representing a reality that exists in the mind or beyond the surface qualities of familiar objects. Feelings are real, they just don’t always look like a normal photograph of cars, trees, or airplanes. The human brain negotiates and processes new sense information coming in to it, the same way. A complex process of association, linking and connecting sense information is always active. It layers it, and updates the relationships, which are constantly being changed. This means that meaning and narrative is always changing as the context changes. This is poetry. Association is poetry. As Alan Watts has stated, “The power of poetry comes from its associative rather than logical qualities.”

Image-sound relationships, or “visual-music” is also a form of poetry, as associative thinking is poetic thinking. Performing music and moving images/spaces at the same time, improvised like jazz -liquid architecture- is poetry. Further, using computers to realize it means using the logic of computers against the logic of computers. It uses their hierarchical logic to create non-hierarchical structures for lateral and associative, poetic thinking.

Maya, 1993, Vibeke Sorensen. Still images from stereoscopic, non-objective computer animation.

Global Visual Music Jam Project

In 1996, a 3 year grant from the Intel Research Council was awarded to the Global Visual Music Jam Session Project to try to realize this vision of liquid architecture and visual music, for real-time performance with networked improvising musicians. This project was a collaboration between myself, composer-programmer Rand Steiger, and Miller Puckette, author of Max, a programming language used widely for multimedia applications and computer music. The three of us have a fluid collaboration, each offering our individual expertise (me-visual direction; Rand-musical direction, and Miller software), while also having a broad and deep mutual respect that allows us to enter into each other’s domains freely, by open discussion, suggestion, and even direct action.

Miller was working on a new language similar to MAX called Pure Data that allows non-programmers, including musicians and visual artists, to program according to the patch programmable paradigm used in audio and video synthesizers, sort of like real-time flow charts. His software allows you to take data from any source (physical or virtual), process and transform it in any way, and then output it again to any source (virtual or physical). Used primarily for computer music, we wanted to extend Pd to images and spaces. We worked with Mark Danks, a student of Miller’s at UCSD now at Stormfront Studios in the San Francisco Bay Area, who developed GEM (Graphics Environment for Multimedia) as an extension to Pure Data. The resulting software environment integrates 2 and 3D computer graphics and animation, video, audio and image signal processing, audio synthesis, and networking. It is in the Public Domain and is freely and easily available on our website at http://visualmusic.org We worked on Pd and GEM while at the same time realizing Lemma 1 and Lemma 2, networked performance works for multiple live improvising musicians.

I consider the computer an associative instrument capable of poetic relationships, as well as a transformative instrument, qualities associated with myth and the “big questions” in life (such as why we are born, live and die). Transformation is something that we wonder about all through life. We are attracted to it naturally and unconsciously, as we instinctively recognize in it the mystery of our own existence. Transformation is central to Pure Data, and the potential of computing in general, as data deriving from any modality can be transformed into any other modality. By changing modality, we can understand the data in another way, and make new connections between them. My interest in visual-music and multi-modal performance and art is to extend the relationships between the senses beyond 1:1 associations, to new relationships that explore deeper realities, including metaphorical relationships based on the larger questions, multiple memories and experiences of the world, freed by improvisation.

We are a fluid amalgam, a liquid architecture, of cultural influences and memories, personal and social, private and public, physical and non-physical. Our memories are mediated by our bodies and our technology. We are a product of memory and the sharing of it, and it is liquid. Our identities are subjective as our memories are in constant flux.

This fluid quality of memory is very similar to dreaming. It applies not just to the mind, but also our technology and our use of it. Networked multimedia is not only shared or collective memory, but it is also a kind of collective dream, a shared lucid dream. In Lemma 2, I wanted to explore this idea of multimedia directly. I asked the musicians to write down their dreams for inclusion in the piece as text elements that they would then interact with in the performance (in the text section). And although much of the imagery for Lemma 2 was already produced, I asked the musicians to describe the images in their dreams, so it could influence my work. Interestingly, Vanessa Thomlinson’s dreams included almost the precise imagery I had already made (although she had not yet seen it). Among other things, there was a ladder on a surface that becomes water, and in her dream she is holding onto the ladder as though it were a boat. In my animation, a ladder falls from space down to the earth and lands on a surface that changes between water, air, fire, and earth. In the performance, the musicians interact with the surfaces, holding on to the ladder, which has its own independent behavior, through their music. There are 13 sections in this piece, each one with a different process and structure for associating musical gesture with space and images, ranging from pure abstraction (non-objective) to representational imagery, and highly formal to completely free associations between music and moving images.

The performance of Lemma 2 involved 4 performers in 2 cities. Two of the performers, Steven Schick (drumset) and Anthony Davis (piano), were at Columbia University in New York City, and the other two, Vanessa Thomlinson (percussion), and Scott Walton (piano), were at Intel Headquarters in Hillsboro, Oregon. Each site had two computers, one for graphics and one for sound. Each site was running the same program on both computers and had similar (but not exactly the same) databases so that each site would have a synchronized, but unique performance. The computers were linked together by an ISDN network so each could send and receive data from the other, and use it to trigger transformations of the sound and images in real-time. The musical gestures were captured with microphones (no wires were placed on the musicians, so as to free them from encumbrances), their signals analyzed and converted to data, which was then sent through the network to be reanimated and re-synthesized at the distant site. The distant musicians were heard at each site coming from speakers in the back of the hall, effectively scaling the 2000 miles of separation to about a hundred feet, so that all 4 musicians could be heard, spatially distinct, at the same time. Both performance sites included a large projection screen showing the locally generated real-time animation that responded to both local and distant signals. In effect, each musical instrument was extended into the visual domain, and vice versa, becoming a new visual-musical instrument that transforms itself with each new process, a meta-instrument, which is itself an instrument of transformation when interacting with the shared virtual space. This project allowed us to explore the effect of distance and latency on live improvised performance, as well as the possibilities for new associations and transformations between musical and visual gestures.

Lemma 1 performed at the Milos Jazz Club, Thessaloniki, Greece,

International Computer Music Conference (ICMC), September 27, 1997

By Miller Puckette, Vibeke Sorensen, and Rand Steiger.

George Lewis, trombone and trombone-cam; Steven Schick, drumset and mallet-cam.

Lemma 1 and Lemma 2 involved real, physical musicians in real, physical spaces interacting physically with virtual space. It explores the terrain of physical-digital interaction with sound and space, and integrates the virtual world with the physical. Physical interaction of the musicians is required for the piece to be completed. It is also a collaborative and social process.

I used the same system, Pure Data and GEM, for my installations. Again, I wanted to put the virtual world inside the physical world, and background the technology while foregrounding the human body and physical interaction. I wanted to involve more of the senses than vision and sound, incorporating the entire body and the audience in the work, and connected even more deeply with memory and its transformation.

Memory: Morocco

Morocco Journal, 1995, Vibeke Sorensen. Left: 6 individual screen pages. Right: Conch structure model.

Morocco Journal (1995) explores personal and cultural memory and identity. It is based on actual experiences in Morocco in the 1970s. I kept a journal then, which I called an “open journal,” because people I met could write or draw in it, and represent themselves directly. This piece partly compares the memories as written from today with the records from my journals and other memory fragments, including photography, of the actual experiences at the time. The computer was used to connect and associate them, making links between them in an additive process, by free association during the process of remembering. After some time connecting memory fragments, I looked to see if a pattern was emerging. I was surprised to discover the shape of a conch shell. The initial memories were aligned like a wheel, each story relating in time to the one next to it. But then the first level of stories had several branches to other memories, and each of those had more branches. From the side, the wheel model looked more like a cone, with the first layer the top and the last layer the large circle at the bottom. When the first layer connected to the last and then to the second last, and so on, the 3-D pattern or model of the connections between memory fragments became a three dimensional spiral, or a conch shell.

My memories of Morocco were re-activated by a visit to Southern France in 1998, as I was in the same place then as during the trip to Morocco in 1972. Being there again physically, and smelling and eating Moroccan food, brought forth a flood of memories, including layers of the intervening 25 years of new memories including second hand experiences from movies and paintings, books, and recorded and live music. The smells connected the memory fragments in my mind, separated by distance and time, and the associations between them became a complex, linked narrative. I wanted to explore this dynamic layering of memories, personal and cultural, in my work, and address their constant shifting and changing as the context changes. I was fascinated by the subjective nature of memory based on physical interaction with the senses, most especially the powerful role of smell, but also the role of physical memory. I was aware of having been there before, like a deja vu, because of the same physical patterns being repeated, that is being somewhere again and responding to the environment with similar movements of one's body and limbs, the hands picking up and touching similar objects, feeling their textures again. All of these different ways of remembering, and the narrative resulting from them, inspired me to include them directly in my work. The actual layering and association of memory fragments would be realized using digital multimedia to activate all of the senses, not just vision.

I wanted to put the entire body into a physical-mental space, a physical-digital space that would externalize this process of dynamic memory. In Morocco Memory II, I wanted to use and consider the following:

- smell, as a way to trigger memory in people and use it to activate and engage interaction: spices

- body movement, or body memory, muscular, physical memory: walking in the space while holding and moving things freely, there should be no wires on input devices, they must be organic and appear "low tech"

- touch, organic materials that have their own/other memories encoded in them but have symbolic meaning to us: spices and boxes, spices are living physical memory of mixing cultures of thousands of years, the surfaces should be “rich” texturally and the objects should be “rich” symbolically

- sound: music and sound fragments should fill the space and immerse the body, bathe the entire body in sound, sonic memory

- space: actual space should surround and immerse the body, including all of the senses, without any encumbering technology. It should be interactive with the virtual world, which should be embedded within the physical: a small house

- the screen should be a “space” and not just a surface: human scale and part of the house, not just something placed on top of it. It should be a kind of skin or semi-permeable membrane of a cell, wherein/upon the physical and the virtual space meet and send messages back and forth

- images: on the screen should be for externalizing interior thoughts, feelings, and memories as well as creating space and transforming experience into memory; they transmit movement, shape and color, connecting that which is living in us to the life in the images and the memories connected with them. The visual gestures, moving images, or animation, tell us on an instinctive level that they are/we are alive

- light: articulates space and the body in it. It is a powerful visceral awareness of being in a real space. It is a very strong part of memory: use light as part of the interaction with virtual space and extend the images off of the screen by surrounding the screen with colored light

- content: multicultural/subjective identity, something fluid and changing

The house is made of wood and satin, and the design a morph between Danish folk architecture and Moroccan polyhedral structures. I was especially impressed by the 3-D, stained-glass lamps. The glowing luminescence of the house in Morocco Memory II derives directly from them. The interface is 6 Moroccan wooden boxes with spices in them. All have embedded chips that transmit signals via radio waves to a remote receiver and computer. The transmitters and receiver were made by James Snook, of the Neurosciences Institute in La Jolla. Rand Steiger worked on the music, and both he and Miller Puckette helped with the programming.

Morocco Memory II, 1999, Vibeke Sorensen. Left: exterior view, installed at Interactive Frictions,

Fisher Gallery, University of Southern California, June 4-18, 1999. Right: screen images.

There are 66 lexia, or short stories in Morocco Memory II. While most are my own memories, many others are from my colleague Marsha Kinder, Professor in the Division of Critical Studies, School of Cinema-Television at USC, who was in Morocco at about the same time I was, although we did not meet then. There are over 3000 still images, and 100 audio and movie fragments. They are layered and navigated through opening and closing boxes, recombined each time someone opens and closes a box. When a box is opened, sound fragments play, and the piece is activated, lights change in the space around the interactor, and a selection from the visual and sound database takes place. Each time another box is opened, another layer of material is juxtaposed. When all of the boxes are open, theoretical texts appear for contemplation. When all of the boxes are closed, the piece stops, colored lights fade, and white lights go up over the table in the space, focused on the boxes. The computer keeps track of each box, and knows whether it is open or closed, how many boxes are open (and closed), and in what order they were opened (and closed). This is the information used to navigate and recombine the data. As a result, there is an almost endless number of ways of to traverse it and create new associations. Single and multiple people can engage the piece, and therefore it is both a individual and social experience. When multiple people are engaged, negotiation and cooperation are encouraged in order to move through the piece.

In short, Morocco Memory II is not about forgetting, but rather about freeing memory from stasis. It is about activating, engaging, and exploring it in new ways that engages the entire body, and transforming it into new narratives through dynamic association and interaction. (Please see http://visualmusic.org/text/MMdoc.htm for more information.)

Nature: Sanctuary

Sanctuary (work-in-progress) is an alternative vision of how technology can relate to nature and culture. Similar to Morocco Memory II, it uses real objects within an architectural pace. Live plants are involved, as well as organic objects such as shells, stones, etc, and will also engage the entire body and most of the senses, including vision, hearing, touch and smell interacting with them and the physical-virtual space. The technology used to send signals from the physical to the virtual space, is non-invasive, primarily charge and capacitance based. The space itself is made of wood, and there are 4 skins or walls in an open temple-like design meant to bring outside and inside together for contemplation.

The content is cross-cultural interpretations of safe haven, and considers the transformation of landscape in light of virtual space and new possibilities at the intersection. It addresses the urgent problem of disappearing nature, species, and cultures, as western media and technology reaches cultures and places previously isolated. One of the questions I asked myself while working on this piece is: how do we encourage people to work together to protect and celebrate diversity, ecological and cultural? My answer is to start by placing the virtual in a larger physical context that asks people to engage nature with the all of the senses directly, in order to interact with this piece.

What is the goal of this piece? It is not a Christian dream of paradise or a quest to escape a flawed and deteriorating world into an idealized space. But rather it is to explore the intersection of nature and the virtual world, and possibilities arising from it. It is also meant to question the implicit goals of western technology for omnipotence, omniscience, and immortality. Instead, this work should remind us that we are mortal, limited, and a mixture of many influences, interconnected with other people and nature, and must live in harmony with each other and the universe in order to survive. It is meant to remind us that there are consequences for our actions, even in virtual space. It is meant to reconnect the mind and the body, the body with nature, and the heart with both.

Conclusion

What is the purpose of all of the technology we are developing? What should we be using it for? My view is that we should use it to affirm life, and put us into harmony with the universe. It is a conscious choice.