Vibeke Sorensen
School of Film and Video
California Institute of the Arts
The Contribution of the Artist to Scientific Visualization

1987 delivered as a talk at the Jet Propulsion Laboratory, Pasadena, California

1989 published in condensed form by the San Diego Supercomputer Center, California

Introduction


It has been stated that approximately 50 - 75 % of our brains are comprised of neurons devoted to processing visual stimuli. 99% of the American population watches television, with a large proportion relying on it as their primary source of news. The saying "seeing is believing" attests to the trust we place in our visual sensory apparatus. We speak of "eye-witnesses" in legal proceedings, and in the accuracy of visual memory. This concept of visual, or "photographic memory" is powerful and useful. Having a very high bandwidth (as opposed to our ears), the eyes are sending vast amounts of data to the brain every second for processing. The method by which we process this information is still largely a mystery, but it is clear that "visual thinking" is based upon an inter-relationship between visual perception and memory. E.W. Sinnot, the American biologist and philosopher, has stated that, "Stored images in the mind are the basis for new creative ideas."

With this in mind, we can begin to look at the history of visual art as a recorded history of ideas and knowledge. We know that cave paintings showed us not only elegant and aesthetic (even by today's standards) representations of ancient concerns, but also that primitive man, to the best of his abilities, possessed a functional understanding of the nature of the materials used to make the paintings. As Paul Klee has stated, "The artist knows the nature of all his materials, for part of his work has consisted in examining them precisely, one after the other, in order to master them with intelligence." So, from the cave painter up through current time, artists have to some degree also been chemists or technologists. In fact, until last few hundred years, art and science were closely allied. The relationship between creative thinking and scientific advancement has always been close. Specifically, the role of intuition in the creation of a theory or hypothesis, the application of a knowledge of geometry, in addition to the use of a rigorous method for recording observed experience, has formed the basis for many scientific advances over the years. Albert Einstein, in "Essays in Science," stated, "Man tries to make for himself in the fashion that suits him best, a simplified and intelligible picture of the world; he then tries to some extent to substitute this cosmos of his for the world of experience, and thus to overcome it. This is what the painter, the poet, the speculative philosopher and the natural scientist do, each in his own fashion." ("Principles of Research" from Essays in Science, 1934)

In light of Einstein's insight, and in the context of Scientific Visualization, I have asked myself three questions:
  1. What is an artist?
  2. How are artists like or not like scientists?
  3. How can artists and scientists work together?

1. What is an artist?

An artist is many things, but generally it could be said that artists are individuals who are working towards a deeper understanding of the world in and around them. Artists are acutely sensitive to the environment, organizers of large amounts of data into an aesthetic expression of his or her world. Artists are explainers and representers, pattern matchers, people who find unusual relationships between events and images. Artists are creative intedisciplinarians. They rework and integrate information, subject data to functions and tests, render or give the data shape, express essence and make clear. They help us to see in new ways and understand that which is difficult to see, making the invisible visible.

Artists are idea people, creators and conceptualizers, predictors, people with ideas connected to the future. They are people who create something completely original and new, something beyond the known boundaries of the information base. By using or inventing new tools, they show new uses and applications that synergize and synthesize fields. Artists push the limits of technologies, bringing them to previously unattained goals. They see the next steps, make leaps, put together elements in new and powerful ways.

Artists are bridges to the rest of the world. They understand perception and know how to communicate information. They know the technologies and language of media. Their work is linked to the general population as their inventions and ideas translate directly, often seen as improvements in understanding and the quality of life, improvements upon less efficient and more laborious methods of working, becoming mainstream as society discovers their work and catches up.

2. How are artists like or not like scientists?

Mutual Respect and Common Background

A good number of scientists have backgrounds in visual art, having formally studied both art history and creative art production. Many have a deep respect for art, and rather than studying it formally, attend museums and galleries and maintain contacts with colleagues in the art world. Scientists know from these associations that the work of an artist is often as difficult and complicated as work in any field. They appreciate and respect the vision and the hard work. Conversely, many artists, especially in the more technical fields of photography, film, video and now computer art, have good science backgrounds, and have a reverence for the achievements of science. There is a parallel respect and a fair amount of common knowledge and concern. This results from living in the contemporary world with widespread exposure to current ideas and technologies through communication media and personal computers, and from a good education that includes training in both the humanities and the sciences.

Aesthetics

Artists as well as scientists work with abstract symbols, representations for various realities and working tools. Even the language used by the two groups is similar. Scientists working with mathematics frequently describe a particularly good explanation or solution as "elegant." Usually, "elegant" means simple and original. This has also been said of good art and design. Scientists apply their taste in visual aesthetics to their visual displays of data as do artists, as in the case of a scientist who carefully selects colors or arranges forms used in his/her design of a chip or rendered image. These are aesthetic decisions, signs that the scientist is to some degree thinking as does a visual artist. The intellectual bridge of abstraction and aesthetic consideration is fundamental to both groups.

Art and the Scientific Method

Just as scientists consider aesthetics, artists often use the scientific method in their work, consciously and unconsciously. For example, a ceramicist developing a new glaze may begin with a hypothesis he/she wishes to test, and with the results of the experiments, will continue to test and modify the original hypothesis until the results confirm the hypothesis. This process may actually lead the artist to the exploration of fundamental technological problems, even though the original motivation for the work was otherwise. Eventually, the tests yield data which we see in the form of exotic creations, some successful and some unsuccessful. The craft or technique developed may be used to bring forth statements by the artist, or as in experimental or process art, the tests and/or their results may be viewed as aesthetic statements in themselves. Whatever the purpose, it is through the artist's methodology of experimentation and creation that new visions, new techniques, and new ideas enter the collective knowledge base of the culture.

Once an experiment has proven repeatable, the results of the experiments end up influencing the culture in that they become things to be imitated. Techniques and ideas are appropriated by the mainstream and become known and constant ways of creating images and representing information. This was, and still is, the case in computer graphics. Art Durinski, computer artist/animator formerly with Information International Inc. (III), Omnibus Computer Graphics, and Toyo-Links of Japan, told me that when he was at III in the late seventies, he "made lots of flying logos over grids because they were easy to make." That look first set forth by Durinski and his colleagues has been imitated not so much because it was easy, but because it was synonymous with computer graphics. The same is true of today's metallic and ray traced objects. I have recently seen a demo reel of a production company in Southern California, wherein the metallic spinning logo was actually made by hand using an airbrush. This computer look is the current icon of sophistication desired by our visually demanding, high-tech society.

Visual Thinking

Visual artists usually think visually, working with drawings and creating spacial models in their heads which lead directly to finished works. Art schools investigate the models from which artists work in depth, studying their goals, work processes, and end results in formal critiques. Like the 3-D artist, the sculptor, many of the best scientists with whom I have spoken tell me that they frequently work from visual geometric models in their minds, and identify research problems on the basis of these visual models or paradigms. From the paradigm, the analysis and experiment is derived. This ability to pre-visualize a potential solution to a problem and to build 3 or 4-D (3-D plus time) conceptual models is basic to their scientific process. But, unlike the art schools, science schools do not generally teach or analyze this spatial-conceptual skill. Dr. Carver Mead, of the California Institute of Technology Computational Neurosystems and Computer Science Departments, often works with visual models and informed me that the problem in the sciences is that those who do visualize take it for granted, and those who do not know about it, do not know about it. Dr. Mead has suggested the importance of teaching conceptual-visual skills in secondary and university science education, including drawing, filmmaking, and related perception and communication courses. He feels that it is important to articulate and formalize the role that conceptual visualization (creation of visual paradigms) as well as descriptive visualization (display of data) skills play in science education and research. He has stated that the lack of training in communication skills has led to a slowing of knowledge transfer between various disciplines, an important element in scientific research and in the development of new branches of science. This is especially relevant to the emerging field of Scientific Visualization, wherein visual thinking applies directly to many levels of the process. Being a good communicator, he says, is an extremely valuable asset for a scientist.

Clarity and the Imagination

How do artists and scientists create models and paradigms? How do they come up with fresh and new ideas? What is creative thinking? Both groups are working with their imaginations and their intuition, the powerful organizing force of the brain which is always active, reaching beyond "normal" thinking processes. Creative people often use "educated guessing" or inherent trust in their "intuition," the natural organizing force which seems indirect, but often is more direct than realized. Intuition builds upon stored knowledge and to me means "clarity of perception." It seems that the brain naturally forges neural pathways linking information in new ways, reorganizing the data, and then reaching beyond it, creating something new and logical, perhaps just not so obvious.This form of logic may not necessarily be verbal or mathematical, it may instead utilize visual thinking techniques, such as geometric pattern matching. The path to the idea may be based on the acceptance of string of unproven assumptions, so that part of the work of the scientist or artist involves exploration of the assumptions. In any case, the combined skills of logical (visual, mathematical, or verbal) and free(the ability to work with unproven assumptions) thinking can lead to interesting questions and potential breakthroughs.

This idea formation or "insight of clarity" sometimes occurs in one's sleep, and at other times when playing or daydreaming. This process is useful for artists in identifying a concept or image, and helps researchers of all kinds in identifying problems and their solutions. The subconscious mind keeps working as our conscious mind goes onto something else, then suddenly, the Eureka! experience. Both artists and scientists use their imaginations and logic, both consciously and unconsciously, within a given framework on a particular set of events or problems. Creative thinking is inherent to all fields of endeavor, since it is a human characteristic. The brain appears to synthesize the accumulated information and leap to new ideas, new conceptions, new models for understanding the world.

Art is to Design as Science is to Engineering

In visual art, the database of information with which a creative mind works consists to a large degree of visual images, drawn from a myriad of sources. Sorting and making sense of this data by the artist is a highly specialized and refined task, potentially occurring partly in the retina or optic nerve, and partly in the brain. (In a way, an artist is like a very good computer system!)

While both artists and designers have large stored databases of images, it is important to consider the difference between the two. Designers usually work as rearrangers of existing data, applying it to an existing model. The fine artist, however, works with and beyond repetition of existing patterns, if necessary synthesizing wholly new modes of representing the data. The fine artist works from his/her own thinking process, and is capable of creating original ideas and models, forging new relationships beyond the known, stored patterns. A good designer can also work with the spirit of risk-taking, exploring unusual options. Normally, however, this is not the case, as most designers are strongly discouraged from exploring unconventional approaches to design while students. This attitude typically goes with them into the professional world. Ironically, the winners of design competitions are those who break out of the rigid visual molds, those working more in the realm of fine artist. Hence, it is truly the spirit of the fine artist that makes the greatest contributions to our visual environment. An individual considering new information, new problems with new data, synthesizing it and reaching towards the new and the fresh, towards clarity and refinement, is either a fine artist or designer working as a fine artist. This person brings a fresh perspective to the field and may be able to create new conceptual or visual models which better fit the new problems.

Art is to design as science is to engineering. An artist, like a scientist, is involved with the imagination in the conceptualization stage of a problem, whereas a designer and an engineer is usually executing the application of art and science, usually in repetition of what has been done before. This is not to belittle the work of the engineer and designer, who is grounded in the pragmatic application of known principles. It is just that the roles are very different. If one is discussing the research environment, the focus is usually upon the future and on gaining a deeper understanding of the world, and hence, a fine artist is more sympathetic to the purpose of the scientist than is a designer.

3. How can artists and scientists work together?

Unless the scientist is also an artist, it seems that the very best situation in a scientific environment is for scientists to work with a fine artist in a team. This artist should be a person with enough science background to understand the basis of the research, and potentially a person whose own work is related to the research field. This person can help to identify significant problem areas and keep a stimulating dialog with the scientists. This person should also be sensitive to the purpose of the research, as well as applications to society, a thinker with a conscience. This is very important, for such a person could give very valuable feedback about long term implications, which could very well impact the next level of research undertaken. This artist should also be able to work as a designer, applying their sense of the pragmatic to original, fresh representations, utilizing their well developed visual perception and communication skills in the final display of the data. As in any field, care should be taken in interviewing a new member of a research team, to make sure that the personalities are a good match.

I have broken down the research process into 5 parts, and have indicated with an * where I feel artists "plug in" well:
  1. * Identification of research problems, conceptualization of visual models, especially if research is related to vision and visual perception, image processing, image making and control of space. An artist with some mathematics and science background, whose own work relates to the research, may have good insights and may be very helpful and stimulating.

  3. Description of the experiment

  5. * Running of the experiment, gathering data, testing, and modeling for study (very important). In the case of tool development and evaluation, artists are great testers, they push a medium to its limits, working very hard and long hours. It has been said that "If there is a bug, they will find it." Artists spend a great deal of time making and refining the data, where scientists spend more time making the tools.

  7. Studying of results and drawing conclusions; alteration of experiment, then go on to next problem (go to 1.)

  9. * Dissemination, giving form to the display of the results, for communication purposes. Artists plug in here very well. This is because of their fluency with visual communication skills and media. For example, an animator knows how to pace changes in time, what movements and positions in space "read" well. A painter knows how brightness and color saturation affect apparent spacial location, and how the eye and brain responds to juxtaposition of form and color. An architect or sculptor knows how spatial constructs will look from different vantage points, even before the representation is made, speeding up the visual selection process as well as improving understanding of a final design, and providing more aesthetic results.
Artists "plug" directly into 3 of the items listed above, including (1) conceptualization, (3) data gathering, modeling and (5) dissemination, or designing the display of the information for communication. While typically, artists are thought to be useful in the last stage only, and though I feel that this application of the artist to Visualization is important, it is a limited use of the artist, missing out on the full benefit of interchange between disciplines. There exist excellent models from industry and education to support the integration of artists into stages 1. and 3., conceptualization and testing, as well as dissemination. I would even argue that in Scientific Visualization, the most important may be the first stage.

Sample Models from Industry and Education

There exist many excellent examples of research environments which make use of artists, exploring their ideas and integrating them into the process. The examples I am listing below in this and in the section called Historical Models in Related Media, are but a fraction of those in existence, but will serve as models to make my point. Often, it seems as though the use of artists is a kind of trade secret! Apple Computer is known to make use of artists, especially conceptual artists, in their user interface design and research groups. The Apple MacIntosh computer is very popular, and it is likely that the artists had a great deal to do with the success of the Mac. The use of black type on a white field, which we take for granted, is actually unusual for a graphics system. Most computer displays are white on black, which is more difficult to look at. Using the printed page is an excellent model, probably the suggestion of an artist.

Pacific Data Images (PDI), a major computer animation company which is known for its continuingly fresh ideas and innovative look, has hired artists and animators for various production tasks, but expects that they spend approximately 25% of their time on-the-job on their own work. It appears that PDI understands that the work the artists do on their own, feeds back to the work they do for the company.

Examples from Caltech

Computer scientist, Brian Von Herzen of California Institute of Technology's Computer Science Graphics Group, has stated that access to art and artists has helped him in his own computer graphic work. He has become more aware of aesthetic considerations such as color, from looking at art and from studying his own perceptions. He incorporates his, by now, very developed sensitivity to color into his work. In addition, he tells me that while he can compute a scene which, according to the algorithm, should model nature correctly, he calibrates his creation with an artist's eyes. The artist can tell very quickly if the scene in fact is "correct" and looks like reality, since the artist has spent so much time studying the visual world.

Dr. Dan Whelan, who received his Ph.D. in Computer Science from Caltech in 1985, wrote a rendering program which I used for computing a sequence of images during the same year. There was a serious problem which I discussed with him: there were no shadows in the scene, and it just didn't look right. If the program couldn't compute the shadows, then I would add them with the paint program, which I did. Over the following months, Dr. Whelan developed a shadow algorithm, which ended up being a significant portion of his Ph.D. thesis. Perhaps he would have written it anyway, but my feedback surely helped to accelerate this development.

Misha Mahowald, graduate student in Computational Neurosystems at Caltech, has designed a synthetic retina which makes use of the hexagon as a design element. She came up with the idea for that structure from looking at the work of M.C. Escher. Ms. Mahowald has stated that "artists are windows to understanding perception." She and I keep an active dialog about the relationship between art and science, especially interactive computer graphics and music as it relates to research in understanding and computer modeling of vision and hearing.

Professor Alan Barr, also of the Caltech Computer Science Graphics Group, works with the simulation of natural phenomena. He once worked as a graphic designer, and this work, he says, has helped him a great deal in his current work. His concern for aesthetics is evidenced by his careful and elegant selections of color, composition, and form throughout his visual work. His needs for more control over the design and structure of time in animated sequences, has led him to some of his current research objectives. His work in dynamic constraints developed largely from this need for aesthetic control. In addition, Dr. Barr sees the benefits of studying the working process of such artists as character animators, in order to understand how artists control form for human expression. Dr. Barr has been a major force in teaching his students concepts of visual aesthetics as well serving in his formal role as a Professor of Computer Science.

The University of Illinois at Urbana-Champaign

The University of Illinois at Urbana-Champaign Center for Supercomputing, is currently setting an excellent example through joint efforts between artists and scientists. Donna Cox, an artist working at the Center, has spoken about this fruitful interaction, what she calls Renaissance Teams. These teams make use of specialists in art and science, where neither person has to "backtrack" and learn the other field in order to work. This is an excellent prototype for the future of Scientific Visualization, an approach which serves both the mature scientist and artist. The artist focuses on difficult visual decisions, while the scientist focuses on the difficult scientific problems. It helps, of course, that Ms. Cox has a lengthy experience with science, so that her aesthetic decisions are made with an intelligent approach to the research. The interaction of specialists also serves as an education of one about the other, thus enriching the two on several levels.

My Work with Dr. Weinberg at USC

Among my efforts to stimulate communication and collaboration between artists and scientists, I have worked with Dr. Richard Weinberg of the University of Southern California (USC) on the creation of an animated sequence depicting two models of the Solar System, the Platonic and Ptolemaic, for the Omnimax film, "The Seasons," directed by Ben Shedd. I designed the databases and animation, and Dr.Weinberg converted the data into a format for display in high resolution for filming in 70mm film format. Through my own rediscovery of the models of solar system and their relationships, I arrived at a 12 day year, each day spanning 30 degrees of a 360 degree rotation about the sun. In working with Dr. Weinberg, we separated the tasks of art/design and computation, so that we each focused on our strengths. Support for this film came from the Science Museum of Minnesota, among others.

Historical Models in related Media:
John Whitney, Sr.

John Whitney, Sr., considered the father of computer graphics and animation, was an experimental filmmaker who came to the use of computers from his need to control graphic artwork below a film camera. He used a World War II analog computer, an M-5 Antiaircraft Gun Director, augmented later by an M-7, and hybridized it to make the first motion-control films. As Gene Youngblood writes in Expanded Cinema (1971 - page 208), "The analogue computer work gained Whitney a worldwide reputation, and in the spring of 1966 International Business Machines became the first major corporation to take an "artist in residence" to explore the aesthetic potentials of computer graphics. IBM awarded Whitney a continuing grant that has resulted in several significant developments in the area of cybernetic art."

Computer Art at Bell Labs

Kenneth Knowlton, computer scientist working in the sixties and seventies at Bell Labs in Murray Hill, New Jersey, collaborated with a number of artists, including Stan Vanderbeek and Lillian Schwartz. Vanderbeek worked with Knowlton on a number of films, including Poem Fields, "whose micro-patterns seem to permutate in a constant process of metamorphosis which could very likely include a hundred thousand miniscule changes every second."

Gene Youngblood in Expanded Cinema (page 248) interviewed Vanderbeek about his work with Knowlton at Bell Labs:

"The present state of design of graphics display systems," Vanderbeek explains, "is to integrate small points of light turned on or off at high speeds. A picture is 'resolved' from the mosaic points of light." The artist seems to feel that this process bears some physiognomic similarities to human perception. "The eye," he notes, "is a mosaic of rods and cones." '

Other scientists, such as B. Julescz and C. Bosche, made very interesting and aesthetic works as a result of experiments in the psychology of perception and human vision. The works of these and other collaborative efforts over the past two decades has led directly to the development of our current computer imaging and display technologies and applications.

Bell Labs Sound and Speech Synthesis Research

While Bell Labs is famous for its computer graphics artists, it is also famous for its Composer-in-Residence program, in existence for over two decades. The Composer-in-Residence Program, as headed by Dr. Max Mathews, was set up in the early 60's in an effort to make computers available for cultural use, as well as an aid in the research effort. As Dr. Mathews, in Computer Composers -- Comments and Case Histories stated,

"Encouragement was offered from a feeling of responsibility for bringing new technology to the attention of those in the cultural community who can make use of the new tools, and partly because increased understanding of sounds may improve speech communication. Composers are very creative and have very good ears for making and analyzing sounds. Much more could be said about the impact of art on technology..."

In my interview with Dr. Mathews, now at Stanford University, I asked, "Did the composers help to accelerate the development of the sound and speech synthesis technology? "Yes," said Dr. Mathews. "A good example is Jean-Claude Risset," who was both a physicist and composer. Risset made the first catalog of sounds with a computer, including brass sounds such as a trumpet, flutes, woodwinds, pianos, bells, and "a host of percussion instruments plus numerous non-instrumental sounds. In addition," writes Mathews, "he has invented another musical dimension--spectral pitch control--which an be used to produce tones which seem to be simultaneously ascending and descending in pitch. He has become so expert in tone quality that he can achieve a desired timbre with almost no experimentation. He has made his expertise available to the world by making a catalogue of sounds. I believe that this is the most important musical document of the century." This concept of a sound catalog, and the models set forth by Dr. Risset, is part of virtually every synthesizer on the market today.

I asked Dr. Mathews if the composers at Bell Labs had to be their own technicians and scientists. He said that while many were able to develop their own technological devices and tools, this was not required. The Laboratory was set up to provide a supportive scientific environment. Since the composers were formally invited into the Lab, computer time was set aside for their work, though sometimes the hours were odd and in the middle of the night. They were invited to make experiments in music as well as to compose pieces.

Recognition, Documentation, and Formal Integration

As discussed, artists and scientists have worked together for many, many years, but now it is beginning to be a more formal collaboration. The examples of current team-work at such places as the University of Illinois Center for Supercomputing, imply a movement,a profound change that is now being noticed. These formal collaborations are, however, still more the exception than the rule. Even though artists have been and continue to be of great influence, their positions in research environments are relatively informal. This results from a general lack of articulation of the artists' contributions, and lack of justification for support of the artist. Therefore, the influence and contribution of the artist has been virtually undocumented. Leonardo,an excellent publication on the subject of art and science, begins to address this need, but it is not nearly enough. And while the work of Ms. Cox at the University of Illinois is an excellent model and starting to be documented, much remains elsewhere to be recognized and formalized. There is a great need for an honest and expanded study of the contributions artists make to science throughout the United States. An investigation should be undertaken, a report written and published, and formal changes made in the granting and support procedures in agencies, so that serious consideration is given to the artist, reflecting this awareness. Arguments and justifications for the formal integration of artists into scientific research should be made at the highest levels, so that this becomes a standard, not an exotic and unusual situation. Changes should be made so that artists are not expected to work for free.The institutions that include artists will probably be seen as visionary, forward thinking environments, and may in the end have better chances for support, because their research may be better. This support and reputation would attract the best artists in the field, bringing strong ideas and the best energies to the research environment. The formalization of the artist in Scientific Visualization in particular, is especially important since the work draws so much from the visual artist who is already very much involved.

Artist-in-Residence Programs

Not only should there be justification for Artist-Scientist Teams, but there should be formal Artist-in-Residence Programs, where an artist is working on their own work, as did the composers at Bell Labs. These people should be paid like any other researcher, and should have access to funding from the same sources as the scientists. Special grants should be made available to artists from government agencies, as well as from organizations that benefit from the work, such as ACM SIGGRAPH and large computer companies. A portion of the funds for science should be designated for the artist's research. Artists should be considered equal members of the research environment, and their work should be recognized as such. Their work should be seen in the context of the working environment, the influence it has on the sciences, and should be published in the scientific journals when the content merits it. The artists selected for residencies should be carefully screened, so that their capabilities, interests and goals relate to the overall goals of the research environment. It is very important, though, that the artist be free to pursue their own research ideas, as this will bring an increased possibility for advancement of the field.

Scientist-in-Residence Programs

Scientists can benefit from immersion in the art environment. They can learn a great deal about their own perception and develop excellent communication skills. They can work from these skills towards solving problems of not only aesthetics and refined visual composition, but also user interface design, and new applications as set forth by artists working in various media. For example, multi-screen interactive digital-video performance, computer controlled sculpture, or the integration of sound and image may stimulate new ideas and directions for research. They may be learning more about the psychology of perception and the working and thinking processes of artists, so as to make better tools for artists, and other users of computers. This could lead to many, many new applications and insights. It can also be of great benefit to the art environment, helping both the artist and scientist to develop new tools and reach new goals. Because this exploration of tools and methods used in, and the methods for computer art and animation, the work really is research. But because it takes place in the art environment, it is not formally designated as such. Therefore, it is my feeling that funds for the work of scientists and artists in art environments should be available from organizations that support research elsewhere.

Art Shows

Just as an individual stimulates the research environment, so does the work of a collective group of artists. The value of art shows in general, and computer art shows in particular cannot be measured in dollars spent by attendees. There are other rewards in life besides money! The artworks can lead us to an improved quality of life, suggesting new ideas or improvements in our working methods and living environments. The true value of art shows like those at ACM SIGGRAPH and NCGA, lies in these intellectual rewards, the seeds they plant in the minds of the viewing and participating audience. It may be years later that the seeds take root and lead to major research efforts or applications. But without the art shows, the seeds cannot be planted. It is of critical importance that art shows such as these be supported, for their impact is invaluable, measurable only over a long period of time.

Art shows should not have to be mounted at the expense of the artists (a common reality). While some of these artists are also scientists with Ph.D.'s who manage to produce their artwork supported by other activities, most artists struggle very hard, without an equivalent amount of support. Because of the nature of their work and their unusual backgrounds, they may not fit into a normally funded slot. Therefore, I feel that it is important to recognize when one is "taking" from the work and ideas of creative artists, and do one's best to "give back." Through active support for artists and art shows, we can continue this important cross-fertilization and help the entire field to grow.

The Role of Education

To help the next generation of researchers, we need educational role models, institutions leading the way, not afraid to step forward and set an example. We need educational programs which actively bring together artists and scientists at all levels. We need to rethink and restructure our curricula, requiring math and science for all art students, and humanities and art for all science students. We should create new courses which specifically discuss the relationship between art and technology, and include, for example, a lecture series which brings in role models for the students to see and hear. We should embrace the concept of hybrid renaissance people, and for older people who are already specialized, create courses which foster mutual understanding and collaboration. We should help to improve visual literacy on the part of sciences, and technological literacy on the part of the humanities. We can make use of existing programs, expanding them where necessary. Art schools have already started this process, now it is up to science institutions to level the balance.

Making use of Existing Art Programs

Art schools have the courses already developed for a good visual training for the sciences. Existing art programs within universities are rich and useful resources. Instead of re-inventing color theory, composition, and media studies, the science schools should be working toward making use of art departments, sending their students to classes in art history and art production, and hiring artists from the art departments to help with teaching as well as research projects which involve visualization. Why is it, though, that so few science departments have made use of these resources? As in research, the problem seems to be one of justifying the role of art in support of science. Call it "Visualization Engineering" instead of "Art," said one scientist. Perhaps this is the solution, finding a new name for the technologically literate artist. In any event, I hope that this problem will be resolved as an articulated need for visual literacy on the part of the science community becomes more and more of an issue. Art History and Visualization

The change from "computer graphics" to "visualization," what I call the second stage of computer imaging, parallels the history of visual art, wherein the desire to accurately represent they optical image realistically gave way to surrealism, impressionism, and abstraction. This developed as artists looked for a way of representing abstract visual concepts and realities beyond the directly perceptible. As the level of sophistication of computer visualization increases, and as researchers seek to make the invisible visible, one can sense the direct application of the history of art to Visualization. By studying both the history of art and contemporary art, it is possible to gain insights into the aesthetic and conceptual manifestations and uses of images in society. The accumulated knowledge of the fine arts can be extremely useful to Scientific Visualization, a field which will rely more and more on visual skills and ideas. A study of art history can help to gain insights into visual form giving and unique ways of solving problems which could enhance the scientific research environment in new and unexpected ways.

Other Art Courses

Since so much of Scientific Visualization is based on visual communication, a formal visual training beyond art history is necessary. If scientists wish to take on the work of the artist themselves, they need better skills for drawing, visual and temporal composition, experience with film and video analysis and production, including animation, as well as 2 and 3 dimensional design.

From artists and art courses, scientists can learn about the psychology of perception and its application to image making. For example, opposite colors on the color wheel cause figure-ground disparities and seem to "jump." In an enclosed space with little ambient light, darker colors imply a greater distance from the viewer than do lighter colors (depth cueing). In a space with a great deal of ambient light, such as outdoors in daylight, colors fade into the grey of the atmosphere, that is, they lose saturation as a function of distance. Artists know this from observation and analysis.

Scientists can learn about composition and visual balance from a graphic design course. As an example, a symmetrical design appears static, whereas a scene set assymetrically and at an angle, appears dynamic. Dynamic images are usually more interesting to look at. In addition, studying various color models, such as the color sphere of Philip Otto Runge (1810), the Albert H. Munsell model (1898), and the CIELAB model, which is known to the science community, can help in understanding color. A study of the work of such masters as Albers, Itten, and Klee can supplement this study and be of immense value in the application of color to form and composition. In film animation one can learn from the work of Hans Richter, who developed a vocabulary of temporal spatial articulation by eliminating form and color, and working only with the movement of the screen itself before a film projector. Through an architecture course, one can combine a formal training in plane and solid geometry with the study of architects such as Frank Lloyd Wright, and begin to understand our relationship to the space in which we live and work. In general, art education can help scientists to understand their relationship to the visual world and how to use it in their work.

Art Schools and Computers

Art schools not only teach traditional art history and production, but computer graphics and animation as well. This should be seen as no surprise, as artists have always worked with the tools of their age. Computers for graphics were not readily available to artists before about 1980, and due to the steady decrease in cost and increase in power, art schools are now providing courses in computer art, programming and mathematics. Where in the past, artists who wanted to learn about computers had to enroll in computer science programs (Duane Palyka, of New York Institute of Technology is an excellent example), this is less the case today. The new generation feels illiterate without training in computers, and virtually all art schools worth their salt offer computer graphics and programming courses. Perhaps not as demanding as full fledged computer science programs, they are forging the way towards an integrated artist-scientist training ground. Science institutions, while not yet embracing the arts with the same conviction that the art schools embrace technology, are beginning to express the need for visual literacy. At some point the two fields will find that their curricula overlap, and that they are much better off if they pool their students and resources.

California Institute of the Arts

More and more scientists, as well as people from other disciplines, are applying to the Computer Animation Program at the California Institute of the Arts (CalArts). We have students entering this program with backgrounds that include biology, computer science, geology, oceanography, physics, chemistry, as well as the more traditional art backgrounds of literature, painting, and photography. All entering students must have a good portfolio, an original vision and a well developed ability to visually express themselves. The artists entering the program without much computer experience are strongly encouraged to study electronics and programming, in order to gain a deeper understanding of the tools they use, so that they may have a larger technical base from which to create. Most of these people find an internal motivation for this, and therefore are not required to take the courses. Curiously, more students take programming and electronics courses when the choice is theirs to make. In the case of a computer scientist who is already far along the technical path, I ask, "Why does this person want to study at a fine art school?" In the case of one computer scientist who recently applied, he answered, " Now that I know the techniques of computer graphics, I want to learn about the meaning of images, how to compose them and work with motion and structure in time. I want to work towards a humanistic application of my knowledge, and to develop my knowledge of and skills for communication through images... I want to realize my inner vision."

What we are seeing is the growth of a diverse and dynamic computer art program, one which is formed largely by the arts moving closer to the sciences as the sciences are moving closer to the arts. It is a field in terrific flux and growth, as each new student brings a wealth of educational and personal experience to their work and to the new field. Being both highly technical as well as highly malleable, the uses of computers in art and animation at CalArts borders closely on research, since the possibilities for serious creative work is vast. There is a great deal of freedom for development of computer applications at CalArts, perhaps more so than in more rigid environments. Though there exist very real financial and hence, technical limitations, the intellectual limitations are few. Perhaps this freedom is what attracts the unusual multi-disciplinary student to computer art programs like that at CalArts.

CalArts and Caltech

In addition to my work at CalArts, where I am Director of the Computer Animation Laboratory, I hold a position as Visiting Associate in Computer Science at Caltech, where I pursue my personal work and try to improve my understanding of computing through auditing of classes. I am also active on several levels in trying to catalyse increased interaction between the fields of art and science. I continue to invite researchers from throughout the field, including Caltech and the Jet propulsion Laboratory (JPL), to speak in my classes at CalArts. I try to keep an ongoing verbal and visual dialog between CalArts and Caltech. A few years ago, I mounted a gallery exhibit and live performance of computer art, animation and music called "ArtsTech '85," in order to bring together artists and scientists from the CalArts, Caltech, and Jet Propulsion Lab (JPL) in a celebration of technology and art. This show received major attention in several international journals, including Novum, a European graphic art publication. The evolving relationship between CalArts and Caltech has led directly to the employment of a number of my students at Caltech and JPL, where their duties have included computer aided illustration and design for scientific texts, as well as the creation of computer animation for the simulation of Galileo, the un-manned space probe to Jupiter. In addition, I have discussed the formation of joint educational programs for students at CalArts and Caltech. A sample curriculum for institutions which house computer science and art departments is described in general, below.

Sample Curriculum

A program which would prepare a student for work in the field of Scientific Visualization could be twofold, a Master of Fine Arts Degree, as well as Master of Science Degree in Visualization. Students would focus in either the arts or the sciences, or an extended program could join the two, yielding a double degree. Faculty and curricula could be created entirely anew, or through joint educational efforts formed from existing programs. In the latter case, an equal number of students from both disciplines would be accepted into the program so that neither the the sciences nor the humanities departments would bear the major burden of resource allocation, as facilities would be shared. Syllabi would be geared towards students and their special backgrounds; artists would learn applied math, as well as general computer science courses, including programming, electronics, Very Large Scale Integration (VLSI), and algorithms; and for the scientists, drawing, film, video, animation, as well as art history and aesthetics would be offered. It is very important to have the very best teachers for this purpose, as attitudes impacting the future of the field will surely be formed through these courses. In addition, new courses in Scientific Visualization would be created and taught by experts in the field.

The following college level courses would be spread out over a 4 year period: 

Art students would take 1 per semester:     Science students would take 1 per semester:

  * Algebra                                 ** Art History, incl. architecture 
  * Trigonometry                            ** Drawing and Painting
  * Plane and solid geometry                   Sculpture
  * Differential and Integral Calculus         Photography
    Topology                                ** Film and Video analysis and production
    Biology                                    Animation
    Chemistry                               ** Graphic Design
  * Physics                                    Electronic and Computer Music
  * Electronics                                Performance Art
 ** Programming Languages:                  ** Interdisciplinary Art      
    C                                       ** Computer Art   
    Lisp                                    ** History of Technology in Art
    Prologue                                 * Geometry
    other new languages
 ** Algorithms for Graphics
 ** VLSI and Simulation
 ** Artificial Intelligence and Expert Systems                        
    Statistics
    Engineering Courses as related to Specialty interest

* = required if not taken in high school     ** = required courses
Special Courses bringing together artists and scientists could include:

Summary

99% of the population is affected by visual language and communications media, and while visually sophisticated, most people are, including our scientists, relatively unconscious of this visual language being used. Scientific visualization leads scientists to the need for an understanding of this visual language, as images become the major mode of data representation and communication. Since the eyes take in vast amounts of data in parallel, images can be thought of as a high-bandwidth data representations. Kinetic, three dimensional models can describe complex behaviors which may be impossible to understand or communicate any other way. Since so much of our invisible natural world is spatial, the computed models which give these phenomena form can communicate better and more widely, what before only a few people could model in their minds. Great care should be taken to the display of this data, as visual decisions can dramatically affect its comprehension. Visual representation of information is a skill which anyone can study and learn, either through example, or through formal study of the visual arts.

Visual thinking relates to not just the output stage of science. For many, many years, science has made use of visual models or paradigms. Science education should reflect this fact, teaching geometry and art in an effort to help the next generation of scientists. The integration and application of art to science and vice-versa, should be taught at all levels of our educational systems, so that as our technology for communication develops, our users are not left behind. An organized effort to merge the fields would enhance the efforts and accelerate the growth of Scientific Visualization.

To facilitate and formalize the particularly considerable contribution of the arts to science, we should create funded Artist-in-Residence Programs, funded by the government and private companies, where artists would work on their own research projects, the product of which could integrate very well with overall technological research and development. This is based partly on companies such as Apple Computer, which employs artists in various capacities, including in user interface design teams, and partly on the model of the Bell Laboratories sound and speech synthesis research effort. Bell Labs has had a formal Composer-in-Residence program for over 25 years, which has led directly to the sound and music technology available on the consumer market today. In the field of Scientific Visualization, the artist's understanding of visual perception and communication, and their contributions to the identification of research problems, should be formally recognized and integrated into the scientific environment. In addition, I encourage the formation of Artist-Scientist Research Teams, where artists would aid in the visual display of data collected from scientific research. This is already taking place at the Center for Supercomputing at the University of Illinois at Urbana-Champaign. I also support the creation of Scientist-in-Residence Programs, where scientists would pursue their work in the context of an art school, in order to gain as much insight into visual thinking and aesthetics as possible. We should support Computer Art Shows and Exhibitions which bring together artists and scientists for a cross-fertilization of ideas. Finally, I encourage mutual curriculum development between art and science institutions which could help to bring together the two worlds in a common endeavor. It would yield a new generation of visually and culturally literate scientists and technologically literate artists, and potentially providing us with a unified individual with dual capabilities, the truly contemporary Renaissance Person, well prepared for the emerging field of Scientific Visualization.

First written and delivered as a talk with the same title in 1987 at JPL

Email: vsorensen@gmail.com
Go to
Go to
Go to
Go to
Go to