Leonardo

Snow's Gap

Although Art Center College of Design and the California Institute of Technology (Caltech) are located no more than 5 miles apart in Pasadena, California, the institutions have over the years developed highly specialized academic cultures. A few students take advantage of the opportunity to cross-register for classes, but otherwise we share no infrastructure; we lack even a common academic calendar. Recently, however, the presidents of both institutions established an initiative to build a new relationship that would include project-based collaboration. Rather than inaugurating the process with a large investment or a new interdisciplinary study center, the schools have chosen to begin by researching existing relationships that have formed between students and faculty at the two institutions. Despite the obstacles, students from our schools have met and conducted informal collaborative projects. In one example, an input device for a Caltech doctoral student's gesture-based 3D drawing program was developed using the silicone casting techniques of one of our graduate design students. We hope to take what we learn from our research and build new institutional structures that will foster this sort of give and take.

In the summer of 2000 we began our first joint effort, and the experience was illuminating. Caltech and Art Center applied jointly for a U.S. National Science Foundation grant to fund post-graduate fellowships in entrepreneurship that would team students from our institutions. Some of our counterparts at Caltech, however, thought of design as something to be applied near the end of the product development process: there was lots of talk of "form factors." At Art Center, we have been trying to move away from the sort of understanding that reduces design to a thin veneer of "look and feel." Conversely, I have spoken with Caltech researchers who resent being treated by their collaborative partners as lacking in creativity and useful only when harnessed for their programming skills. I am getting tired of quoting C.P. Snow, but it is clear to me that his 40-year-old thesis is still valid. Artistic and scientific communities still fail to communicate adequately, and this lack of communication is reinforced by educational structures. "Closing the gap between our cultures," Snow wrote, "is a necessity in the most abstract intellectual sense, as well as the most practical" [1]. But I also firmly believe that this "clashing point," as Snow puts it, offers "creative chances" [2] to spark the development of new ways of thinking and seeing.

Besides developing new institutional structures to address this gap, I propose a complementary academic practice. A discipline devoted to navigating the gap between art and science would be of value if it helps us better understand the creative processes embedded in artistic and scientific artifacts and shows us the way to better science and design. As a means of outlining the way in which this practice could be pursued, let us consider the Cornell Box. Mention "Cornell Box" to art aficionados and they are likely to conjure up an image of Untitled (Medici Prince) (Fig. 1) or another of the shadow boxes of American artist Joseph Cornell. To computer-graphics researchers, on the other hand, "Cornell Box" refers to the evaluative environment in which the Cornell University Program of Computer Graphics developed its radiosity rendering algorithms (Figs 2 and 3).

Click for larger view
View full resolution

Fig. 1.

Joseph Cornell, Untitled (Medici Prince), construction, 43.8 X 27.6 X 11.4 cm, ca. 1942-1952. Cornell's constructions are "affirmations of serenity, recollection, enchantment, beauty, the extraordinary" [40]. (© The Joseph and Robert Cornell Memorial Foundation/Licensed by VAGA, New York, NY)

[End Page 249]

This study will invite these audiences to contemplate these boxes together. Juxtaposing Cornell boxes reveals an unexpected program shared by a reclusive genius making objects in the art world and graphics researchers exploring computer modeling of light. In both cases, the Cornell box can serve as a locus for exploring creativity, perception and light. And just as light takes on a bit of coloration from each surface as it reverberates through a room, so this study will attempt to transform the gap between two communities into a shared space of illumination.

Click for larger view
View full resolution

Fig. 2.

Cornell University Box, computer-generated rendering, 1985. The Cornell University Box is a physical object that corresponds to a computer-generated model. (Reprinted by permission)

The Two Cornell Boxes

Joseph Cornell's shadow boxes occupy a unique place in the history of twentieth-century American art. Originally shaped by the Surrealist taste for assemblage, Cornell's work also resonated with the subsequent movements of Neo-Romanticism, Abstract Expressionism and Pop Art. Cornell began to create his small, glass-fronted boxes in the mid 1930s, working for the most part in series. His obsessive interests, which included movie actresses and nineteenth-century ballerinas, informed the vocabulary of the boxes; and their physical design was influenced by the traditions of the diorama and Wunderkammer, as well as vernacular sources such as the peepshow and the arcade game. Critical appraisal has focused on the poetic nature of Cornell's work, its capacity to evoke wonder and the processes of memory. "Founded in the magic and mystery of the poetic experience," wrote one critic, "his collages, films, and constructions are affirmations of serenity, recollection, enchantment, beauty, the extraordinary" [3].

Click for larger view
View full resolution

Fig. 3.

Cornell University Program of Computer Graphics web site <http://www.graphics. cornell.edu/online/box/>. (Reprinted by permission)

The Cornell University Box (and here I alter the name slightly for clarity) is a physical object that corresponds to a computer-generated model, and it is used to test the predictive accuracy of light propagation algorithms. Beginning in 1984, researchers, headed by Donald P. Greenberg, published studies that altered the way in which people thought about modeling light behavior in the computer and led to substantially improved computer-generated renderings [4]. The use of the box was central to their approach. Light algorithms can only be predictive, the reasoning goes, if the results can be verified in a real-world environment. Further, computer-generated renderings of an environment can only be considered accurate if they are proven indistinguishable to the eye from a built version of that environment [5]. So the results of the calculations from the virtual box were compared with light-meter readings from the physical box, and perceptual studies were run to compare the rendered results to a viewing of the box itself [6]. Since its first publication, the box has become the de facto test bed for advanced rendering systems [7]; and, as more challenging materials are attempted, the renderings have begun to take on the strange sense of displacement one finds in the paintings of de Chirico, for example [8]. The box had no name in the early papers. When queried about the name and Joseph Cornell's artwork, Greenberg offered only that the box gained the Cornell moniker after a number of publications from the university [9]. The lack of kinship between the two boxes, and the two cultures, would appear to be complete.

Joseph Cornell's Basement

Fiction can help us navigate this gap between computation and art. In his 1986 [End Page 250] novel Count Zero, William Gibson conceives of a machine capable of producing the Cornell boxes of the future: "There were dozens of . . . arms, manipulators, tipped with pliers, hexdrivers, knives, a subminiature circular saw, a dentist's drill. . . . Two of the arms, tipped with delicate force-feedback devices, were extended; the soft pads cradled an unfinished box" [10].

Click for larger view
View full resolution

Fig. 4.

Cornell's studio, 3708 Utopia Parkway, Flushing, NY, 1969, silver gelatin print. (Photo © Hans Namuth Ltd.) The sheer quantity of unused material directs critical attention to the importance of filtering in Cornell's method.

Orbiting the earth in an abandoned mainframe core, the machine is surrounded by a whirling drift of weightless objects used to fill its boxes, including "A yellowing kid glove, the faceted stopper from some vial of vanished perfume, an armless doll with a face of French porcelain . . . " [11]. The machine's activity of selection and rejection creates a kind of dance through which it fashions its constructions, which embody, as Gibson puts it, "the solid residues of love and memory" [12]. The procedural construction of Cornell boxes in Count Zero serves as a metaphor for the working of the creative mind. The creative act always involves rejection of options: the excised or obscured element, the brush stroke withheld. But the archives of the collage artist (Fig. 4) constitute a rich store of visual material that has not found its way into a completed piece.

Cornell's work lends itself to this sort of fictional treatment because of the fascinating way in which much of the artist's creative process can be visualized spatially. Unlike the boxmaking machine in Count Zero, Joseph Cornell was an ambulatory gatherer. In his journals and published statements, the artist attributed much of his inspiration to strolls around Manhattan, including the seminal insight to put objects into boxes. If moving through the world in search of material was one pole of Cornell's working process, the other was sorting his finds in his basement workspace and filtering them into boxes. Photographs of his workspace (see Fig. 4) would appear to confirm his biographer's contention that the odds were one in a thousand that a piece of archived material would actually end up in a box [13]. The sheer quantity of material directs critical attention to the importance of filtering in his method; indeed his biographer calls Cornell's art an art of "distillation" [14].

The Creative Algorithm

Creative-algorithm generation can similarly be appreciated as a process of distillation. The behavior of light in even a simple space is an exceedingly complex phenomenon; in any room, there are billions of photons bouncing around. The development of algorithms for modeling light behavior is thus a great place to explore creative thought in graphics research. An important conceptual breakthrough in the history of rendering was the development of raytracing. The principle of economy that makes raytracing effective is the recognition that, while there are billions of light photons bouncing around in any environment, the photons that really matter are those that enter the eye (Fig. 5) [15]. The group at Cornell University further refined the ray-tracing principle of economy. The photons that are important are those that enter the eye, and the eye itself has limitations and needs. The thresholds of the visual system inform the parameters of the algorithm. Moreover, the eye also has needs that are not met by the raytracing process, and it is here that the Cornell [End Page 251] group made its greatest contribution to rendering. Raytracing algorithms fail to produce renderings that capture the richness of a scene because they are not complex enough to take into account most of the secondary reflections that occur from surface to surface in real space. In any environment, we might say that there are surfaces directly illuminated by light from sources and surfaces that are illuminated indirectly by the light reflected from other surfaces. This indirect illumination we call ambient light.

Click for larger view
View full resolution

Fig. 5.

Raytracing. Illustration (Lightscape Visualization Systems. © Discreet, Inc.). While there are billions of light photons bouncing around in any environment, the photons that really matter are those that enter the eye.

The conceptual breakthrough that led to the modeling of ambient light in the Cornell radiosity algorithm was in recontextualizing existing models of thermal energy transfer to address the problem of light propagation [16]. Greenberg himself prefers to emphasize the incremental advances, the give and take of collaboration, and the insights that allowed some vexing problem to be solved [17]. Radiosity algorithms model interreflection by creating a mesh that stores the illumination values associated with each surface (Fig. 6). After distributing energy from the light sources, the algorithm checks each surface and determines which has the most energy to reflect back into the environment. The progressive distribution of light energy continues until all of the energy in the environment has been absorbed, or until the process is stopped.

The Light-Filled Space

Reversing our strategy, and examining Joseph Cornell's work through the lens of the Cornell University Box, has the benefit of directing our attention to the theme of light in Cornell's work, a consideration that has drawn little critical attention. One of the defining characteristics of Joseph Cornell's boxes is that they present just enough depth to engage light on objects in real space, while still drawing upon 2D pictorial conventions. The way in which Cornell boxes engage their illumination environment is crucial to the play of dimensionality that is one of their most interesting features, and the exquisite sensitivity to light and its effects that emerges from the artist's journals suggests that this feature was carefully considered. Indeed, Cornell was a master of raising his perception of light to conscious reflection. An entry from Cornell's 1945 journal celebrates the exhilaration of an early bike ride: "effects of light striking & shining through—now indistinct as though filled with smoke" (Cornell's emphasis) [18]. It is instructive to see how the artist practiced his sensitivity to light in his day-to-day environment; the light of his dreams was equally transcendent. In 1963 Cornell recorded this journal entry after waking from a nap: "in a large whitish room—sense of light strong—spacious . . . great sense of everything white—but more than just physical ambience—a sense of illumination" [19].

The artist also exhibited considerable care for the interaction of his boxes with the light of their environment. In a wonderfully telling journal entry from 1945, a dismissive description of an encounter with Salvador Dalí is contrasted with the rapturous satisfaction he felt with one of his swan boxes: "1st snow . . . Met Dalí . . . Transcendent feeling about swan box—outdoors bright clear air white clouds etc. reflected in mirrors of box with blue glass and white fluffy feathers" [20]. It is sometimes difficult to speculate about the ways in which artists design light interaction into their work; after all, it often is not possible for the artist to control the conditions of viewing upon giving up the work, unless it was designed for a particular place. There were, however, certain illumination environments in which Cornell did have a say: his gallery exhibitions.

Cornell's exhibition at the Julian Levy gallery in 1939 was one in which darkness prevailed. The "lights were turned down, and thimbles, dolls and bits of broken glass could be found both in and out of shadow boxes" [21]. The effect of this subdued level of illumination upon Cornell's work of the period would create deep boxes, obliquely lit dramatic spaces. The exhibition at the Egan Gallery in 1949 had much more of a high-key look; if Joseph Cornell and the researchers at Cornell University made something together, perhaps it would look like that space [22]. In the design of the space, the Cornell Box is projected into a full three dimensions: the internal divisions of the boxes are extruded out to become wall elements, and the space itself is transformed into a light environment that echoes the illumination layout of the 1985 Cornell University Box. Let us reverse this dynamic imaginatively and consider Cornell Boxes as spaces in which light effects are explored. Cornell's Egan Gallery exhibition was installed a white room bathed in even light. The aviary boxes in this series featured white linings, some presenting a kind of flatness of aspect that echoes the work of Cornell's Abstract Expressionist contemporaries, others offering hidden recesses and depth. The play of flatness and depth, reflection and transparency, is clearly a major source of the power of Cornell's work.

Light and Surface

A further benefit of considering artistic and scientific artifacts together is the possibility of identifying or generating shared vocabularies and terms of reference that can help support collaboration. Both types of Cornell Box enable a kind of vision that is informed by the perceptual psychology of James J. Gibson (and, for clarity, further mentions of Gibson will refer to James, not William) [23]. Gibson's concept of natural vision takes the human body and its environment as its starting point; it "depends on the eyes in the head on a body supported by the ground" [24]. Moreover, Gibson's model assumes a moving perceiver, afoot not in three-dimensional Cartesian space, but in a medium with vertical polarity that corresponds to our experience of the pull of gravity. And unlike the geometrically defined world of classical optics—in opposition to which Gibson framed his theories—the world of ecological optics is rich and nuanced, full of surfaces and clutter that afford the moving perceiver the visual information necessary to make sense of her surroundings [25].

The spatial assumptions from which Gibson proceeds are as different from the process of computer generated rendering [End Page 252] as they are from classical optics, a fact that helps explain the gut aversion that many have to the "cold" look of early computer graphics. A glance at a shading algorithm makes it clear why this is so. The ambient term in most rendering algorithms is an arbitrary constant and has nothing to do with the real dynamics of surface-to-surface interreflection in an environment. And, as Gibson points out, it is the visual information conveyed by ambient light that most powerfully orients us in our environment. Gibson is drawn to light that reverberates, from particle to particle, surface to surface [26]. One can easily understand why, given his presumption of a moving observer. The information transmitted by ambient light is view-independent and provides the constant—or "invariant," in Gibson's terminology—that helps people understand their environment as they move through it. The radiosity algorithm developed at Cornell adds the missing component of ambient light to the digital rendering process. "Reverberating" light is made present in the algorithm by considering each object in the environment as "a secondary light source" [27], and visual needs are thus inscribed in the radiosity algorithm.

Perceptual Limits

The status of Joseph Cornell's boxes as objects of perception can best be grasped through contrast to the work of James Turrell, a light and space artist whose work—like Cornell's—tends to evoke critical praise for its mysterious, spiritual qualities [28]. Some of Turrell's light installations also bear a resemblance to the 1985 version of the Cornell University Box [29]. It is no surprise that Turrell's work engages issues of visual perception; he studied psychology in college and participated collaboratively with Edwin Wortz—a physiological psychologist—in the Art and Technology program at the Los Angeles County Museum of Art in the late 1960s [30]. Together Turrell and Wortz explored the Ganzfeld effect, producing small hemispheres in which viewers experienced a homogeneous, undifferentiated light field. In the Ganzfeld effect the retina is stimulated by light but no visual detail is present, a phenomenon that played a central role in the development of Gibson's theories of thresholds and visual information. Gibson described Metzger's experiment, the original exploration of the Ganzfeld effect in the 1920s, as follows:

He faced the eyes of his observer with a large, dimly lighted plaster wall, which rendered the light coming to the visual system unfocusable . . . The total field (Ganzfeld) was, as he put it, homogeneous. Under high illumination, the observer simply perceived the wall . . . But under low illumination, the fine-grained texture of the surface was no longer registered by the human eye, and the observer reported seeing what he called a fog or haze or mist of light [31].

The implications of the Metzger experiment were highly significant to Gibson. "The experiment provides discontinuities in the light to an eye at one extreme," he wrote, "and eliminates them at the other" [32]. This allowed Gibson to differentiate his theories from classical perceptual psychology precepts of stimulus and response. In darkness, Gibson writes, "vision fails for lack of stimulation. In homogeneous ambient light, vision fails for lack of information" [33].

Click for larger view
View full resolution

Fig. 6.

Radiosity. Illustration (Lightscape Visualization Systems. © Discreet, Inc.). Radiosity algorithms model interreflection by creating a mesh that stores the illumination values associated with each surface.

If we were to consider the art of Cornell and Turrell in terms of Metzger's experiment, letting Cornell Boxes stand for the fine-grained wall surface, and letting Turrell's work correspond to the Ganzfeld, we can then see that each artist explored a perceptual limit. As objects of perception, Cornell Boxes offer the body intimate clutter, small, easily manipulated spaces that are progressively subdivided to the level of micro texture, affording the eye plenty of the information it craves [34]. The reach of the arms defines the scale of the box, and within that range the box is easily scanned by the area of the retina that specializes in the resolution of fine detail. The experience of a Turrell Ganzfeld space is nearly the opposite. It is an encompassing space that is full of light but devoid of visual information, and the effect on the body can be striking. Some visitors to a Turrell Ganzfeld installation at the Stedelijk museum in Amsterdam felt so disembodied they had to crawl through the space on hands and knees; eventually, a path had to be cut through the exhibition on the floor [35].

Rather than creating works that encompass the viewer, Cornell's work is characterized by the process of miniaturization and progressive refinement of textures. Cornell's working method supported this focus on texture. His biographer noted that after constructing the shell of a box, Cornell tried to make the surfaces of the interior look antique: "He might line a box with blue velvet, aged wallpaper, or torn-out pages from a nineteenth-century book. Or, using house paint, he might apply six or seven coats to the interior walls of the box, until the surface looked crusty and acquired a seeming patina of age" [36]. Cornell used other techniques to speed up the aging process, such as baking his boxes in the oven [37]. Moreover, he reworked his boxes frequently over time, and much of the recursive effort documented in his journal was directed towards refining the textures of the boxes: "Cleaned up some and got at a box started long time ago. Good accidental antique effect by applying black stain after surface had been treated with paste filler (years ago)" [38].

One obvious practical strategy for collaborative design—if one accepts the perceptual significance of the art of Turrell and Cornell—would be for artists and scientific researchers to explore other threshold experiences that one encounters in perceptual psychology. What could one make that would allow the viewer to feel extremes of low-light environments, or the boundaries between textured and non-textured spaces? [End Page 253]

Humanizing the Infinite

One winter night, when Cornell was home from Andover Academy as a youth, he walked into his sister's room, trembling violently. According to his biographer, Cornell "walked to the window, gazed into the darkness, and explained that he had been studying the concept of infinity in his astronomy class" [39]. In the end, both Joseph Cornell and the graphics group at Cornell University were trying to humanize the infinite. Whether one trembles before a starry sky or the expanse of unbounded Cartesian cyberspace, the fact is that the human visual system is nourished by clutter, the microtexture, the light that is reflected under the desk. By learning from this tale of two Cornell boxes, we can hope to design new interfaces and experiences that combine the richness of Joseph Cornell's box with the complexity afforded by computing.