The Groping Ey(e)ar - Sound In Interactive Installations, Objects, and Performances II - Among the most noteworthy projects featured in the 1996 Ars Electronica festival in the section of interactive art was Knowbotic Research (KR+cF) that presented the dynamic infrastructure of the public information system and put forward the potentials of nature with the help of a mathematical apparatus.
Subjective musical worlds of the project’s participants are through interconnectivity (computer networking) defined as one space. Every participant in the projects submits his or her musical contribution via email; this becomes attributed to a homogeneous body in a mathematically defined environment. When spectators enter and immerse themselves in such an interactive, three-dimensional, computer-generated space through virtual reality, they may with the help of an identifier of their hands’ positions choose what composition they wish to listen to in any particular moment. Here is how the setup works out in the acoustic and visual reality: passing from a small cube that plays a fragment from Bach you may enter a nebula floating over your head that represents songs by the Beatles. If you change your hand’s position between these two locations, but in their various sections, you may mix them together, thus creating your own “co-mix”. By remembering the locations of sounds that you liked, you may use the system as a sampler and create your own composition out of subjective musical pieces submitted by the project participants.

The musical interactive project In the Body (In Corpus) by M. Redolfi and L. Martinez allowed spectators swimming in a pool to listen to sounds in an environment where they are transmitted four times faster than in the air. Cameras identified spectators by the colour of their goggles; then, through a mathematically defined composition, they delivered an individual acoustic experience to each spectator as he or she was swimming about in the pool. Swimming under water evoked virtual reality; this was the authors’ way of stressing how rich in experience the “poor, non-virtual” reality can be.

The most interesting project visitors could see at the 1996 AE was Tod Machover’s Brain Opera.

The Brain Opera project is based on texts and ideas by Marvin Minski (Mind Society) and technological innovations of the Massachusetts Institute of Technology – Media Lab in Boston, and is the largest presented project containing new elements in interactive art. The cooperation and interaction of artists, scientists, musicians, and technologists produced its result in a complex images-and-sounds network environment. It is one of the ways to approach the fulfilment of Marvin Minski’s idea that “in future, when more reliable machines for reading brain waves will have been invented, it will perhaps be possible for people to hire composers to create works aimed not at any audience but create them for specific clients. Then it will perhaps be considered mastery to compose a piece that will delight this particular client, although everyone else may hate it.”

In the Brain Opera project, Tod Machover afforded 40% of space to active listeners. His concept allows for the ambition of every human being to compose a musical piece at least once in a lifetime; to this end he created hyper-instruments whose control is made possible by the most elementary physical movements and actions of humans. Instruments forming the Mind Forest are constructed so that it is easy to learn to play them. Talking Trees employ Marvin Minski’s recorded voice whose selection is influenced by the interactive communication instigated by the listener. Singing Trees receive a simple tune sung by the listener to a microphone and reproduce the tune creating an additional musical “aura”. Rhythmical Trees enable by strokes of hands to start the playing of preselected samplings of fragments of musical works, vocals, or electronically manipulated sounds. The Melodious Music-Stand enables the listener to draw a melody by the movement of hands; the melody will then be heard as a remix of fragments of Brain Opera composed afresh. An interesting facet was the openness towards direct input from the Internet that had the capacity to edit the final version of the composition.

Throughout the day musical input from active listeners on all hyper-instruments was being collected; afterwards, within an hour’s time, they were mixed into the final version following a predetermined pattern, so that in the evening the concert could take place that consisted in 60% from the original composition by Tod Machover and in 40% from modifications stemming from the input gathered during the day on the hyper-instruments and via the Internet.

The concert thus transformed into a musical work by a free and active audience, a collective improvisation that employed a spontaneous as well as deliberate usage of sound-creating hyper-instruments based on samplings. The concept counts on the creation of one’s own computer and electroacoustic compositions on the basis of the “immersed” spectator’s emotions; at the same time it is a step towards the creation of one’s own Home Operas that would become intimate for each creator. The future “non-professional” music lover may become a composer and may at any time attach his or her own music to a computer-animated movie that might feature celebrated movie stars – even these may today be concocted on one’s own desktop as much as easily as home printing and publishing is accomplished. All such activities may be distributed via webpages on the Internet. The meaning of this is not to connect music, text, favourite melodies, ideas, and memories, but the way in which our mind transforms such fragments into a coherent attitude towards the world by realizing the mind’s own archetypal music.

Jon Rose is a protagonist of “improvised music” that combines improvised and contemporary classical music. It arose as a result of new ways of life (satellites, MTV, teleshopping) while absorbing the most recent philosophic attitudes (deconstruction) and integrating them into music (collage, samplings); it continues to rely on the interpret’s mastery in the handling of the musical instrument. At the 1996 festival, Jon Rose introduced the project Improvised Sport and Evolution – an interactive musical show featuring two badminton-players with rackets connected to a computer. The players’ strokes set off samplings that depended on the dynamics of the play. The interactive improvised game was a wry and humorous, but at the same time musically interesting project that attempted to point out the similarity of music and sports in two aspects: improvisation and conservatism. Rose made a dent in both myths so that the match ended in his victory.

LOW-TECH usage was typical of AE in 1995-97. On the one hand it is indeed the utilization of low-level technology, but what is stressed is the strong visual aspect of wastage and ugly material; yet these, too, are connected to the most recent computer hardware. It is primarily two projects that have been successful in retaining their sub-character. Near a dump of scraps in Linz’s local steel foundry VOEST, a station for artists was founded where young artists use wastage and electronic appliances to create their mobile, acoustic, and interactive sculptures. The other enterprise is SUB-TRONIC – a joint name for the presentation of VJs and DJs from Austria and the US. Young artists inspired by this cyber-techno-culture create cyber-space works with an artistic (visual and conceptual) shift. During the festival, clubs present techno-parties, titled SubTronic, hosted by star DJs; special attention is paid to the interior decoration of the space accommodating such events. This cyber-techno-visual image is used by young artists in their environments and the interactive space they create. An example is the Internet-based IPcentrum installation that psycho-physically attacked the spectator with a stroboscopic light, bass vibrations felt in the entire body, and by a floor that was throbbing with the magnitude of 9 degrees on the Richter scale (that one ordinarily refers to only in times of earthquakes).
Peter Stelarc presented himself at the AE in 1992 with his multimedia performance Third Hand, today regarded as a classic. The performance’s concept had several levels. The sound that complemented the visual experience of observing a moving cybernetic “living machine” was created with the help of a computer and synthesizers from the information on muscle impulses (EMG) and the status of heart activity (ECG), and from the analysis of brain waves (EEG), blood pressure, and the movement of various body parts. In creating sound, Stelarc used medical apparatuses that controlled analogous synthesizers. The changes of lighting as a further complement of the performance were modulated by EMG, ECG, and EEG signals and by body positions once again recorded by the computer. Still another interesting component were the ceaseless inadvertent shoulder movements initiated by two muscle stimulators that were modifying the electric tension in Stelarc’s shoulder muscles throughout the performance. To Stelarc’s right hand, a “third”, metal hand was attached. It could be turned around by up to 290 degrees and was initiated by EMG impulses from muscles on legs and the abdomen. The hand could grasp, hold, turn around, and let go of objects. Throughout the performance, Stelarc was followed by a video camera suspended on an industrial jib with five swivel joints. The programmed movement made it possible to follow, on a screen, in more detail the relationship between the body’s movement and light, sound, or kinetic reactions. During one segment of the performance, the video was switched over to an interactive module: now it was quicksilver sensors attached to the head and shoulder that served as indicators of the body’s position, thus becoming the source of impulses for camera movements. An endoscope (internal observer) inserted into the performer’s stomach and connected to a large-screen video projection brought live, internal pictures of his stomach; these were constantly changing their dimensions, composed and “choreographed” by the movement of Stelarc’s body.

Five years later Stelarc returned with a new project, Parasite. His body and third hand were connected to the Internet and participants could manipulate them via the Internet with the help of a system of muscle stimulators. The performance was accompanied by the sound of pistons of accelerators and reflexive sensors that controlled the movements of Stelarc’s fingers and joints, but also the sound of music from the Internet as it was mixed by specific movements of his body. In the symposium, Stelarc rejected the interpretation of his performance as “a mechanical marionette”. He favoured the conception that saw his body acting as a host for parasites arriving via the Internet. These parasites choreograph Stelarc’s movement; visually attractive are mainly the manipulations of large muscles such as biceps, deltoid muscles, shoulders, and bendings of knees. During the performance, cyborg/Stelarc selects the Internet participants who will be allowed to manipulate his body. Sometimes, with the help of Internet video broadcast, he enters a conversation with spectators.

The 1997 Ars Electronica Golden Nike for interactive art was in Linz awarded to Toshi Iwai’s project Music Plays Images x Images Play Music - an interactive multimedia project and a computer visualization of the musical production by the piano virtuoso Ryuichi Sakamoto. Via a computer, the pianist during ordinary play created dynamic, geometric elements that were projected onto a screen located behind the piano. The reverse process produced piano play with the help of geometric figures that were touching the keyboard following Iwai’s instructions executed by a computer mouse. The dynamics of geometric figures on the screen was copied by the computer-controlled piano play.

In his visual and acoustic computer performances, the American Matt Heckert intentionally refrains from using samplings or prerecorded tapes; each of his acoustic and visually brutal machines has to have a variable scope of application or variable rhythm. The movement of apocalyptically looking objects is controlled by a computer and each change of input produces a different final sound. The noisy moving machines create a mystical aura over intelligent and would-be scientific machines. The revolving speed may be controlled far more effectively via a computer; the switches between various activities may follow each other with a speed unthinkable in manual control. For playing machines featured in the Munich Samba project, Heckert won the Golden Nike award in the section digital music.

Kouichirou Eto’s Soundcreatures were presented simultaneously on webpages and in a real installation. In a confined area, intelligent robots were moving about detecting obstacles and emitting sounds that allowed them to identify one another. Upon encounter, sound signals were evaluated, with each encounter causing certain modifications in the participating robots’ identification signals. Frequent modifications eventually changed the entire basic sound used by a robot to introduce itself.

Christa Sommerer’s and Laurent Mignonneau’s installation HAZE Express utilized a touch screen and evoked the sense of looking at the countryside during a train-ride; the experience could be enhanced by acoustic and visual and abstract figures generated depending on the surface area, intensity, and frequency with which a spectator chose to apply the touch screen.

In 1999 in cooperation with C3, Budapest, a small, approximately 15 metres long bridge with the floor made of movable boards was installed in front of the AE CENTRE building. If you put your foot on any of the boards, the movement was captured by a sensor and via the Internet a signal was sent to Budapest where, in a public space, a similar bridge was installed: here the corresponding board rose up in the air and fell down again, emitting characteristic clatter. This worked in both directions, so that an interesting, not just visual (kinetic) and haptic, but also acoustic communication dialogue was set up between AE visitors and the population of Budapest. The project, titled BUMP, was afterwards several times demonstrated, or presented on video, in many exhibitions of interactive Internet art in Europe.

This brief overview of interactive and multimedia installations highlights the possibilities offered by multimedia and is by no means exhaustive; its aim was to report on a sufficient amount of representative projects. The works mentioned above show the areas an artist’s activity may access when it is supported by new technologies and latest software.