Hybridity: things come together as they fall apart

3.2Hybridity: things come together as they fall apart

“For instance, in motion graphics text takes on many properties which were previously unique to cinema, animation, or graphic design. To put this differently, while retaining its old typographic dimensions such as font size or line spacing, text also acquires cinematographic and computer animation dimensions. It can now move in a virtual space as any other 3D computer graphics object. Its proportions will change depending on what virtual lens the designer has selected. The individual letters, which make up a text string can be exploded into many small particles. As a word moves closer to us, it can appear out of focus; and so on. In short, in the process of hybridization, the language of typography does not stay “as is.” Instead we end up with a new metalanguage that combines the techniques of all previously distinct languages, including that of typography.” (Manovich. Pg. 89. 2008 draft)

Language’s assimilation by image (and the development of a new metalanguage) is a story that precedes digital mediation; it begins with pictures and language occurring in the same inscription. In the following sections, I outline in rough the processes by which language and images have co-evolved toward a space where digital media (joining them with audio and interactivity) fuses them into a singular entity and makes aesthetic animism both probable and possible.

3.2.1Language’s Latent Tongue

René Thom. Structural Stability and Morphogenesis.20

I extracted the René Thom quotation that opens this section from an essay⁰ by the digital poets Stephanie Strickland and Cynthia Lawson Jaramillo, who also contemplate the confluence of mathematical systems and poetic language. Poets, programmers and linguists each utilize recursion. Poetic recursion occurs semantically. In Chomskey-inspired linguistic theory, language is a system defined by recursive (i.e. formulaic) relations. Letterforms, scripts and signs are outward expressions of the system’s underlying system. When combined with a script or alphabet, language extrudes from its neurological network into a representational and contingent form.

Up until now, occidental letterforms appear mostly arbitrary, bearing little resemblance to the structures of speech sounds. Unlike ideograms which often refer to real physical pictorial processes, occidental letters seem arbitrarily conjoined. Digital technology offers space to heal this gap between form and content; by modelling the geometric resonance of speech, visually expressive letterforms emerge. Digital media offers a means to construct letterforms that more closely approximate the actual structure of morphemes (the constituent sounds of language)⁰.

In the following sections, I trace a history and offer examples (from vispo, digipo, ads and media art) that show (quickly and partially) how technical methods of inscription have evolved. Previously, I described why I think visual language evolution is on a trajectory toward becoming a real-world object. The shape of these letterform objects might correspond to embodied structures: visual analogs of mathematical structures that arise from the acoustic resonance inside our bodies. It can be argued that much of proportional aesthetics (theories of golden mean etc.) arises from embodiment, evolutionary activity over millennia etching patterns in physiognomy.

What I am suggesting is that innate shapes (geometry in Thom’s terms) already exist for letterforms. They implicitly underlie our oral audible language, they are subconscious sculptures intuited from the shape of diaphragm, larynx, mouth, lips and tongue. They have been etched there by speaking. Some shapes are personal, some shapes are cross-cultural. Yet it is these shapes and vibrational presences that are being given birth and dimensional form within 3D animation, ads, and digital poetry.

Audible language already existed physically so as it is mediated it is not likely to mutate faster. Interactivity (as many hypertext theorists have already pointed out) has always been part of reading, so it is not without precedent. The bulk of my discussion will therefore deal with the introduction of malleable dimensionality in language’s visual aspect. Interactivity and multimedia discussions will build on these foundations.

3.2.2Bouba/Kiki : Shape-Sound Synaesthesia

What this suggests is that at the locus of shape, sound and semantics, various proprioceptive mechanisms arise. Humans understand shapes within their bodies; sound-shape associations have been found in many cultures.⁰Sound-shapes pairs are also not arbitrary because they arise from repetitive activity: spoken muscle memory like song’s scar on the trachea. Our tongues and breath passageways have memorized how to create specific sets of sounds; and thus, the morphemes that comprise language are cross-referenced as volumetric forms. Children inherit these morpheme-shape pairs as they learn words. As they say something, they feel it (i.e. Put the tip of your tongue behind the teeth, and say sssssss … do u see the snake?).

In the poetic realm, sound poetry obviously investigates the acoustics of the body. Kurt Schwitter’s merz performances recognized the mouth as a sculptural form, a tunnel for a torrent of morphemes⁰. The Four Horsemen performances in the 60s and 70s (continued on by Paul Dutton) explored breath and sound, scream and guttural glottal, as means for poetic expression. Joan LaBarbara, Diamanda Galas and Meredith Monk have each unknotted voice from conventional formulations. Beatbox stylist crossover poets like Reggie Watts⁰ lithely intertwine pop, rap, post-modern comedy and breath-based table-esque stylings. All of which suggests a fertile range for visible text to augment the voice’s innate spatiality that has not been significantly developed.

Poets are uniquely adapted to explore this terrain of embodied language. And since both volumetric kinetic text and breath-based sonic-poetry are marginal outgrowths perhaps there is a reciprocal necessity for amalgamation. A potential research direction for volumetric text could investigate how often it is accepted or rejected on the basis of widely-distributed shape-sound archetypes; as methodologies for non-invasive monitoring of neural states increases, poetic (that problematic term) experiences might be amenable to quantification. Additionally sound-shape archetypes (that correspond to the shapes our bodies make when speaking, the physiological constraints of our internal tubes vibrating as sound waves pass through them) are viable targets for modelling with digital media, -- physical synthesis of component anatomy and acoustic modeling is proliferating. Why don’t we have letters that breath with us? An avatar in this region is Diane Gromala’s (2000) BioMorphic Typography project : “ a family of fonts that respond, in real-time, to a user's changing physical states, as measured by a biofeedback device.”⁰ Computation implies (if not the necessity then the capacity for) synaesthetic cross-modal language.