Procedural Taxonomy: An Analytical Model for Artificial Aesthetics

This paper proposes an analytical model for computational aesthetic artifacts based on Espen Aarseth's work. It reflects procedural affinities that may not be found when focusing on surface structures and aesthetic analyses developed from them. The model attests to the importance of computational characteristics and of procedurality, both as conceptual groundings and as aesthetic focuses, as aesthetics pleasures in themselves.

Author(s)

Introduction

The growing presence of computational media and tools in many areas of contemporary life brings massive change to all who interface with these systems, either as consumers or producers, as spectators or interactors, as writers, readers or wreaders

‘Artificial poïesis,’ the production of computational aesthetic artifacts, is widespread. Computational aesthetic artifacts are created by practitioners with diverse backgrounds, methodologies and terminologies that are not always reconcilable and that create obstacles to mutual understanding, effective cooperation and criticism. However, in spite of contextual variations inherent to each particular field or project, and regardless of the specific functions, contexts or settings of production, there are many commonalities to be found among these works. Various phenomena discovered with or through these media are genuinely new and unprecedented, lacking clear references in other arts or fields of study, as well as a clear nomenclature, a disadvantage for their practice and study.

This work hopes to contribute to the development of a terminology for computational media, by proposing a framework for their study and criticism that is versatile and plastic enough to accompany their ongoing transformation and its effects in creative practices.

Motivation

The starting point for this work was Espen Aarseth’s model for the analysis of cybertexts. [4] Although tailored to textual artifacts, this model presents several advantages: 1) it is focused on the structural, functional and procedural traits of the texts, rather than on their surface features or contents; 2) it is extensive enough to encompass different media and expressions; 3) it emphasizes common features found across most of the artifacts, rather than aspects that may be specific to some; 4) it acknowledges the interactive potential of the artifacts, without establishing a precedence over other important characteristics for the production of meaning and the development of the aesthetic experience; and finally, 5) it is workable, with a set of seven variables and eighteen possible values that create a space of 576 unique media positions.

By applying Aarseth’s analytical model to a broader range of aesthetic artifacts, we asserted its efficacy and were then able to adapt and expand it, in search of a more comprehensive description of the works. The variables were tested for suitability and with the exception of one, all proved to be usable in the new model. 

The model

Dynamics

The first variable in Aarseth’s typology describes the contrasting behavior of signs in static systems – where they are constant – and in dynamic systems, where we repurposed the original values to describe surface unit dynamics (SUD) and deep unit dynamics (DUD), following a nomenclature inspired by Krome Barratt. [5] SUD describes rearrangements of perceivable structures without the transformation of their foundations which is described by DUD. 

Determinability

Determinability concerns the stability of what Aarseth defines as the “traversal function” [4] of the artifact. This is the set of conventions and mechanisms that combine and project surface and deep units to the user. [3] If multiple experiences of the same artifact may result in similar behaviors or even in exact repetitions, we classify it as determinable. If on the contrary the artifact may lead the traversal function as much as, or even more than the users themselves, driving the experience into unknown territories and forcing users to adapt or react to new usage scenarios, we classify it as indeterminable.

Transiency

Transiency describes the temporal existence of the artifact. If the mere passing of time causes changes in the artifact’s outputs then it is transient, otherwise it is intransient.

Access

Access describes whether the totality of the artifact or its outputs are available to the user at all time, in which case the access is random, otherwise being controlled.

Linking

Linking describes the existence of rules or devices that may lead the user through the traversal and whether the access to these is explicit or conditional.

User Functions

The last variable in Aarseth’s typology describes which functions are available to the user besides the omnipresent interpretative function. In the explorative function, the user chooses which paths to follow along the traversal while in the configurative function new structures, i.e. surface or deep units, may be rearranged or created. These two functions are what “in addition to the obligatory interpretative function” [4] define an ergodic medium. 

Modalities

Modalities will quantify the levels of perception involved in the user functions. They are defined sensorially [8] – visual, audial, haptic – and expanded with the perceptions of motion and procedurality – that of mathematics and of logical structures [11] – raising their total number to five.

Autonomy

Autonomy is a descriptor of the system’s capacity to generate novelty – or to be somewhat creative – without resorting to external inputs. Autonomous systems either contain or generate all the data they need to produce novel outputs, while systems fed by external sources – or that include extensive sets of hard-coded data, digital data structures or digital streams, according to Berry [2] – are classified as being data-driven.

Class

This variable details the computational class – understood after Stephen Wolfram’s definition [12] and Rudy Rucker’s interpretation [10] – that better describes the outputs of a system. Static intransient outputs were classified as class 1, most of the static transient outputs as class 2, and those that exhibit complex behaviors as either classes 3 or 4, using the structure of the outputs to determine whether the system was class 3 (random, totally unpredictable) or class 4 (structured, at least locally, and at least partially predictable).

Variables and Possible values

  1. Dynamics: static, SUD, DUD;
  2. Determinability: determinable, indeterminable;
  3. Transiency: transient, intransient;
  4. Access: random, controlled;
  5. Linking: none, conditional, explicit;
  6. User functions: interpretative, explorative, configurative;
  7. Modalities: 1-5;
  8. Autonomy: autonomous, data-driven;
  9. Class: 1-4.

Data collected

We compiled a set of representative samples, collecting diverse approaches to procedural creation and focusing on visual arts and design. Besides a set of pieces of our own choosing, we collected an independent selection of works, trying to avoid a bias towards the model under development. The complete list of 54 works and the details of their analysis are to extensive to present in this article, but can be found in our previous works. [6] [7]

Analysis

After classifying the works according to the model, and still following Aarseth’s methodology, we used the R environment for statistical computing and the CA package [9] to develop a Multiple Correspondence Analysis (MCA). The first synthetic variable achieved 54.1% inertia, but a plotting as a one-dimensional graph revealed the lack of indispensable information that was added by the extra 8.6% of data variation provided by the second synthetic variable. We therefore, opted for plotting the MCA as a two-dimensional graph describing 62.7% of the data variation.

Control Analysis

This model was developed with the purpose of allowing objective classifications and of minimizing subjective factors. Trying to test the definitions of the variables and our own analysis, we developed a control analysis, providing the list of systems and a description of the model to an independent analyzer.

The understanding of most of the variables was straightforward. The greatest challenge was found with modalities variable, especially with the classification of the procedural and haptic modalities. The control analysis tended to classify as haptic all those systems that allowed any degree of interaction, regardless of which devices were used in the process. Our analysis used different criteria: standard controllers (e.g. mice or keyboards) used in established ways (e.g. as in operating systems or productivity tools) were not classified as haptic; only works that used dedicated controllers or that employed standard controllers in non-conventional ways were considered to heighten haptic awareness and involvement. The control analysis also found the procedural modality in more instances, something that may be due to regarding the outputs of a work as being part of its system and not as independent artifacts, that may or may not be procedural or able to communicate procedurality. The procedural modality is tied to the perception, understanding or intuition of mathematics and logical structures. It is only when the outputs of a system present a minimum of clues for that understanding that this modality can be identified. In some cases this classification can be somewhat subjective, because it is historical, it deals with acquired knowledge and learning.

The control analysis revealed a divergence of 7.4% – 36 contrasting classifications in a total of 486. The divergence in the classification of modalities is not a sign of arbitrariness, but the effect of the false positives created by different understandings of the variables described above. We found that in a majority of cases, the divergence was explained by the extra classification of procedural (eight) or haptic (twelve) modalities in a work. Should we choose to disregard this effect, we could interpret the divergence in modalities as a much lower 5.5%, lowering the total divergence to 3.29%.

Divergences in the control analysis

  1. Dynamics: 3 divergences, 5.55%;
  2. Determinability: 0 divergences;
  3. Transiency: 0 divergences;
  4. Access: 0 divergences;
  5. User Functions: 1 divergence, 1.85%;
  6. Linking: 2 divergences, 3.7%;
  7. Modalities: 23 divergences, 42.59%;
  8. Autonomy: 0 divergences;
  9. Class: 7 divergences, 12.96%.

Findings

Studying the plot of the MCA, we find that the periphery is taken by works that originally stood somewhat apart from the rest of the selection due to their contrasting physical characteristics. These are Christa Sommerer and Laurent Mignonneau’s A-Volve (#4), Carvalhais, Tudela and Lia’s 30x1 (#27) and Andreas Muxel’s Connect (#40). The work that is most isolated is Olia Lialina’s My Boyfriend Came Back From the War (#6), which is also the only narrative hypertext, plotted logically and consistently.

In the east edge of the plot, we find a series of printed or otherwise static outputs, such as Roman Verostko’s Seven Sisters: The Pleiades (#9) or Andy Huntington and Drew Allan’s Cylinder (#16). The west area, in contrast, is predominantly populated by interactive systems. By circumscribing both areas, we find that there is no overlap and that two well-defined islands are created in the graph.

A closer look at the categories encompassed by the areas allows us to understand which values are more typically associated with them. In the eastern quadrant, we discover works that are mostly static, determinable, intransient, randomly accessible and with no linking. Deep unit dynamics, conditional linking and the explorative and configurative user functions characterize the interactive systems that also tend to concentrate more modalities and to develop higher computational classes.

The single book among the pieces, Raymond Queneau’s Cent Mille Milliards de Poèmes (#1), is found in the middle of the non-interactive island, a placement that raises the question of whether books can ever be understood as interactive devices. Following Schubiger’s definition [1] of interactive systems as supporting communication from user to the system and back, or Lippman’s definition of interaction as a “mutual and simultaneous activity,” [4] it becomes clear that regardless of any manual reconfigurations that may be developed, a printed book should never be classified as interactive. Although the configurative user function is involved, it does not follow that a cybernetic feedback loop can be established because the system is not able to act on its own. If we circumscribe the systems that produce computer-based outputs or real-time computations, we also find a clear division between two sets.

It is not possible to infer much about an eventual genre partitioning. We wondered whether this could be a shortcoming of the model or if traditional genres may be unsuitable to the description of computational media. If we study pieces plotted in coincident coordinates, we discover that traditional descriptions such as sculpture, painting or drawing, do not prove to be very useful. We can find two of the works most easily identifiable as sculptural – Cylinder (#16) and Andreas Nicolas Fischer’s A Week in the Life (#39) – plotted very closely but still in different coordinates, sharing positions with systems that produce visual-only bidimensional outputs. We find linear videos plotted in neighboring positions, but still not necessarily in the same coordinates, something far more common among systems that produce printed outputs. It is also interesting to discover that two of the pieces where a strong directionality (and irreversibility) of time is patent – William Gibson’s Agrippa (a book of the dead) (#3) and John F. Simon Jr.’s Every Icon (#7) – are plotted in the same position. Although, in an initial analysis, they may seem to be very different systems, belonging to different genres or artistic typologies, they share strong procedural traits, turning out to be much more similar than one would originally expect.

The coherent distribution of the classified artifacts that is found in the plot of the MCA contributes to a validation of the current state of the model. The analysis of clustering may eventually lead to the discovery of new genre descriptors.

Future research

This work studied systems that could broadly be classified as visual arts or communication design. Aarseth’s previous analysis, from which some works were preserved, focused on pieces that could generally be classified as literary. In the future we expect to broaden our field of analysis, by increasing the quantity and variety of works. The common characteristics discovered in this set of works lead us to believe that such a follow-up study needs to be developed, allowing us to refine the model and to further develop the study of the procedural and haptic modalities as better definitions of both are undoubtedly necessary.

A complementary path to follow is the approach to the 'perspective' variable from Aarseth’s model, that focused on the text requiring the user to play a strategic role as a character in its diegesis, and that we did not succeed to integrate in the presented model. Artificial aesthetic systems are created from processes, and narrative aspects may be generated from procedurality and the procedural modality, from the user’s desire to witness the unfolding of processes and from the simulations and predictions that are inevitably created. A complete study of procedural media must include their narrative properties without loosing sight of the remaining procedural aspects so far surveyed. Although a partition between the study of rule-based and story-based aspects of systems is certainly possible, we search for a dialectic model, where one is able to reintegrate perspective and understand how narrative emerges from rules.

Acknowledgements

This work was only possible due to the help, advice and insight provided by Heitor Alvelos and Penousal Machado, supervisors of the dissertation in which context it was developed. [7] We are also indebted to Golan Levin, Lia, Luísa Ribas, Marius Watz and Florian Cramer, for invaluable advice and collaboration. This work was developed with the financial aid of the Fundação para a Ciência e Tecnologia (FCT), under the Programa Operacional Potencial Humano (SFRH / BD / 43877 / 2008).

References and Notes: 
  1. Caroline Schubiger, “Interaction Design: Definition and Tasks,” in Total Interaction, ed. Gerhard Buurman (Basel: Birkhäuser, 2005).
  2. David M. Berry, The Philosophy of Software (Basingstoke: Palgrave Macmillan, 2011).
  3. Espen J. Aarseth, “Nonlinearity and Literary Theory,” in Hyper / Text / Theory, ed. George Landow (Baltimore: Johns Hopkins University Press, 1994).
  4. Espen J. Aarseth, Cybertext (Baltimore: Johns Hopkins University Press, 1997).
  5. Krome Barratt, Logic and Design (Guilford: Design Books, 1980).
  6. Miguel Carvalhais, “Towards a Model for Artificial Aesthetics,” in Generative Art, ed. Celestino Soddu (Milan, 2010).
  7. Miguel Carvalhais, Towards a Model for Artificial Aesthetics: Contributions to the Study of Creative Practices in Procedural and Computational Systems (U. Porto, 2010).
  8. Mitchell Whitelaw, “Synesthesia and Cross-Modality in Contemporary Audiovisuals,” Senses & Society 3, 3 (2008): 259-76.
  9. Oleg Nenadić, Michael Greenacre, “Correspondence Analysis in R,” Journal of Statistical Software 20, 3 (2007).
  10. Rudy Rucker, The Lifebox, the Seashell, and the Soul (New York: Thunder's Mouth Press, 2005).
  11. Stephanie Strickland, “Quantum Poetics: Six Thoughts,” in Media Poetry, ed. Eduardo Kac (Bristol: Intellect, 2007).
  12. Stephen Wolfram, A New Kind of Science (Champaign: Wolfram Media, 2002).