PreOrder Cinematic Articulation in Motion Graphics

story © Michael Betancourt, May 25, 2021 all rights reserved.


Michael Betancourt's Cinematic Articulation in Motion Graphics develops a critical and theoretical approach to the semiotics of motion pictures as they are applied to a broader range of constructions than traditional commercial narrative productions.

This interdisciplinary approach begins with the problems posed by motion perception to develop a model of cinematic interpretation that includes both narrative and non-narrative types of productions. Contrasting traditional theatrical projection and varieties of new media, this book integrates analyses of title sequences, music videos, and visual effects with discussions on classic and avant-garde films. It further explores the intersection between formative audio-visual cues identified by viewers and how viewers’ desires direct engagement with the motion picture to present a framework for understanding cinematic articulation. This new theoretical model incorporates much of what was neglected and gives greater prominence to formerly critically marginalized productions by showing the fundamental connections that link all moving imagery and text, whether it tells a story or not.

This insightful work will appeal to students and academics in film and media studies.

PreOrder from Routledge here

Here is a preview of the chapter abstracts:


The introduction identifies the broad scope of materials considered in the study and introduces the approach to semiotics employed throughout the analysis as well as its relationship to past theorizations. This book is not meant to be a replacement for existing theory, but a corrective expansion of scope that allows the integration of semiotic approaches to narrative cinema with those offered by its new model for addressing the marginal and neglected non-narrative media of the avant-garde, animation, and motion graphics. This opening to the analysis introduces the concepts of audience desires as guiding interpretation and the role of ‘intentional’ cues in directing the audiences to understand what they see as-if it has been encoded, an essential decision in semiosis for photographic and motion imagery.

Theoretical Approach

Existing semiotic theories are not incompatible with this proposal. This section of the introduction reviews earlier theories to develop an integrated, synthetic model based in pragmatics of use. This approach constrains how the present theory derives from immanent exemplars. It situates the current proposal as a development from earlier, general semiotic theory written by Umberto Eco in Theory of Semiotics, and Eero Tarasti’s useful distinctions of exo-sign (perception) and endo-sign (apperception) are adapted for the consideration of motion pictures. The analytic offered by this study is specifically an expansion of Eco’s en passant observations about ‘mobile articulation’ in motion pictures that was not developed in his discussion: kinesis in motion pictures offers a new, unique expressive articulation.

Avoiding Narrative Assumptions

Cinema semiotics has been almost entirely concerned with questions of narrative elaboration. This section surveys the problems created by this focus and distinguishes the approach taken in this book from earlier cinema semiotics proposed by Jean Mitry, Christian Metz, Peter Wollen, Warren Buckland, and Jurij Lotman in order to propose its own model as an expansion and corrective. The issue of sources matters for this analysis since those theories based on a consideration of only works of narrative cinema makes assumptions about the medium and its organization that explicitly exclude the work of both animation and avant-garde film; motion graphics is not even addressed.

Scope of Analysis

This section discusses the types of evidence and research employed in this study, distinguishing between three types of material that inform the analysis: [1] primary sources, [2] experiential evidence, and [3] diagrammatic summation. All three types serve important evidentiary roles in the argument, providing a material restraint on generalizations and theoretical proposals without basis in practice. This discussion identifies the range of materials employed and how they are used to discursively. Experiential evidence is especially important for the discussion of perception. Cinematic articulation is theorized through a model where the categorical assignment of roles enables the distinction between different types of interpretation without requiring those interpretations to become fixed and unchanging.

§ 1. The Cinematic Sign

Chapter 1 develops a general definition of ‘kinesis,’ and concerns its identification as a third type of articulation unique to motion pictures; the ‘cinematic sign’ and its contingent, mobile articulations develop from this basis. Tarasti’s proposal of exo- and endo-signs allow a differentiation between perception and apperception, and expand on Roy Harris’s integration of signs with perception that serve as a reference point for the proposal of a ‘cinematic sign’ as a contingent and unstable identification. This basis explains how cinematic signs are examples of what Eco identified as “mobile articulation.” His model of “mobile articulation” and “sign-functions” developed in Theory of Semiotics enables a variability of interpretation in motion pictures that radically undermines the conception of a singular or fixed meaning (or series of meanings) for any cinematic sign.

§ 1.1 The ‘Natural’ and the ‘Encoded’

Cinema begins as an experience of events in time: this proposal of ‘event’ clarifies Tarasti’s exo-sign for motion pictures. Insisting on the distinction between depiction and reality for movies allows a consideration of the question whether cinema originates as an encoded structure with the sender (as in language) or are a natural event that can be interpreted as a sign. This discussion demonstrates the dependence of higher level interpretations on low level articulations. The capacity of the viewer to shift between a diagnostic and symbolic mode of engagement with cinema depends on the fluidity of engagements with ‘the real’ offered by photographic depiction.

§ 1.2 Depiction (Diagnostic) and Signification (Symbolic)

The immediacy of perception in motion pictures provides a basis for an exploration of indexicality and realism and the shifting roles for cinematic signs at different levels of interpretation. Eco’s proposal of “sign-functions” are used to consider the entanglement of perception and apperception in higher levels of interpretation.

§ 2. Refractive Judgment

The sorting of perception (a process termed “refractive judgment”) into relevant and irrelevant (signal and noise) is the foundational moment for the transformation of perception into apperception and its development as articulation. This section proposes a theory of refractive judgment as the mechanism responsible for imposing order and coherence on perception based in both gestalt theory and cognitive perception, using Harris’s work as a reference point, thus allowing the advance into higher levels of articulation by sorting and then re-sorting apperception.

§ 2.1 Poetry and Articulation

The issues of “poetic” discourse proposed by Roman Jakobson and adapted for application to cinema are a recurrent theme of this analysis since poetic interpretations depend on high level transformations of what appear to be fixed and certain lower level interpretations. This section introduces these dimensions of the analysis, providing a framework that becomes more complex and dynamic as the analysis progresses.

§ 2.2 Refraction

Refractive judgment is used as a mechanism to shift from one level to the next throughout this theorization: the initial separation of signal and noise is only the first act of refractive sorting. This section discusses how this process repeats as the mechanism productive of higher level signs, a demonstration of how cinematic signs are instances of Eco’s “mobile articulation” that expands on his proposal of interpretations that proceed through “sign-functions,” but without signs.

§ 2.3 Re-Articulation

Higher levels of interpretation are products of how existing cinematic signs become the building blocks for new, derivative signs. This process of re-articulation is explained as a process were refractive judgment reassesses lower level signs as the foundation material for higher level articulations. It produces a recursive semiosis by design, one in which changes to initial perceptions radically alter higher level understanding.

§ 3. Interpreting Cinematics

This chapter concerns the problematics of semiotic engagement with visual imagery and motion pictures. It develops its argument using the experiential evidence gained from examining optical illusions such as the Necker Cube, Duck-Rabbit illusion, and metamorphic paintings. This approach to perception develops the thesis that audiences engage with cinema as-if it were intentionally encoded, using this initial assumption as the justification for their engagement.

§ 3.1 Ambivalent Perceptions

The cinematic sign is a polyvalent object whose contingency is a reflection of the ambivalence/instability of perception of perception, demonstrated by optical illusions; the unstable perceptions exploited by avant-garde films such as T,O,U,C,H,I,N,G by Paul Sharits connect these observations to cinema. This recognition develops into an argument for the imposed nature of semiosis and articulation that serves as a demonstration of “mobile articulation” in action.

§ 3.2 Motion Perception: Kinesis

This section develops a working definition for kinesis and its role in cinema. It connects contemporary cognitive semiotics to earlier work with gestalt psychology, addressing well known problems with that earlier work and arguing for the understanding of motion pictures as beginning with the correlations between any two successive frames proposed by animator Norman McLaren and avant-garde filmmaker Peter Kubelka.

§ 4. Synchronization

Sound and its relationship to image serves an important enunciative role in the organization and parsing of cinematic articulations into higher levels of complexity. Synchronized sound acts as a mediator between articulation and enunciation, developing more complex structures through its capacity to link disparate shots. This chapter addresses the role and use of sound in cinematic articulation.

§ 4.1 Ordering Statements

This section develops a theory of synchronization and its relationship to cinematic realism. The role of sound in establishing the expectations for realism (either as naturalistic or stylized) informs the types of synchronization possible in cinema: the direct connection of sound-image, as in ‘lip sync,’ or the emergent relationships of counterpoint.

§ 4.2 The Unity of Sound and Image

The role of past experience and established knowledge for interpreting sound-image relationships is the focus of this section: the ways that sound becomes an enunciation in itself. This section explores the role of sound and synchronization as an expression in itself using the title design for The Man With The Golden Arm by Saul Bass as a case study.

§ 5. The Presentation

The substance of the depiction is complemented by how those materials are presented to the audience; however, the exhibition is typically ignored in analysis of cinema because the proscenium showing of a standard movie theater is the assumed method for showing movies. This section expands beyond that assumption, developing the presentation as an essential mediation of all cinematic signs. This model considers the neglected dimensions of cinematic morphology and structure, in the process offering insights into the identification of shots, sequences, and the higher level organization based on the role of audience perception in distinguishing cues that indicate change. This chapter provides insights into how the encounter directs higher level interpretations by constraining apperceptions.

§ 5.1 Time

Time in cinema is a problematic concept that requires elaboration, providing a foundation for the other discussions in this chapter. The distinctions of depicted time and real (actual) time enable a distinction to be made between the fictive construct, the subjective experience of it, and the actual running time of the motion picture. How time is parsed into discrete units—shots—is foundational to the understanding of cinematic depiction, making the articulation of duration the most basic dimension of kinesis.

§ 5.1.1 The Modulation of the Shot

This section concerns the identification of changes in shot (editing) and its early theorization as montage by Sergei Eisenstein. It uses his typology as a mechanism to discuss the emergent orders created by changes of shot without requiring his Marxist ideological framework. This transformation of a formalist argument into a semiotic taxonomy allows the acknowledgment of degrees of change between images as a key mediator in the parsing of shots into sequences.

§ 5.1.2 Expressing Duration

Those concerns with ‘time’ that lead to the identification of distinct shots depends on a distinction between ‘depicted time’ and the ‘real time’ of exhibition. The issue of apparently continuous shots (“long takes”) used in trik films by Georges Méliès links the expression of duration to the audience’s perceptions of uninterrupted presentation (i.e. the organization of singular shots). This construction of continuous time demonstrates the artifice of cinematic realism depends on the audience’s past experiences with reality as a reference point for their engagement. It prepares for the argument, developed in § 6.1 about the confusion of reality and depiction common to the diagnostic mode.

§ 5.2 Motion

Motion in cinema is a product of kinesis; this section begins with a consideration of different models for understanding movement shown on-screen: as a matter of shots, or as the correlations/differences between successive frames. This discussion expands on observations made in § 3.2: the extent and scope of the shot is the focus of this section. The recognition of changes of shot are a product of a more fundamental distinction based on the degree of change between successive frames and their contextual organization supported by synchronized sound.

§ 5.2.1 The Datastream

Digital technology has replaced the film strip composed from distinct frames with a continuous datastream. The capacity to elide the distinctions between multiple shots is not a new development, but the creation of extended ‘long takes’ has become an increasingly common part of motion pictures. This section considers the ramifications posed by the technique of datamoshing (a process used in glitch videos) which eliminates the distinctions between frames, effectively eliminating the separation of distinct shots.

§ 5.2.2 Deleuze’s ‘Perception-Image’

Giles Deleuze’s philosophical engagement with narrative films makes several claims about shots and duration that are the subject of analysis in this section. The dependence of this interpretive approach to motion pictures on the explanations posed by the high level understandings produced by narrative and story in explaining and organizing motion/space on-screen bring their reliance on lower level constructions into consciousness when addressing non-narrative works.

§ 5.3 Windowing

The combination of multiple images on-screen at the same time has a variety of distinct terms, including mosaic and spatial montage. This section discusses these combinations and juxtapositions as distinct articulations, independent of the modulation of duration or the break-up into discrete shots. This section identifies the discursive, organization role provided by the combination and juxtaposition of imagery on-screen.

§ 5.4 Space

There are two types of space in cinema: that implied by the depiction and the physical space that contains the depiction. This section concerns their relationships and distinguishes between them. This section identifies the assumption of presentation within a proscenium construction, and develops a general framework to consider presentations outside that traditional exhibition mode.

§ 5.4.1 Graphic Space

All screens establish a field that offers a series of internal relationships. This section considers relationships of depicted space, rather than combinations/juxtapositions of discrete spaces (Windowing). The role of realism and naturalism in these identifications links cinematic articulation to everyday experiences and the Pepper’s Ghost stage illusion.

§ 5.4.2 Multiscreen Projection

This section develops the distinction between multiple projections that produce a singular image and the use of different projectors to create spatial juxtapositions of imagery in space.

§ 5.4.3 Physical Space

The actual physical space of the exhibition is the subject of this discussion that considers motion pictures as environmental phenomena rather than as imagery confined to a proscenium by considering distinctions between 2D and 3D screens and their use in art.

§ 6. Diagnostic

The diagnostic mode is concerned solely with considering the depiction’s contents. It does not create symbolic meanings, but acts to identify what appears in the motion picture as an end in itself. This concern with the identification of depictions in themselves is often confused with the use of narrative to explain the depictions: making this distinction enables the disentanglement of realism from claims of cinematic indexicality as well as shows that narrative explanations are symbolic interpretations rather than objective identifications of things-in-themselves. The semiosis resulting from cinematic articulation is a multivalent process shifting between diagnostic and symbolic modes, entangling their significances precisely because cinematic articulation is fundamentally pictorial rather than linguistic.

§ 6.1 Entanglement with the Symbolic

Understanding the diagnostic via realism entangles it with symbolic interpretations. The approach to realism common to Bazin, Cavell and Deleuze provides examples of this conflation of distinct interpretations and the fallacy that connection creates.

§ 6.2 The Diagnostic Modality

Treating cinema diagnostically involves an engagement with the image as representative of what it depicts: this approach is common to the presentation of spectacles, such as sporting events, where the event itself and what happens is of primary importance, rather than its role in an explanation for what happens.

§ 6.3 Against Realism as Diagnostic

The conflation of realism and diagnostic engagements is a fallacy leading to a confusion of fiction and documentary. This discussion establishes the diagnostic mode as an independent, parallel engagement to the symbolic mode.

§ 6.4 Causal Identifications

The apparently “diagnostic” identifications of story and character are distinct from the recognition of what appears in an image: for example, the difference between a picture of an automobile, and the recognition of a specific car that belongs to a particular story is the distinction between a diagnostic identification of content and the interpretive recognition of something within a symbolic construct.

§ 7. Symbolic

Entirely independent of diagnostic recognition, the symbolic realm is a recursive and unbounded domain of re-articulation and reappraisal of signs and their signification. The ‘symbolic engagement’ in motion pictures identifies all those interpretations that depend on intratextual (contextual) as well as intertextual knowledge for their interpretation. Semiotic encoding depends on the audience’s assumption of the image-object having an intentional construction, that it is a symbolic “vehicle” for communication whose organization can be addressed by encultured expertise. These interpretations are the typical focus of cinema semiotics since they concern the meaning and identification of characters, story, allegory, poeisis, etc. That realm of meanings that are the typical concern of semiotics belong to the symbolic. The symbolic modality re-articulates apperception according to the functions that it fulfills for semiosis—first as a lexical/material addressing of ‘the image,’ then again as the organization of these lower level identifications as new formative elements employed by higher levels of signification.

§ 7.1 The ‘Lexical Function’

There are two primary symbolic functions: the ‘lexical function’ concerns the depiction shown by the ‘image-object.’ This dominant ‘transparent’ engagement differentiates linguistic from visual content. It is the most familiar domain of symbolic meaning, encompassing a range that includes both the written text that appears on screen and the identification of depicted content, such as characters and their relationships. These interpretations of the depiction specifically concern those recognitions made by the audience are what produce the identification of the depiction as-typo/graphics or as-image. These engagements constrain the derivative sign-functions productive of story, narrative and linguistic meaning.

§ 7.1.1 The ‘Reading-Image’

The ‘reading-image’ is a specialized presentation of text on-screen where the animation of the typography, its placement, and appearance become expressive dimensions of its meaning. This expression of meaning through the form, animation, and presentation of text is specific to motion pictures since it depends on kinesis, fundamentally distinguishing motion graphics from graphic design.

§ 7.1.2 Text–Image Composites

There are three relationships between text and its background imagery: [1] the figure-ground mode, [2] the calligram, [3] the rebus. This section explains each type and their relationships to each other using examples drawn from title sequences. The correlation of text and image constrain the derivative sign-functions that allow their meaning to emerge as a higher level re-articulation.

§ 7.2 The ‘Material Function’

The second primary sign-function arises from the ‘image-carrier’ becoming a symbolic form in itself, establishing the ‘material function’ as a parallel domain to the ‘lexical function.’ This expressive modulation symbolically represents/transforms the physical medium into an expressive dimension of signification in parallel to the depiction contained in the ‘image-object.’ It is a domain equal in scope to the ‘lexical function,’ but is more familiar from its expressive use in avant-garde film and video art. Understanding the ‘material function’ as an expressive modulation of the ‘image-carrier’ rather than the depiction allows a more robust analysis of symbolic articulations.

§ 7.3 Materiality and Technical Failure

This section discusses the formative cues used to identity the ‘material function’ as a dimension of symbolic articulation familiar from the avant-garde film and contemporary glitch videos, but which has not been fully elaborated upon. Each of the five sources for these expressions is addressed in turn since their different qualities inform their symbolic use and have parallel histories of use in media art.

§ 7.3.1 Distortion

A discussion of the uses for physical distortions of the imagery shown on-screen. Distortions of faces and recognizable naturalist photography are a recurring dimension of optical “play” used discursively since early avant-garde films of the 1910s that have become increasingly common in motion graphics produced with digital technology since the 1990s. This approach includes discussions of imagery in La Folie du docteur Tube (The Madness of Dr. Tube, 1915) by French director Able Gance; Ballet Mécanique (1924) by Dudley Murphy and Fernand Léger; the city symphony N. Y., N. Y. (1958) by Francis Thompson; and the music video Corporate Cannibal (Grace Jones, 2009) directed by Nick Hooker.

§ 7.3.2 Misregistration

Misregistration identifies the technical “noise” of historically analogue recording technologies used expressively. It is discussed through examples drawn from the technical “noise” of historically analog recording technologies—the dirt, scratches, smudges and markings that can distort physical media also impact the playback of digital recordings on media such as CDs and DVDs where the data is accessed physically rather than electronically.

§ 7.3.3 Hardware Failure

These physical failures in the technology itself were an source of expressions in video art in the 1960s. They have a consistent use as a variety of self-reference in motion pictures. The material functions produced by assaulting digital technology, as in Digital TV Dinner (1978) by Jamie Fenton, provides a clear explanation of the ‘material function’ created by the breakdown of the machine itself.

§ 7.3.4 Misalignment

The mismatch between an encoded file and the software that decodes is almost exclusively employed in avant-garde media art. It has two mirror-like variants: the first produces errors through the inappropriate application of decompression by decoding of a compressed file using an incorrect codec, resulting in a transformation of the encoded data, but without damaging the file itself; the second involves the induced opening/reading of a file using an incorrect software application.

§ 7.3.5 Data Manipulation

Data manipulation covers a range of approaches and protocols, all of which are focused on the transformation and alteration of the digital file itself. These are the most common types of imagery used as a ‘material function’ in digital motion pictures. It is this particular engagement with the system as a such as it fails—no matter the particulars of that breakdown—that defines glitch techniques as an ongoing process of investigation, whatever the human readable form might be. The glitch video Monster Movie (2005) by Takeshi Murata provides a clear example of how this process of manipulation transforms the normally unseen encoding into cinematic imagery.

§ 7.4 Derivative Sign-Functions

Both the ‘lexical function’ and the ‘material function’ provide signs that are the foundation for re-articulation into higher levels of semiotic interpretation. These ‘derivative sign-functions’ are the familiar identifications and signifying processes that are the typical focus of cinema semiotics—such as the identification of narrative, issues around realism, the allegorical and metaphoric meanings of fiction—are derivations from the primary symbolic functions. This section discusses several of these functions, contextualizing their emergence from lower level interpretations; these higher level modulations emerge from refractive judgments that modify existing perceptual/apperceptual significations. The shift from the identification of “content” to its meaning that concerns this section.

§ 7.4.1 Cinematic Realism

Cinematic realism emerges from the naturalism of live action photography that mediates the audience’s engagement with depiction via cultural expectations about the organization and nature of reality. This reference to lived experience constrains and informs all aspects of cinematic articulation, but only becomes ‘realism’ when these experiential encounters are assigned a role as a discourse on the nature of ‘the real.’

§ 7.4.2 ‘Narrative Function’

Cinematic narrative depends on the audience inventing explanations for what they identify in both the diagnostic and symbolic modes (the initial identifications made as ‘lexical and material functions’). Concerns with these discursive interpretations are the dominant mechanism for understanding cinema, but these explanations are derived from initial perceptual/apperceptual choices about how and what the motion picture shows. Acknowledging the audience’s role inventing these explanations for why the depicted events and their sequencing in cinema happen contextualizes these imposed interpretations as a superstructure developing over and around non-narrative cues and articulations, thus linking the narrative and non-narrative types of cinematic production.

§ 7.5 Poeisis

This concluding section returns to Roman Jacobsen’s ‘poetic function.’ It is a terminal point for semiosis that puts its transformations of lower level enunciations and a higher level significations into a context. The poetic re-articulation where signs become signifiers within more complex level signs is itself a process of refraction that sheds familiar identification to assume a new meaning. ‘Poetic cinema’ makes its dependence on the mobility and contingency of cinematic articulation apparent.

Afterword: Cinema and Motion Graphics

The conclusions to this analysis reiterate the importance of audience fluency and response as mediators of the cinematic encounter, thus they are decisive in producing the articulations required for both the recognitions of pictorial ‘content’ and those of interpretive/narrative signification. Questions of how perception determines articulation provide a foundation for all higher level interpretations. Returning to this foundational part of cinema semiosis brings the analysis to a close by returning to the challenges reanimated by digital technology (the debates over post-cinema) that began the study. This proposal that places perception at the inaugural moment of semiosis has ramifications for more than just cinema; it suggests a technique to unify visual semiosis with the lexical avoiding the paralinguistic metaphor for non-linguistic signification.

Copyright © Michael Betancourt  May 25, 2021  all rights reserved.

All images, copyrights, and trademarks are owned by their respective owners: any presence here is for purposes of commentary only.