SHOW
AND
TELL
SHOW
AND
TELL
Master thesis
Marieke Pras
2823853
Research Master Literary and Cultural Studies
Rijksuniversiteit Groningen
First Reader: Dr. Miklós Kiss
On the Potential of Narrative Mapping
For Researching Audiovisual Intertextuality
Within Cognitive and Cultural Narratology
1.1 Narrative before mapping 1.1.1 Textual markers for constructing a diegetic world
1.1.2 Delineating Fabula and Syuzhet
1.1.3 Visualizing cognitive processes in mental and narrative spaces 1.1.4 Conclusions for mapping narrative dimensions
1.2 Narrative and mapping forms of knowledge
1.2.1 Scripts for how the world works
1.2.2 Communicating embodied knowledge
1.2.3 Articulating knowledge through blending
1.2.4 A critical note on cultural knowledge
1.3 Narrative after mapping
1.3.1 Reviewing narrative approaches in mapping projects 1.3.2 Results from mapping projects for narrative research 1.3.3 Conclusions from the narrative perspective
2.1 The problem with intertextuality
2.1.1 Then: developing a working definition of intertextuality 2.1.2 Now: situating cognitive approaches to intertextuality 2.1.3 Future: concentrating on audiovisual intertextuality
2.2 Constructing a framework
2.2.1 Cognitive perspective on intertextuality in audiovisual texts 2.2.2 Producing intertextuality in audiovisual texts 2.2.3 Discursive perspective on intertextuality in audiovisual texts
2.3 Mapping intertextuality
2.3.1 Mapping intertextuality: analysis
2.3.2 Mapping intertextuality: articulation
3.1 The (his)story of mapping
3.1.1 A short history
3.1.2 The practice and the theory
3.2 From cognitive mapping to mapping cognitive aspects 3.2.1 Extending on cognitive mapping; organizing knowledge 3.2.2 Extending on narrative mapping; subjective knowledge 3.2.3 The Power of narrative mapping; framing knowledge
3.3 Contextualizing existing projects
3.3.1 Visualizing frameworks
3.3.2 Visualizing formal elements
3.3.3 Visualizing networks
3.4 Strategies for mapping
3.4.1 Research strategies
3.4.2 Communication strategies
3.4.3 Visual design strategies
3.4.4 Conclusions from mapping (future strategies)
4.1 Overview
4.1.1 Summary
4.1.2 Focus of analysis
4.1.3 Timeline
4.2 Weaving stories
4.2.1 Laura and Shadow Moon; intersecting perspective
4.2.2 Somewhere in America; narrative database
4.2.3 Mapping narrative structures
4.3 Audiovisual stories
4.3.1 Mr. Ibis and Mr. Jacquel; the intertextual backbone 4.3.2 Czernobog in Chicago; constructing narrative spaces
4.3.3 Mapping audiovisual elements
4.4 Cultural stories
4.4.1 Mainstream Media; audiovisual quotations and the star persona
1. On Narrative
2. On Intertextuality
3. On Mapping
4. Case study: American Gods 10 10 12 15 18 19 19 23 25 29 30 30 37 39 41 41 44 47 49 49 52 54 57 57 61 64 64 67 69 69 71 73 75 75 81 83 84 84 87 89 93 96 96 99 100 101 101 105 109 111 111 115 123 125 125 Introduction 4
on narrative structures
1.1.1 Textual markers for constructing a diegetic world 1.1.2 Delineating Fabula and Syuzhet
3.3.1 Visualizing frameworks 3.4.2 Communication strategies 3.4.3 Design strategies
4.1.3 Timeline
4.2.1 Laura and Shadow Moon; intersecting perspectives 4.2.2 Somewhere in America; narrative database
4.2.3 Mapping narrative structures
on mental and narrative spaces
1.1.3 Visualizing cognitive processes in mental and narrative spaces 1.2.2 Communicating embodied knowledge
1.2.3 Articulating knowledge through blending
2.2.1 Cognitive perspective on intertextuality in audiovisual texts 2.3.2 Mapping intertextuality: articulation
3.2.3 The Power of mapping; framing knowledge 3.4.3 Design strategies
4.3.1 Mr. Ibis and Mr. Jacquel; the intertextual backbone 4.3.2 Czernobog in Chicago; constructing narrative spaces 4.3.3 Mapping audiovisual elements
on articulation and discourse
1.2.3 Articulating knowledge: blending the old and the new 1.2.4 A critical note on cultural knowledge
2.2.2 Producing intertextuality in audiovisual texts
2.2.3 Discursive perspective on intertextuality in audiovisual texts 2.3.1 Mapping intertextuality: analysis
2.3.2 Mapping intertextuality: articulation 3.1.1 A short history
3.1.2 The practice and the theory 3.3.3 Visualizing networks
3.4.4 Conclusions from mapping (future strategies)
4.4.2 Bilquis; rebranding as negotiating cultural knowledge 4.4.3 ‘When we are forgotten’; the critical text
4.4.4 Mapping cultural networks
on mapping networks
1.3.1 Application of narratological concepts
1.3.2 Results from mapping projects for narrative research 1.3.3 Conclusions from narrative
2.3.1 Mapping intertextuality: analysis 2.3.2 Mapping intertextuality: articulation 3.3.3 Visualizing networks
3.4.1 Research strategies 3.4.2 Communication strategies 3.4.3 Design strategies
3.4.4 Conclusions from mapping (future strategies) 4.4.4 Mapping cultural networks
on the use of embodied image schemas
1.1.3 Visualizing cognitive processes in mental and narrative spaces 1.2.2 Communicating embodied knowledge
3.2.1 Extending on cognitive mapping; organizing knowledge 3.3.1 Mapping frameworks
3.4.2 Communication strategies 3.4.3 Design strategies
4.3.2 Czernobog in Chicago; constructing narrative spaces
Alternative table of contents
on pre-existing mapping projects
1.3.1 Reviewing narrative approaches in mapping projects 1.3.2 Results from mapping projects for narrative research 2.3.1 Mapping intertextuality: analysis
2.3.2 Mapping intertextuality: articulation 3.3.1 Visualizing frameworks
3.3.2 Visualizing formal elements 3.3.3 Visualizing networks 3.4.1 Research strategies 3.4.2 Communication strategies
Alternative readings
To encourage non-linear readings, the alternative
Introduction
There was a line that Dutch commercials used in the early 2000’s: “In het verleden behaalde resultaten bieden geen garantie voor de toekomst” or “Past results do not guarantee future returns”. In this thesis I am arguing the opposite: when it comes to stories, the past (experience) does affect the future (expectations). Whether we engage with stories, our environment, or how we remember aspects of both; we use shortcuts all the time. It’s convenient. In this thesis I am interested in the ways we understand and articulate our understanding of stories… through mapping them.
There are two things that motivate this thesis. First, I want to extend on current theories on intertextuality for their application to audiovisual narratives. We can simultaneously watch film 1, understand a reference to film 2 and incorporate this reference in our assessment and expectations for film 1. This intertextual process (seemingly) does not disrupt our overall experience but is part of it. But how do we do that?
Secondly, I want to improve how theory and practice can align for doing and presenting research through narrative mapping. Narrative mapping, in this context, opens the door for investigating the potential (and limitations) of visual-based or time-based modes of research. Systems and standards for academic research prioritize text-based research. In this thesis I want to explore the advantages and pitfalls of narrative mapping as another form of doing and/or presenting research. Narrative mapping projects such as The Wrong House (Steven Jacobs, 2007), Cinemetrics (Yuri Tsivian, 2005), or Letteratura Grafica (Francesco Franchi, 2008) have focused on mapping textual markers as a basis for further discussions. Other projects, like Culturegraphy (Kim Albrecht, 2014) have reconfigured data points into visual networks of connections and spheres of influence. What is missing here is a more experiential component. How do we not just map the connections between textual markers, but between textual markers and the audience’s under-standing? How do audiovisual cues structure the audience’s overall experience? How does this progress over the time it takes to watch a film or episode? This is
set of stories. What enables the audience to do so? Can this intertextual process be articulated, and can mapping be a useful part of that process? All these questions lead to the main question I want to answer in this thesis:
What is the potential for mapping in cognitive and cultural research on how viewers understand and articulate their experience of intertextuality in audiovisual
narratives? Scope and Definitions
In order to provide answers, I will have to align different theories and practices from fields such as cognitive narratology, cartography, and film theory. To build my theoretical framework requires a structure in which different aspects of the main question are prioritized. Therefore, the first two chapters will prior-itize cognitive and cultural approaches to narrative and intertextuality and inves-tigate the place and function of narrative mapping therein. Following that, the third chapter will primarily focus on mapping theory and practices and from this perspective continue to contextualize the potential of mapping in narratological research. The final chapter is a case study analysis of the TV-series American Gods (season 1, developed by Bryan Fuller and Michael Green, 2017) where different aspects of the series are analysed to showcase the various applications of my overall theoretical framework.
In order to avoid cluttering later arguments with definitions, I want to give a working definition of some of the concepts in this introduction. In order to fully define these concepts, their functions, and their nuances, entire books have been written - but that is not the purpose of this thesis. Here I am using working defini-tions, grounded in larger discourses and histories in order to explore their potential within the specific context of this thesis.
Narrative: Within the context of this thesis I am considering narrative as a
mental activity where the theoretical focus is on how the audience can interact with text (following Dancygier and Sweetser 2014; Coëgnarts and Kravanja 2015; Van Duijn 2016; Kiss and Willemsen 2017). How do viewers prioritize, connect and remember given information both during and after viewing? How do these processes affect audience construction of (and expectations for) the story at hand? Such questions will mainly be examined through cognitive narratological theories
Audiovisual narrative: The focus is on audiovisual narratives, e.g. films
and TV-series. On the level of the audience-text interaction this means that I look at the specific influence of audiovisual cues on the construction of narrative, as opposed to written texts. This raises questions like: how do mise-en-scène and cinematography create a time and space in which characters (re)act? How can the audience’s memories be triggered through visual and sound design? On the level of production this means accounting for the systems in which film and TV-series are produced and distributed (Bordwell 1985; Mittell 2004; Bordwell and Thompson 2010).
Intertextuality: Here I employ an audience-driven perspective on
intertextu-ality as process based (Panagiotidou 2010; Shonoda 2012; Karpenko-Seccombe 2016), and in that sense a process that happens on multiple levels (a spectrum) embedded in the overall narrative experience. From that perspective this means that intertextuality is enabled by textual cues with intertextual potential. These are (often) the same cues that structure the audience’s mental model of the text. I will argue for this more extensively in chapter two.
Narrative mapping: For a working definition in line with existing narrative
mapping projects discussed here, I argue that the act of mapping is a process of analysis, synthesis and remediation. The produced object is considered a tool for communication or further analysis. This is a broad definition that raises several questions. Is the act and resulting object of mapping always a tool though? At what point do we talk about ‘a map’ or say that we are ‘mapping’? Throughout this thesis I will refine this definition and the use of the terms mapping and map by critically reviewing the application in existing projects.
Relevance and Current Situation
The relevance of this thesis research is motivated by two things. First, there is a gap in the theory on intertextuality in audiovisual texts and audience-centered analysis of the experience of intertextuality. Current cognitive theories on intertex-tuality examine literary texts and mostly focus on how the text cues for intertextual
produce intertextuality (Herman and Vervaeck 2009; Hills 2004, Mittell 2004). What is missing is how the audience can connect knowledge from a text and environment. Secondly, I believe that communicating and presenting research can be done from different and not-standardized perspectives, but they do require critical thinking about what you communicate and how that shapes your research design. Currently many projects, such as Cinemetrics (Tsivian), mentioned earlier, or A Literary Atlas of Europe (Piatti et al.) employ narrative mapping as an analytical tool, and separate the act of mapping in research from the results of said research.
Aims and Methods
One of the aims in this thesis is to develop (and apply) strategies that improve the integration of (narrative) theory and (design) practice. The other aim is to extend on current cultural and cognitive models for analysing intertextuality in audiovisual texts. In order to do so I want to see how cultural and cognitive theories overlap and how they can move away from textual analysis towards the audience and narrative experience
In order to achieve this, my first step in the research was to review current literature, and review and contextualize existing mapping projects. The next step was to align and extend theories in current literature. These steps formed the basis of the theoretical framework presented in the first three chapters of this thesis. The next step in my research was to apply said theoretical framework to singular case study: American Gods. The use of this case study is to show both the application of my arguments and to act as a basis for proposing more practical and large-scale lines of research that are not possible right here, right now.
Outline Chapters
1) Narrative: In order to discuss narrative mapping, this first chapter primarily
focuses on narrative with a secondary focus on where mapping is involved. By discussing how we employ different forms of knowledge in order to engage with and articulate our understanding of narrative the first chapter aims to provide some of the theoretical groundwork for this thesis.
2) Intertextuality: How can we approach a concept like intertextuality that was
used to destabilise the interpretation of text (Barthes 1977; Allen 2000) through a cognitive perspective? How does intertextuality work in audiovisual narrative texts? In this chapter I continue to build on the concepts of chapter one but focusing
specifically on what they can mean for researching intertextuality.
3) Mapping: At this point I will have discussed where research on mapping and
narrative meet and where cognitive and cultural theories to intertextuality can be expanded. This chapter therefore focuses on two central questions: How mapping can improve not only the analytical aspects of intertextual research, but also the communication of those results? How can abstract or theoretical concepts, as applied to time-based narratives, be communicated to audiences – both academic and non-academic? In order to answer these questions, I will end this chapter with strategies, based on the conclusions of the chapters on narrative, intertextuality, and mapping.
4) American Gods: Within American Gods, characters are personifications of
people’s beliefs, and their identities are formed by culturally transferred knowledge and beliefs. The relevance of the series for this thesis is how its characters are shaped by stories and they are aware of this. This concept sets up a lot of potential intertextual referencing for the audience, so in this chapter I analyse various aspects from the first season to examine this. How does the series incorporate intertextual cues in its overall narrative structure? How does it create and employ audiovisual intertextual cues? How do its characters reflect the effects of (enforced) socio-cul-tural changes and the power of narrative? Based on these questions and different aspects of the TV-series I will explore how the audience can engage with this story and how these insights can be mapped and communicated in different forms. The aim of this chapter is to explore potentials and use this as a basis for future lines of research.
Conclusions
To summarise, in this thesis I will examine the potential of narrative mapping for researching audiovisual intertextuality. In order to do so, I will focus on narrative, intertextuality and mapping from a cognitive and cultural perspective, and contex-tualize theory with contemporary mapping practices. My aim is to both extend on current approaches to audiovisual intertextuality, and to explore future strategies and meeting grounds for theory and practice. To get there I will critically analyse, argue for, and map out these theories as I apply to existing projects and stories. Although the proverb is usually show, don’t tell, this is my show and tell about
“Real people have embodied minds whose conceptual systems arise from,
are shaped by, and are given meaning through living human bodies.”
(Lakoff & Johnson, Philosophy in the Flesh 6)
Chapter 1
1.1.
Narrative before mapping
How do we engage with stories, as audience? How do different elements and dimensions of narrative contribute to the narrative experience? In order to start understanding audiovisual narratives, we often use cues in the text to construct worlds in our minds, to make connections based on the information we are presented with, and to use what we know to infer what we do not know (yet). As such, this first section discusses different dimensions of storytelling, in order to see how they contribute to our overall understanding of audiovisual narratives. Within the scope of this thesis I will concentrate on narrative mapping with regards to intertextuality in audiovisual narratives. In order to examine narrative mapping, the first question is: how do we approach narrative before mapping?
1.1.1 Textual Markers for constructing a diegetic world
Narrative structures can be difficult to get a grip on for a myriad of reasons. For example, the complete Lord of the Rings film trilogy (directed by Peter Jackson, 2001, 2002, 2003) runs around 11-12 hours1; the book House of Leaves (Mark Z.
Danielewski, 2000) has at least three or four different narrators discussing impos-sible spaces; the TV-series Westworld (created by Jonathan Nolan and Lisa Joy, 2016-present) continually undermines the events shown by going back and forth in timelines and providing alternative perspectives. These stories are being mapped by their franchises as well as all over the internet (see Figure1 above). One of the reasons people map stories like these, both for entertainment and research, is that they want to know how do they work? What is happening when, and with whom? What stories like these have in common is that they provide a lot of information to
1. Readers of the book
trilogy will also deal with a narrative of epic length, same as the filmviewers.
Figure 1
Map of character movements over the complete film trilogy of Lord of the Rings, produced by XKCD
One of the functions of narrative mapping is that one can create an overview of what we/the audience perceives as the diegetic world (the world in which the story happens) or how other elements are positioned in that world (like character movements or geographical places). An effective way to get an overview of how characters intersect or diverge through the diegetic world can be to single out a specific element of the overall narrative and map that, as can be seen in Figure 1 (page 7). I find that most narrative mapping of the diegetic world starts with textual markers. Cinematic time and space is created through text-based spatial and temporal markers in the film text which demarcate the diegetic boundaries of the presented.
My interest within this thesis is in audiovisual narratives where such textual markers complicate our mental construct of the narrative text: when a bird does start talking, when people time travel, when the rules of the story are broken and the audience needs to rewrite them in their own mental models. I would argue that mapping the structure of narratives such as these can provide a frame that enhances our understanding of how the world is constructed. This frame allows us to analyse how the story provides cues and how the audience handles them. What needs to be questioned is how the text inhibits or enables the audience’s under-standing of events, and how far this can be challenged. This is why in the case study of the fourth chapter, American Gods (2017-present), I will delve deeper into how the audience can construct a cohesive mental model of stories that complicate usual markers for time and space through a variety of means2.
Textual markers influence how we interpret both what is told and how it is told. That is why I find it useful to see how narrative mapping comes into play, to analyse (and synthesize) the different levels of storytelling which also determine how the audience can engage with a text.
2. For example, one of
the complicating factors of American Gods is that the series presents partial or fragmented stories set in multiple time periods and spaces, and leaves it to the viewer to make connections to the main storylines.
1.1.2 Delineating Fabula and Syuzhet
In The Wrong House: The Architecture of Alfred Hitchcock (2007), Steven Jacobs aligns impossible cinematic spaces in blueprints. However, reconstructing textual markers is not the only function of mapping. As Hitchcock’s films are constructed to provide comprehensible narrative texts (if unnerving or impossible at times), the audience will (by default) overlook gaps such as misaligned floors or non-existent doors3. This is not the case with impossible spaces in a story like House of Leaves or twists in Over the Garden Wall (created by Patrick McHale, 2014). The difficulty in stories like House of Leaves or Over the Garden Wall lies not just in how the time and space of the diegetic world are structured; the main factor that complicates the audience’s understanding is how the structure is told. Textual markers for cinematic time and space are a part of the meaning-making processes, as they set the audience up for creating a spatio-temporal framework through which other cues can be interpreted. But how textual elements are configured also influences the audience’s interpretation of them. In this section I will focus on how the actual form of telling (syuzhet) shapes the audience’s constructions of the told (fabula), and specifically how this process is influenced by earlier narrative experi-ences.
Narrative is not just text, but the activity of creating a mental construct of our understanding of the text. The syuzhet structures the audience’s attention by either enabling or hindering the creation of such mental constructs. How information is presented allows the audience to act on or dismiss elements like character reactions or spatial/temporal inconsistencies.
As audience, our experience of narrative is based on both the processing of cues and drawing on pre-existing knowledge. In order to get a grip on “how the world works” we use smaller cues to activate scripts that set up our expectations, as I will discuss more in section 1.2. In order to make sense of the story, we draw upon what we already know.
By mapping narrative texts that have complicated structures, we can get a better understanding of how and where the text (mis-)leads the audience in a
3. Unless they are made
aware of it, for reasons relating to story events for example.
Leaves, for example, presents information on different and contradicting layers, simultaneously. Westworld combines jumping back and forth in timelines with characters that have unreliable memories. These are two narratives, one written and one audiovisual, that purposefully complicate the audience’s understanding by giving an information-overload of cues.
Maps can present a restructured overview of the syuzhet, making it easier to analyse how the audience is enabled to deal with individual cues. Mapping out the cues means we can order and prioritize information within the narrative experience. Our experience of a narrative does not necessarily focus on every textual element separately, but on how most of them can be effi ciently categorized together in order for us to construct a cohesive fabula. When something doesn’t fi t by itself, one makes it fi t: the audience can discard that element (at fi rst), adjust their expectations, or quit and disengage from the text.
Creating an information overload in the syuzhet is one way of complicating an audience’s understanding. Another method with a similar effect was used in narratives like Over the Garden Wall and American Gods, which depend on the audience picking up on certain connotations; either they rely on associative set-ups to effectively execute a plot twist (Westworld, Over the Garden Wall) or to create a cohesive mental map of connecting elements (American Gods). The associative set-up allows for leeway in what we accept as a part of the story (see Figure 2): if it looks like a fairy tale, sounds like a fairy tale, is structured like a fairy tale quest, then why would we question a talking bird or a character wearing a teapot for a hat? Initially, we wouldn’t because we were cued for framing the narrative through a fairy tale lens.
Figure 2 Still from Over the
Earlier experiences with fairy tales have created a knowledge structure, like a script, which the audience can use to fill in the gaps left open by the narrative. The narrative can not only rely on that process, but also exploit it, adding a different perspective for those gaps at a later point, resulting in the plot twist.
Mapping can enable us to understand the conflict we experience (and, arguably, want to resolve) between the fabula and the syuzhet because we analyse and synthesize information on multiple levels of storytelling. So far, I have discussed the possibilities of mapping textual elements of narrative which allows for a different understanding of how the text is structured, and how that structure can affect the audience’s meaning making processes. In the next section the emphasis is on visualizing that meaning making process, instead of the text that ‘merely’ sets in motion that process.
1.1.3 Visualizing cognitive processes in mental and
narrative spaces
One of the key concepts in this thesis is that of narrative spaces. Narrative spaces are, as Barbara Dancygier defines them, “cognitive domains active or set up in the reader’s mind by the use of linguistic forms, for the purposes of on-line story construction.” (Dancygier 137). Where the concept of mental spaces can encompass general meaning making, narrative spaces apply specifically to how meaning making processes enable story-construction, and thus inform the narrative experience.
This section focuses on the relation between mapping and narrative spaces in general, while the next chapter will focus on the relation between intertextuality and narrative spaces.4 In order to trigger the set-up of narrative spaces in viewers,
a text provides cues. So far, I have discussed the relevance of such cues in the form of temporal/spatial markers in the text and how the configuration of the text can challenge the audience’s narrative experience. In this section, I will focus on narrative spaces and how mapping narratives can visualize the way people understand a story. How we conceptualize and (re)position ourselves in relation to narrative spaces is a mental process that can be made explicit through mapping.
In her article ‘Cognitive Maps and the Construction of Narrative Space’ (2003), Marie-Laure Ryan discusses a study on mapping she did with a small test-group. The group was presented with a story and asked to draw maps that represented the space and events happening within the story space. Ryan shows that in these situations, maps are not visual transcriptions of the text, but an artic-ulation of our understanding of the text and its narrative spaces. The aim of her study was to see how subjects would map textual elements (spatial and temporal), but the outcome was that they actually articulate their narrative experience5 or as Ryan states: “the reader’s reading” (228).
Ryan’s study gives a starting point for as to why the concepts of mental spaces6 and narrative spaces are relevant in combination with narrative mapping. The act of mapping allows us to understand and visually communicate (to a later self or to others) how specific parts of a story connect or overlap, both within one text or across texts.
4. Within this thesis I
am mostly interested in mapping intertextual narratives, as the audience has to rely on their ability (and willingness) to connect their current narrative experience with prior knowledge whilst watching or reading. For example, Rosencrantz and Guildenstern Are Dead is dependent on the audience’s familiarity with Hamlet, as they have to embed the narrative space of the first within the narrative space of the second. American Gods is grounded in multiple world mythologies that form the basis for most of its characters.
5. Still shaped in part by
those textual markers.
6. Gilles Fauconnier,
cognitive linguist defines mental spaces as “distinct from linguistic structures but built up in any discourse according to guidelines provide by the linguistic expressions.” (Mental Spaces 16)
Max Van Duijn stated in ‘The lazy mindreader: a humanities perspective on mindreading and multiple-order intentionality’ (2016) that we are so-called ‘lazy mindreaders’. We are primed to understand the overlapping mental spaces in situations where someone says something like “he said that she thought that they would know that she would not be aware of that thing that happened that we all heard about”. The hard part is communicating that same understanding on a more abstract level where it is removed from its original narrative context. The visual advantage of a diagram like Figure 3 is that the spatial relations can clarify the connections between the different narrative spaces as the reader constructs them based on a single narrative text or across texts.7
7. This effect coincides
with Barbara Tversky description that a specifi c aspect of visual communication is “using elements, marks on a page, virtual or actual, and spatial relations, proximity and place on a page, to convey literal and metaphoric elements and relations.” (“Visualizing Thought”, 502).
Figure 3
Diagram from Lazy Mindreader, an example of reader position and multiple embedded spaces in narrative texts.
In this thesis I want to extend on this observation regarding visual-spatial aspects of mapping with the idea that narrative spaces are constructed over time. We use these constructs while we read a book, or watch a movie, or play videogames. As audience, be it a reader, viewer, or gamer, the construction of narrative spaces is a dynamic process that is happening while we engage with narrative texts. Information is added over time and we continually need to adjust our map to allow for new input. By examining this process of mapping I want to question if we can represent the dynamic nature of this process more accurately.
So far I have argued how space and time feature in mapping practices and narrative spaces. My aim for this thesis is to visualize how the audience constructs narrative spaces in texts that depend on the audience’s ability to pick up on inter-textual cues. How does our experience with other stories influence how we engage with a narrative text that (on different levels) refers to those stories?
1.1.4 Conclusions for mapping narrative dimensions
In this section, so far I have focused on three different narrative dimensions (out of possibly many) where narratological concepts can and have intersected with mapping. With regards to intertextual theories, these concepts are all relevant for mapping where the text has markers for the audience (1.1.1), how the structure affects how audience engages with the text (1.1.2) and how referencing one text can influence the mental models of another text (1.1.3). When researching the intertextual viewing experience, mapping would be useful for analysing how these varying narrative dimensions intersect with each other and the overall narrative experience. Mapping allows for lifting elements out of their previous context and
synthesizing them, and as such presenting insights from research in a different –
augmented – form. Mapping, as visual communication, has the potential to open up new pathways for exploring how intertextual frameworks can influence the narrative experience.
Visual communication has been a part of how we experience the world and represent that experience for thousands of years. In her article on visualizing thought, Barbara Tversky states that a specific aspect of visual communication is “using elements, marks on a page, virtual or actual, and spatial relations, proximity and place on a page, to convey literal and metaphoric elements and relations” (“Visualizing Thought” 502). Embodied cognitive processes have a significant part to play in the development of visual communication. Section 1.2 will therefore mostly focus on an embodied cognitive approach to unfold the production and analysis of visual communication and narrative research. There, I will illustrate
how (to a certain degree) concepts like image schemas inform how we tend to
structure (visual) elements in a composition, or categorize them when interpreting such a composition. However, I do not want to underestimate the importance of
film viewing as a social phenomenon, and one that is embedded within cultural
processes as well. Therefore, the next section will also address how these cognitive processes are embedded within socio-cultural systems.
1.2. Narrative mapping; forms of knowledge
In ‘Philosophy in the Flesh’, Lakoff and Johnson start off by saying that “the same neural and cognitive mechanisms that allow us to perceive and move around also create our conceptual systems and modes of reasons.” (4). They continue by claiming that “[r]eal people have embodied minds whose conceptual systems arise from, are shaped by, and are given meaning through living human bodies.” (6). Their insights relate to this thesis as my focus is on the cognitive processes that are shaped by ‘living human bodies’, and how these intersect with the social realities that people experience. In section 1.2, I will focus on the intersection of embodied-cognitive approaches and lived experience. My aim is to demonstrate how we employ different forms of knowledge in order to engage with and articulate our understanding of audiovisual narratives.
1.2.1 Scripts for how the world works
Wednesday: So, you’re perfectly okay believing that tiny people on television can predict the weather. But you crinkle with consternation at the mere suggestion that you could make it snow.
Shadow: One of those things is science, okay? The other is fantasy. Wednesday: You’re talking about it like it’s apples and oranges. Shadow: It’s not apples and oranges, okay? It’s reality and fantasy. Wednesday: Oh, so that’s how the world works! It’s either real or it’s fantasy? Shadow: Yeah, that’s how the world works.
(American Gods, season 1 episode 3: “Let There Be Snow”)
We connect and group information together in what can most broadly be defined as knowledge structures (Herman 1047).Schemas, scripts and frames are elementary ways to conceptualize knowledge structures. They explain how we mentally group together (new) information based on past experience and why we tend to have certain expectations regarding the information we are yet to receive.8
What I will start with here is outlining how we efficiently assess narrative elements in a text based on information we already have with the use of scripts and schemas, through the example of Over the Garden Wall. This in order to examine how
8. On example of this was in the earlier discussion on syuzhet and Over the Garden Wall. Here I already referenced how stories can be structured to divert the audience’s attention and expectations in a specific
scripts are constructed by a combination of cognitive and cultural elements, and how this affects intertextual referencing.
In our experience of narrative, we do not necessarily focus on every textual cue separately, but instead employ efficiency-based processes that allow us to construct a cohesive mental model.9 In ‘Twist Blindness; The Role of Primacy, Priming, Schemas, and Reconstructive Memory in a First-Time Viewing of the Sixth Sense’, Daniel Barratt, a cognitive film theorist, suggests the following about the relation between memory and fabula:
In the way that we construct the fabula by employing certain schemas, we
reconstruct our memories of the fabula by employing those very same
schemas once again; both the syuzhet and the style (the precise content and the precise form) drop out of the equation. (65)
This means that we do not just use schemas to perceive and assess elements in the story when we encounter them, but we also use them when we have to remember something that happened earlier in the story. Take, for example, Over the Garden Wall. In the section on syuzhet and fabula (1.1.2), I used Over the Garden Wall because for the most part it looks, sounds, and acts like a fairy tale while subverting fairy tale tropes. The format of the series encourages a fairy tale categorization, but at the same time there is one character, Wirt, who reacts surprised to every fairy tale element and continually refers to details that do not fit with a fairy tale script. Everyone else ignores it when he does this, it is played for laughs every time and has little to no bearing on the general plot of the episodes. While Wirt does contradict the fairy tale aspects, his credibility is so low that this does not affect our interpretations. For example:
Wirt: “No, Greg, a bird’s brain isn’t big enough for cognizant speech.” Bird: “Hey, what was that?”
Wirt’s observations, as in this exchange, would make sense in an everyday context, but are overwritten by the reality of talking birds and other fairy tale
9. Daniel Barratt, for
example, focusses on attention decrement and conservative cognition in relation to plot twists. According to him “… we are usually more “discerning” at the beginning of a film (or a film sequence) than at the end.” (Twist Blindness 68).
sions and go for the most obvious interpretations of events (by using our schematic knowledge).” (68).
In this story, contradictory information is usually not absent in a situation, but its value is purposefully minimized. Logically, the audience has no reason to pay more attention to this minor aspect (Wirt’s rational mind) than to the monster of the week. In our construction of the fabula we therefore fail to include this friction and use a fairy tale schema to guide our expectations. Following Barratt’s reasoning, in our memory of the fabula that we constructed earlier, we use that same fairy tale schema.
In order to apply a fairy tale schema to Over the Garden Wall, the audience needs to have experience with fairy tales, in order to have a script for how the
fairy tale works. Roger Schank, cognitive psychologist, and Robert P. Abelson,
psychologist, argue in ‘Knowledge and Memory: The Real Story’ that knowledge
is selectively structured through scripts. Viewers construct scripts over time, using their past experience. These scripts are flexible, to a degree. Tatyana Karpen-ko-Seccombe builds on this hypothesis and connects it to intertextuality, as she claims that “...intertextual references are decoded by mapping the structure of the situation, event, or scene... of the source text onto the target text whose situations have an analogical structure” (Intertextuality as cognitive modelling, 15). This means that we, as viewers or readers, take selective cues from the narrative text which we connect to past texts or knowledge by applying our previously
estab-lished scripts and schemas.10 To use Karpenko-Seccombe’s argument, part of the
intertextual narrative experience is finding the ‘analogical structures’ (15) that allow us as reader/viewer to map our pre-existing knowledge onto the text at hand.
Similar to the dyadic process of mapping, narrative is “simultaneously being produced and consumed” (Kitchin et al. 17) by an audience. Within the context of this thesis, that applies to those knowledge structures, like scripts and schemas, that we use, which are largely formed by our experience. As such, that process of shaping and being shaped, of producing and consuming, is a culturally embedded cognitive process. Our pre-existing knowledge (gathered from various sources) comes into play for interpreting cues during the viewing/reading experience. In this thesis, intertextuality is viewed as following that pattern and thus as a process of linking different cues to pre-existing knowledge. The cognitive processes that 10. For example, when a
text introduces a talking bird character, this is (usually more of) a cue for us to think of fairy tales with talking birds, as opposed to nature documentaries about bird wildlife. In turn, in the process of constructing the narrative, ‘talking bird’ combined with a narrator that says “once upon a time” and a main character with a cape can get the audience to link to the script ‘fairy tale’. The fairy tale script then allows for the predictable (magic forest,
quest-narrative, singing frogs) or minor variety (a deconstructive fairy tale or the singing frog uses a boombox) to be easily incorporated. At the same time, we tend to smooth over minor dissonant elements (singing frog uses a
enable intertextuality intersect with cultural ones as viewers construct something that is both shaped by and shaping the network of relations and knowledge they have.
1.2.2 Communicating embodied knowledge
This section is focused on how embodied image schemas, as developed by Lakoff and Johnson, form one of the cornerstones for our verbal and visual communication – in general, but also specifically in narrative mapping. n daily conversation, we rely on image schemas to construct and understand metaphors for explaining or describing something. In visual communication, we use them to intuit how aspects relate to each other differently in different diagrams, for example. As I will show, maps and diagrams can be analysed as a visual representation of the embodied-cognitive processes that structure the knowledge which we want to communicate. In the following section I will demonstrate how image schemas, like the source-path-goal schema and the container schema, can be and have been a part of how we understand aspects of narrative, and visually communicate that understanding.
Our ‘living human bodies’ (Philosophy in the Flesh 6) provide the basis for embodied-cognitive theories used for analysing how we set up ‘linguistic forms’ (Dancygier 136), conceptual metaphors or cues in our communication. These schemas are based on proprioceptive knowledge; on our understanding of our embodied existence within the world with specific affordances, in the most basic sense. While we can use these schemas intuitively in daily life to communicate, we can also use them for analysing narrative structures and how we experience them. A source-path-goal schema, for example, is useful for narrative analyses because it is so universal that it is easy to grasp. For that reason, the schema is just as useful for explaining how our understanding of narrative is structured as for explaining how our understanding of narrative can be disrupted when one of these elements is missing. Arthouse films, for example, tend to complicate things in part by removing the source or goal, while ‘mainstream’ cinema does follow a causal structure. Similarly, our expectations can be subverted because we had the wrong source or the wrong goal in mind when we start to construct the fabula. For example, in the first season of American Gods we are seemingly presented with a source (conflict between New and Old Gods), a path (the events along the way) and a goal (resolving the conflict).
When the intention is to use mapping as a method to visually capture complex or abstract aspects of our narrative experience in order to get a grip on and communicate them (for, for example, scholarly or creative purposes), we generally try to do that as efficiently as possible. Therefore, when it comes to mapping, image schemas are useful for constructing visual representations of the mental models of narrative. On a basic level this can, for example, mean mapping out a timeline. A slightly more complex variant could be mapping out how a timeline and diegetic space relate to each other. On an abstract level, this can mean mapping out narrative or mental spaces, or multiple layers of metalepsis. Academic research on narrative has often used diagrams as visual support to compliment a text-based argument, and has done so most effectively by modelling them after image schemas. An example of this are the diagrams Max Van Duijn uses for explaining how readers position themselves in relation to multiple layers of narrative spaces and in reading intentionality; or the diagrams used by Miklós Kiss and Steven Willemsen to explain the different diegetic layers that exist within one story (see figure 4 and 5 on the right). Both depend on the container schema for visualizing how the different elements relate. In the chapter on mapping (3), I will come back to this in order to show how information design can utilize image schemas for narrative mapping.
EXTRADIEGETIC LEVEL
intradiegetic level (the fictional world) hypodiegetic level
hypo-hypodiegetic level hypo-...diegetic level SV-Space Omniscient narrator
MNS
Focaliser Omniscient narrator
writing the letter to the New York Times at Lady Bruton’s house
NS1 Focaliser: Hugh
pen being in service for 20 years (indirect discourse)
NS2 Focaliser: Hugh pst event: showing pen to its makers (free indirect discourse)
NS3 S: pen’s makers no reason why pen should wear out (indirect discourse)
NS4 Focaliser: Richard
feeling that pen’s qualities are to Hugh’s credit (indirect thought)
NS5
Focaliser: Lady Bruton feeling that Hugh’s grammar will impress Times editors (indirect thought)
NS6
Focaliser: Lady Bruton imagined future event: Times editors respecting letter (indirect thought)
Participant: Hugh telling about his pen Participant: Richard picking up on Hugh’s words about the pen. Participant: Lady Bruton watching Hugh write
SV-space = Story-viewpoint space MN-space = Main narrative space NS = Narrative Space
Figure 4
Redrawn diagram from Lazy Mindreader. A Humanities Perspective on Multiple-Order Intentionality (Van Duijn). An example of reader position and multiple embedded spaces in narrative texts.
Figure 5
Redrawn diagram from Impossible Puzzle Films; A cognitive approach to Contemporary Complex Cinema (Kiss & Willemsen 42). Visualizing how plotlines can be embedded in complex structures, resulting in stories within stories.
1.2.3
Articulating knowledge through blending
When we pick up a book, we do judge by its cover, and when we watch a film, there is usually a selection process beforehand. That process is socially struc-tured, both in direct ways (someone we know recommended it; someone picked it for a movie night) and indirect ways (it is all that is playing at the cinema; it cannot be accessed through Netflix; it is censored). On a social level, I argue that we often place a narrative text in a bigger network when we discuss it with others. We blend our own experience with that of the other by focusing on aspects that we can connect, so we will say “It’s like Harry Potter meets Law and Order” or “It’s like Star Wars, but with fewer lasers and more space pirates”.
So far I have discussed the role of knowledge structures like scripts and image schemas for our understanding of narrative, and how our embodied and cultural knowledge shapes those structures. In this third part I will discuss Blending Theory and mental spaces as analytical tools for working out how we use stories and aspects that we are already familiar with and what we think someone else will be familiar with to communicate.
As Barbara Tversky states: “…to be understood by another or by self at another time entails fashioning communications to fit the presumed mental states of others or of one’s self at another time.” (“Visualizing Thought” 500). We need to estimate the common ground where our reference and another person’s reference are similar enough to communicate: a mental space that “we set up as we talk or listen, and that we structure with elements, roles, strategies and relations.” (Fauco-nnier 1).
What is important here is that mental spaces, similar to what I argued with narrative spaces (1.1.3), are not fixed constructs from beginning to end, but “… constructed and revised in people’s minds in response to linguistic and other prompts.” (Tobin 352). So, over the course of holding a conversation, watching a film, or reading a book, we continually construct and reconstruct mental and narrative spaces. With regards to intertextuality this is interesting, because inter-textuality can require us to draw on and incorporate different source domains to create an intertextual frame of reference.
The difficulty in setting up these spaces in communication is parodied in the TV-series Psych, as the main character’s reliance on pop culture references work only until he gets to a remote town:
Shawn: What is it with all the secret relationships in this town? It’s like General Hospital.
[confused stares]
You’re kidding me. All right, that’s it. Is there a film, or a television series that you people actually saw?
Randy: The town gets together every Thursday night to watch reruns of Everwood.
Shawn: Okay, I can work with that. Later in the conversation
Randy: Look, I’m telling you guys the truth, plus I have an alibi, remember? Shawn: All right, but we’re on to you, Randy. Just like the townspeople of
Everwood were on to the fact that Nina was a surrogate mother. Gus: Will you stop it?
(Psych, season 5 episode 12: “Dual Spires”)
This level of blending personal knowledge with someone else’s knowledge for minor referencing is usually less enforced (and more part of the conversation) than in this scene.11 According to Barbara Dancygier and Eve Sweetser, a blend
can be described as “an emergent conceptual construct, resulting from integration of other (often already complex) construct and serving new meaning-construction needs.” (Figurative Language 76). As shown in Figure 6 below, they argue that a blend emerges from a source domain being mapped onto a target domain (termed input spaces), while constrained by an aspectual, causal and/or scalar structure (termed generic space).
11. Of course the
misconnecting in cultural references is purposefully played for laughs here.
generic space
aspectual, causal, scalar structure (acts as constraint)
input 1
(source) (target)input 2 Figure 6
Diagram based on an example of a blend in Figurative Language
In my opinion, blending theory has potential for examining how we under-stand and articulate our underunder-standing of intertextuality in audio-visual narratives. With regards to understanding intertextuality, blending theory is interesting because it provides a model for how the audience can map the familiar unto the unfamiliar and in doing so creates an intertextual framework. This framework or blend is then distinct from both source and target, and its construction is constrained by following specific structures. For example, in American Gods a source would be the audience’s knowledge of various world mythologies, with the characters in the story as target. The audience blends what they know with the presented infor-mation, which functions as a framework in which future “elements, roles, strategies and relations” (Fauconnier 1) can be situated.
This thesis focuses on how the audience deals with intertextuality in audio-visual narratives, which means that it examines when and how the audience is cued for intertextuality and how this can affect their overall narrative experience. So far I have discussed concepts like knowledge structures, image schemas, narrative spaces and blending theory. These concepts, as I also argued earlier, pertain to culturally embedded cognitive processes that we use for meaning construction. Although I have argued for the advantages of such an approach for researching intertextuality, there are, from a cognitive side, two potential issues. One issue is the assumption that we always pick up on intertextual cues and that the narrative text wants us to. The problem here is that intertextual cues can be foregrounded/ backgrounded and not all of them affect the audience in the same degree. This is further discussed in the next chapter on intertextuality and demonstrated in the case study analysis of Media (4.4.1).
The second issue is with employing theories as the ones discussed so far for analysing how we articulate knowledge about intertextuality. The problem is the assumption that we use the same multimodal sources, like personal experience and earlier narrative experience, to engage with a narrative text as well as commu-nicate about said text with others. The issue here is finding a method to actually analyse if and how we apply those in both our understanding and articulation
of that understanding of narrative. One approach could be Cognitive Discourse
Analysis (CODA), which in this context can be used to analyse how people articulate their understanding of a narrative text. It provides a possibility of analysing and mapping the construction of mental spaces in discourses about (the experience of
intertextuality in) narrative.
So far my argument has focused on the function of cognitive and cultural elements in meaning-making processes; theorizing on our experience of ‘how the world works’. To augment this, the fourth section is a critical note on the (re) production of cultural knowledge that influences our narrative experience.
1.2.4
A critical note on cultural knowledge
For an audience engaging with a narrative text, maps can be a tool for understanding and articulating that understanding. In this sense the map becomes an object for analysis when it can be used to articulate the intertextual knowledge which enables/enhances/changes that understanding. When it comes to using mapping for research, it is both a tool to analyse and visualize data (as the next section shows) or to communicate abstract concepts/mental processes such as narrative spaces.
So far, I expanded on some of the general cognitive processes involved in narrative mapping and how experience plays a role in this.However, I also want to emphasize how, through mapping, we maintain or destabilize institutionally determined cultural images. As David Mclagan puts it, “cultural images... don’t just document this space but are also incorporated into the idiom of our fantasy, so that they inform or construct it.” (153). A map, both in general and in narrative mapping, is a visual representation of elements that we use or think of as important, and therefore maintain (by default inclusion) or deconstruct. While I have argued for the effectiveness of image schemas in mapping, I want to include tradition as an important influence as well. We base our own map on how we think something should look like, and we base both the graphic elements and how they are struc-tured on past maps as well. Land is green, rivers are blue lines, or houses are pictured as squares with a triangle on top, for example.
The systematic reproduction of specific cultural images influences how an audience will engage with a text. Therefore, it also influences their mental model of narrative,12 and their maps as articulations of those mental models. In that regard,
the map is a representation of formalized knowledge. Critical cartography offers a point of view on this, stating that “…the stories of mapping always need to be considered as historically contingent actor-networks; as timed, place, cultured and negotiated; a Web of interacting possibilities in which the world is complex and nothing is inevitable.” (Kitchin et al. 16). This is why it is necessary to discuss mapping within the context of the intersection of cognitive and cultural approaches to narrative. 12. “Our evaluation of realism in a given representation is based on a characterization of the representations, some concept of what the real world looks like, and a judgement as to the relationship between the given representation and that concept of the real world.” (Grodal 251)
1.3. Narrative after mapping
For research on both how narrative texts function and how the audience engages with them, narrative mapping can provide useful insights. By analysing and synthesizing textual markers, narrative maps can reveal a blueprint for a diegetic world, a cinematic space and time that only exists in the audience’s mental construct of the narrative text. In the previous sections, I focused on how narrative dimensions emerge ‘before’ mapping and how cognitive and narrative approaches in communication can be framed through mapping. Mapping can reveal how tools – such as camera movement, set design, and editing – are used for (making us) constructing the film text, as these create textual markers which cue the audience to mentally construct the diegetic world.
In this section, I want to conclude with narrative analysis ‘after’ mapping – how mapping is used as a tool within narratological research and how such usage affects research. How has narrative theory been applied in large scale narrative mapping projects? How have humanities scholars approached mapping?
1.3.1. Reviewing narrative approaches in mapping
projects
In this first part, I will focus on how existing narrative mapping projects were set up to answer questions raised through narrative research. For this I will discuss four projects: the first two are produced by individual researchers, while the last two are set-up on a larger and more collaborative level. They vary from an audio-visual essay where the use Steadicam is analysed (Kevin B. Lee’s The Career of Paul Thomas Anderson in Five Shots, 2015) to a project like Cinemetrics, which is set up as a database for large corpus systematic research.
As the examples in this section will demonstrate, narrative mapping requires the reduction, transformation and reconnection of elements, which can be detri-mental to such attempts’ communication potential. Here I will examine both the application of narratological theory in existing mapping projects, and the resulting pros and cons of such projects for narrative research.
The Wrong House: The architecture of Alfred Hitchcock Blueprints, stills and written texts
Book 2007
Steven Jacobs
A book project that focuses on spatial markers in cinematic spaces of fi lm directed by Alfred Hitchcock.
The Career of Paul Thomas Anderson in Five Shots Audiovisual essay
Sight & Sound magazine 2012
Kevin B. Lee
Audiovisual essay consisting of fi lm scenes, animated maps and voice-over.
In The Wrong House Steven Jacobs, an art historian, maps the textual
markers in Hitchcock’s fi lms and then analyses how spatial triggers form other signifi cant aspects of fi lm and consequently the viewing experience. Jacobs uses visual cues from the fi lms to map physical spaces in the diegetic world. Where there are ‘impossible’ fi lmic spaces or gaps in the architecture, Jacobs draws in the blanks, creating ‘possible’ blueprints. One of the examples Jacobs gives of the impossible:
In order to roll a camera back and forth in a four-room penthouse for ten minutes without a halt, Hitchcock needed what he called a “collapsible” apartment. The basic element was a series of wild walls hung from heavily greased overhead U-tracks so that the grips could pull them silently out of the camera’s way as it followed the actors through doors, then roll them back into position before they came back into camera range. (274)
These blueprints (see Figure 7) and stills from the fi lm texts form the basis for his analysis of how the audience is cued for interpreting the fi lm text. Jacobs does this by looking at the production of spatial markers in the text through the lens of an architect. He analyses sets (connected to the architecture they were based on) and camera framing/editing (aligning otherwise impossible physical spaces).13
13. As Jacobs
explains “…a careful reconstruction of [the Bunting House] fl oor plan reveals several spatial inconsistencies. The fl oor level of the lodger’s room, for instance, is unclear since sometimes he climbs only one, on the occasions two fl ights of stars. In addition, the Bunting’s living room is situated on the wrong side of the axis formed by the front door and stairwell. As a result, the direct vertical relation between the Bunting living room and the lodging… is completely Format: Context: Year: Maker: Description: Format: Context: Year: Maker: Description:
A challenge in mapping textual markers in fi lm texts such as Hitchcock’s is that fi lm is a spatio-temporal medium. If through mapping we aim to both analyse and synthesize information, can we then map textual markers in an audiovisual and spatio-temporal manner?
Figure 7 From stills to blueprints
in The Wrong House.
This last issue raised in the discussion of Jacobs’ The Wrong House is addressed by Kevin B. Lee in his audiovisual essay The Career of Paul Thomas Anderson in Five Shots (see Figure 8 on the next page). Here, Lee shows a way of integrating analytical mapping and audiovisual text. A simultaneous presentation of time-based text and analysis gives an opportunity to show
when and how audience gets (audio)visual
infor-mation over the course of the story, in order to analyse how this affects their understanding. He combines specifi c scenes (that were fi lmed with a Steadicam) with an animated map tracking the
Figure 8 Stills fromThe Career of
Paul Thomas Anderson (Kevin B. Lee)
Lee reflects on this essay, saying:
It was at this point that I took a more active interest in thinking about how video essays could show the unseen hands at work behind every frame, and less about how the cinematic frame presents itself as a finished work of art (which remains the primary concern of most video essay work). (emphasis added)
Narrative mapping is one way of visualizing those ‘unseen hands at work’ that shape the cues for how the audience interpret the film as a whole. Mapping can enable us to see the construct of cinematic time and space. When incorporated in audiovisual essays we can see both the mapped construct and the (fragmented) film text remixed as one cohesive text (the essay).
In these examples the focus is on how spatial and temporal markers shape the diegetic world, and how elements (be it characters or Steadicams) move through this world. In its most basic form the audience interprets such textual markers to form a diegetic world bound by time (21st century) and place (United States) and
a set of rules for that world (people can’t fly and birds don’t talk). These textual markers contribute to our overall narrative experience by enforcing a structure.
A Literary Atlas of Europe
Data models and research articles literaryatlas.eu as main platform 2006-present
Barbara Piatti et al.
Collaboration between literary scholars and cartographers mapping geographi-markers in literary texts set in Europe.
Cinemetrics
Database and software (public access), research articles cinemetrics.lv as platform
2005-present
Yuri Tsivian (idea, maintenance)
Software and database for measuring average shotlength and cutting rates in audiovisual texts. Format: Context: Year: Maker: Description: Format: Context: Year: Maker: Description:
The driving force in the A Literary Atlas of Europe project is investigating the connection between different environments and their representation in fi ctional spaces. A Literary Atlas of Europe is focused on answering various questions, such as:
Where is literature set, and for what reason is it set there? How does it, over the course of multiple historical periods, use, modify, alienate or re-model existing geographical spaces? And how are such fi ctionalized landscapes
and cities represented and understood?14
Taking their cue from literary scholar Franco Moretti, the research team has taken an interdisciplinary approach to spatial theory, both in the process of collecting and visualizing data from texts, resulting in maps like Figure 9 (below). The resulting insights which were then integrated in research articles.
A Literary Atlas of Europe claims that “[m]aps […] are never fi nal results but tools of interpretation and sources of inspiration.”The project has an interdis-ciplinary structure in its workfl ow where literary theorists and cartographers have different tasks at different points at the project. For example, literary theorists are “responsible for the overall conceptual structure”. They set up categories, select and interpret texts. The cartographers are responsible for “all technical issues”. This results in a structure where information is selected by one group, fi ltered through another group and then again interpreted by the fi rst group. The project takes an explorative approach to mapping and literary studies, allowing for multiple visualization and articles, with the connection between fi ctionalized spaces and the environments they draw on at the center.
14. Original text in
German: “Wo spielt Literatur und weshalb spielt sie dort? Wie nutzt, überformt, verfremdet oder re-modelliert sie – über mehrere Epochen – bestehende geographische Räume? Und wie lassen sich solche fi ktionalisierten Landschaften und Städte, darstellen und deuten?”
Figure 9
Two visuals by A. K. Reuschel, from the mapping practices by A Literary Atlas of Europe, as published in “Improved Density Estimation for the Visualisation of Literary Spaces” (Bär & Hurni 2011)
Cinemetrics, as developed by film historian Yuri Tsivian, provides open-access measuring tools (software), a database (the website), and connections to both informal (comment boxes and discussion board) and formal (published research) discussions on the subject and use of Cinemetrics.
Both A Literary Atlas of Europe and Cinemetrics are large scale, collabo-rative mapping projects. However, A Literary Atlas of Europe created a back-and-forth workflow with changing parameters for collecting data, methods for collecting and visualizing the data, and the interpretation of the information. Cinemetrics as a project, on the other hand, poses one main question: What can analysis of shot length in film add to what we know about film history and film production? Results from Cinemetrics can be used to identify patterns in how film editing has changed over time, allowing for further analysis of the historical conditions that enabled these changes. As Tsivian explains in his article ‘Cinemetrics, part of the Human-ities’ Cyberinfrastructure’ (2009):
We still do not know enough about this [the factors that make cutting rates change across film history], and it is this gap in our knowledge that Cinemetrics is designed to fill up. What we already know, however, allows us to link changes in cutting rates to various aspects of film history, including the history of film style, the history of film industry, film’s cultural history, and the history of cinema as technology. (94)
The very specific set-up of the project, the measuring shot length through changes (cutting rates), creates both advantages and limitations. Some of the named limitations by its designers and users are the quality of the data (Keith Brisson: “It is not necessarily completely precise, accurate, or complete”) and how can be applied (for example, what can the ASL tell us about tensions in the narrative structure?)15. This mapping project was developed by film historians, to
specifically add to film historical research. As such, the (current) uses of Cinemetrics are very specific: there is a certain amount of knowledge that film historians have about style, industry, technology, and the socio-historical contexts in which these developed. By identifying patterns Cinemetrics can potentially extend on existing knowledge. The project can also provide a springboard for similar approaches that measure and identify patterns in specific elements of audiovisual texts, such as
15. Side by Side:
Data Analysis Across Films, by Keith Brisson (Cinemetrics.lv)
