Charles Forceville: “Visual and multimodal metarepresentation of speech, thought, and sensory perception in comics.”

Comics draw on the visual and the verbal modality, making it a thoroughly multimodal medium. A central strand of comics research is partly or wholly inspired by cognitive linguistics and relevance theory (e.g. Forceville 2005, 2011, 2013, 2014, 2016; Forceville et al. 2010, Forceville and Clark 2014; Yus 2008, Kukkonen 2013, Cohn 2013). As in language, it is centrally important in comics to be able to attribute speech and thoughts as well as sensory perception to a specific character. Since this character lives only in an artificially created world, the various manifestations of “saying/thinking/perceiving that …” are always by definition metarepresented by a narrative agency responsible for this world-making. However, as in purely verbal situations, metarepresentations in comics can feature one or more levels of embedding. In this paper, a number of comics panels will be analysed in order to explore which visual resources play a role in such metarepresentations, and the degree to which these depend on interaction with the verbal mode. The findings will reveal that, and how, there are multimodal and purely visual equivalents for “thinking/perceiving that …” and even for “saying that …” The paper will benefit both the theorization of word & image discourses and help develop cognitive linguistics models, narratology, and relevance theory to better account for such multimodal discourses.

References

Cohn, Neil (2013). The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images. London: Bloomsbury.

Forceville, Charles (2005). “Visual representations of the Idealized Cognitive Model of anger in the Asterix album La Zizanie.” Journal of Pragmatics 37: 69-88.

Forceville, Charles (2011). “Pictorial runes in Tintin and the Picaros.” Journal of Pragmatics 43: 875-890.

Forceville, Charles (2013). “Creative visual duality in comics balloons.” In: Tony Veale, Kurt Feyaerts, and Charles Forceville (eds), The Agile Mind: Creativity and the Agile Mind: A Multi-Disciplinary Exploration of a Multi-Faceted Phenomenon (253-273). Berlin: Mouton de Gruyter.

Forceville, Charles (2014.). “Relevance Theory as model for analysing visual and multimodal communication.” In: David Machin (ed.), Visual Communication (51-70). Berlin: Mouton de Gruyter.

Forceville, Charles (2016). “Conceptual Metaphor Theory, Blending Theory, and other cognitivist perspectives on comics.” In: Neil Cohn (ed.), The Visual Narrative Reader (89-114). London: Bloomsbury.

Forceville, Charles, Tony Veale, and Kurt Feyaerts (2010). “Balloonics: The visuals of balloons in comics.” In: Joyce Goggin and Dan Hassler-Forest (eds), The Rise and Reason of Comics and Graphic Literature: Critical Essays on the Form (56-73). Jefferson NC: McFarland.

Forceville, Charles, and Billy Clark (2014). “Can pictures have explicatures?” Linguagem em (Dis)Curso 14(3): 451-472.

Kukkonen, Karin (2013). Contemporary Comics Storytelling. Nebraska: University of Nebraska Press.

Yus, Francisco (2008). “Inferring from comics: A multi-stage account.” In: Pelegrí Sancho Cremades, Carmen Gregori Signes, and Santiago Renard (eds). El Discurs del Comic (223-249). Valencia: University of Valencia.

Klaus Kaindl: “Mind the gap: Comics, Multimodality and Translation”

Abstract. My starting point will be a short historical overview over the various attempts to integrate multimodality into the field of Translation Studies and an explanation why it was and still is so difficult for TS to consider nonverbal elements as equally important as the verbal mode. I will then sketch out a framework for the analysis of comics drawing on the concept of multimodality as it has been developed by Kress and van Leeuwen. The constitutive elements, ranging from various linguistic components such as text in speech bubbles, onomatopoeia and captions to typographic elements, pictographic elements such as speed lines, ideograms etc. and pictorial representations of persons, objects and situations will be distinguished by their functions and discussed in their translation relevance. The aim of the presentation is to show that translation can never be reduced to a linguistic operation, but always is a multimodal activity.

Olli Philippe Lautenbacher: “Reading Comics – An Eye on the Process”

Abstract. This presentation aims at mapping observable tendencies in comic book reading through eye tracking analysis. Recordings were collected on 30 informants viewing a dozen of spreads from several types of comic books involving different text-image or image-image relations. Our main research question concerns the probability of the “Z-path” (Cohn 2013) in the reading process, by opposition to a “relatively erratic” viewing (Groensteen 2007): Is there a tangible tendency to be observed?

Secondly, if viewing patterns (i.e. reading order and speed) tend not to obey to the basic Z-path, some hypothetical causes can be looked for within the viewed documents themselves (apart from the readers own reading habits, which also were accounted for) such as the presence of: (a) visual saliencies or “eye catchers”; (b) potentially significant regions in the visual scenes, such as faces; (c) text, which might in itself be a strong motivation for linearizing the viewing process as a whole; (d) a specific layout of the page (see Bateman & al. 2016).