The Concept of Multimodality as a Tool for Analysis / Multimodaalisuuden käsite analyysin välineenä

Chairs: Jyrki Pöysä, jyrki.poysa(at)jyu.fi & Tytti Steel, tytti.steel(at)helsinki.fi

Multimodality means the various sectors (modes) of communication or social intercourse. For example, in films multimodality means the combination of moving pictures, sound, text and diverse special effects. In a museum exhibition, the interpretation made by the museum visitor is based on how the space is organized, objects, pictures, various texts and for example films or multimedia. In social interaction, speech and body language are examples of modes. If one of the modes is taken away, it will change the interpretations.

In the panel we shall explore the possibilities of the concept of multimodality in ethnology. For example, how does multimodality influence interpretations of films or museum exhibitions?

Multimodaalisuudella tarkoitetaan viestinnän tai kanssakäymisen monentyyppisiä osa-alueita. Esimerkiksi elokuvassa multimodaalisuus tarkoittaa liikkuvan kuvan, äänen, tekstin ja erilaisten tehosteiden yhdistelmää. Museonäyttelyssä taas kävijän tulkinta pohjautuu näyttelytilan jäsentelyyn, näytteillä oleviin esineisiin ja kuviin sekä erilaisiin näyttelyteksteihin ja vaikkapa näyttelyssä käytettyyn videoaineistoon. Sosiaalisessa kanssakäymisessä moodeja ovat esimerkiksi puhe ja kehon kieli. Jos yksi moodeista otetaan pois, muuttuu myös kohteesta tehty tulkinta.

Työryhmässä pohditaan sitä, miten multimodaalisuuden käsitettä voi käyttää hyödyksi etnologisessa tutkimuksessa. Miten multimodaalisuus vaikuttaa esimerkiksi elokuvista tai museonäyttelyistä tehtyihin tulkintoihin?

 

ABSTRACTS / ABSTRAKTIT

 

Tuomo Hiippala

University of Helsinki

tuomo.hiippala(at)helsinki.fi

A brief introduction to multimodality

What is multimodality? Is it a theory, a toolkit or a field of study? The perspectives may differ, but all agree that meaning is not made in isolation. Spoken language, for instance, is carefully orchestrated with gestures, posture and gaze. Written language, in turn, interacts with images, layout and typography. The aim of multimodal research is to take apart this interaction and describe it systematically.

My presentation traces how multimodal research developed, beginning with its roots in systemic-functional linguistics and social semiotics (Halliday 1978, 1994). Many influential theoretical concepts, which originated in these fields, were later extended beyond language in the seminal works of O’Toole (1994) and Kress & van Leeuwen (1996). I will present the main concepts and their continuing contribution.

The subsequent work has studied multimodality in all aspects of social life. I will outline the main streams of research and their foci, while also highlighting recent criticism towards these approaches. In particular, the calls for increased empiricism have become more frequent after nearly two decades of research.

Finally, I will review the state-of-the-art and identify an issue that stands in the way of further development. I conclude by suggesting that interdisciplinarity, information technology and the place of multimodality among other fields of study constitute important future issues.

References:

Halliday, M.A.K. (1978) Language as a social semiotic: The social interpretation of language and meaning. London: Arnold.

Halliday, M.A.K. (1994) Introduction to functional grammar. Second edition. London: Arnold.

O’Toole, M. (1994) The language of displayed art. London: Leicester University Press.

Kress, G. and van Leeuwen, T. (1996) Reading images: The grammar of visual design. London: Routledge.

 

 

Ester Bardone

Researcher, Department of ethnology, Institute for Cultural Research and Fine Arts, University of Tartu

ester.bardone(at)ut.ee

Susanne Österlund-Pötzsch

Researcher/Archivist, The Archives of Folk Culture, The Society of Swedish Literature in Finland

susanne.osterlund-potzsch(at)sls.fi

Re-considering Performance in Ethnographic Research: Multimodal Reflections

In ethnology and folkloristics performance may be both what we study in culture (an object of research) as well as how we study it (a methodology). Performances are multimodal per se and highlight an unsolvable problem in cultural research – how to capture multimodality in something that is transient, protean and multifaceted.

When we study the intersection of different modes of expression of an event it may acquire rhizomatic characteristics. This indicates far more complex problems than the binary distinctions between verbal and non-verbal or textual and performative. The issue of multimodality poses the question not only about different modes of cultural expression but also of different intersemiotic relations between these.

Based on a series of case-studies from Finland and Estonia – such as festivals, rural entrepreneurship practices, and archival collections – we want to re-examine performance as a methodological and analytical tool from the perspective of multimodality. Considering this we aim to discuss among other things:

• The researcher’s personal experience as a means for bringing out multimodal and multisensory aspects in archive material, and the possibility of reconstructing past performances based on historical records.
• The interplay between traditional ethnographic materials and various new forms of digital media, and the challenges in studying these varied modes/materials as a synthesized whole.
• The role of “the observed” as multimodal meaning-makers in their own right.
• The usefulness of multimodality in studying live performances and experiences, both when it comes to “staged” performances (such as community festivals) and mundane performances (such as walking).
• The challenges of keeping multimodality of the studied material while presenting our research.

 

Pirjo Nenola

MA, Doctoral student / University of Jyväskylä, Department of History and Ethnology

pirjo.nenola(at)jyu.fi

Multimodality in the Choral Field

In my dissertation Choral singing is the best hobby of all my method of data collecting is multi-local field-work. By means of participant observation as active singer and journalist and as passive observer in the Finnish choral field in 2010-2014, interviews, taking photos and videos, applying written experiences, analyzing choral discourse in Internet and Social Media, and in some extend autoethnography, i.e. a data gathered from many sources, I create a big picture that is more comprehensive and elaborated vis-à-vis only one type of data used. For example, in choirs’ performances I observe the repertoire as well as costumes and symbols used, decoration, background multimedia, gestures, manners, social play, rituals and discourse. When observing rehearsals it is possible to pay attention to the relations between the singers and the use of power. The written and told autobiographies reinforce my observations likewise my autoethnographical notes in which it is possible to browse the common mood and reach deeper understanding of liaison between the singers and the conductor. The choral discourse in Internet exposes the image choirs want to display in public and the discourse in Social Media indicates the hidden level of the choral field. My aim is to conceptualize Finnish choral singers’ perceptions of the true meaning of the choral singing in their own life, in their society, and nationwide as well as choirs’ contribution in these levels.

 

Kari Korolainen

FT, post doc -tutkija, perinteentutkimus

kari.korolainen(at)uef.fi

Drawings in Finnish Ethnology and Folkloristics: The Concept of Multimodality within Studies of Research Practices

The Concept of ‘multimodality’ is discussed here in general sense from the viewpoint of the practices of the enquiry, mainly, in context of the drawings, ethnology and art. The idea is to examine Finnish ethnologic and folkloristic studies where drawings are utilised. Drawings, graphs, maps and other graphic representations were employed commonly in Finnish ethnology and folkloristics in the late 19th century and in the 20th century. However, the role of the drawings, especially, from the view point of the scholarly history and the scholarly understanding about the knowledge is less frequently discussed.

The focus is mainly on the research conducted by ethnologists Axel Olai Heikel (1851–1924) because he co-operated with artists in various contexts of his fieldworks. For example, Artist Agathon Reinholm crafted some of the drawings Heikel utilised in his studies. Thus, the research questions are: firstly, what was the role of the drawings in the context of the study as a whole? What was done with the drawings in terms of argumentation, as well as the object of the study? Secondly, how were the authors categorised? This includes the creators of the drawings too. In brief, such categories as, ‘researcher’, ‘artist’, and ‘drawer’ serve as a starting point of the analysis. The material consists of Heikel’s published studies, especially, the lexicon concerning the ethnologic objects in Kalevala (1885), and the doctoral dissertation discussing the Finno-Ugric buildings (1887) are central here. The illustrated edition of Kalevala (1887) is analysed as well. In addition, the archive materials and the public discussions dealing with these studies are examined too.

Heikel and Reinholm made several descriptions of Finnish stoves, for example. The delimiting of the stove as an object of the description was not only justified scientifically, but it was also targeted poetically (Kalevala), and composed artistically in some cases, just to mention a few illustrations. On the other hand, artists were categorised often as “drawers” in the context of ethnology. However, the drawings that these artists crafted were not neutral, since, for example, there occur various uses of framing and shading in the drawings. To conclude, the analysis will demonstrate how the process of constructing an ethnologic account consists of several overlapping spheres, as well as practical tasks and choices.

The concept of ‘multimodality’ is processed here by discussing the multidimensional aspects of the earlier Finnish ethnology. The aim is to focus on the practices of the inquiry, and perhaps, expose a tiny bit of the unexplored history of Finnish ethnology and folkloristics. Furthermore, the objective is to underline those questions which might help scoping with the vastness of the multidisciplinary research and the ambiguity of the knowledge in general.

Key words: drawings, ethnology, ethnomethodology, material environment