What steers the interpretation of a visual or multimodal message? A relevance theory perspective, Charles Forceville.

The quickly growing field of multimodality has hitherto primarily found its theoretical roots in linguistics. Although the best work in linguistics helps multimodality come into its own as a discipline, it is misleading to rely exclusively on linguistic theories to model forms of (mass) communication that are only partially verbal, or not verbal at all.

It is better to analyse “messages” (poems, news, advertisements, political cartoons, films …) in terms of “content” that a more or less specific communicator wants to convey to a more or less specific audience using a more or less specific medium. This medium can draw on one or more “modes” (visuals, language, music, sound …) to convey this message. Importantly, this message exists on the level of cognition before it is presented in a specific medium – although it is of course only its material form that allows the message to be communicated in the first place.

What we need, therefore, is an inclusive model of communication that takes into account the identities of the communicator, the audience, as well as their relation, and that does not privilege specific media and/or modes over others. The contours of such a model actually already exist in Relevance Theory: Communication and Cognition (Blackwell 1995 [1986]). Relevance theory (RT)’s central claim is that each act of communication comes with the presumption of its own optimal relevance to its envisaged audience. Although this claim is not dependent on a specific medium, hitherto RT scholars (typically: linguists) have almost exclusively analysed face-to-face exchanges between two people who stand next to each other. In order to fulfil RT’s potential to develop into an inclusive theory of communication, it is necessary to explore how it can be adapted and refined to account for (1) messages in other modes than (only) the verbal mode; and (2) mass-communication. In Visual and Multimodal Communication: Applying the Relevance Principle (Oxford UP 2020) I take a first step toward this goal by proposing how RT works for mass-communicative messages that involve static visuals.

In my presentation I will focus on how RT approaches the key issue of what factors have an impact on the interpretation of a picture or a multimodal message of the picture-plus-verbal-text variety. Certain (elements of) visuals are coded, and hence have a precise, fully explicit meaning for the message’s envisaged audience, whereas other (elements of) visuals have a meaning that is (strongly or weakly) implicit, that must therefore be inferred on the basis of contextual and situational clues – the latter potentially resulting in small or not so small differences of interpretation by different addressees.

In the talk I will discuss this issue drawing on examples from different genres, including logos & pictograms, advertisements, and cartoons.

Photo: 'Aurora - Connecting Senses’, Cristina Pop-Tiron & Signe Kjær Jensen