World Futures: Multimodal Viewpoint Construction by Russian International Media

(funded by AHRC and DFG: UK-German Funding Initiative in the Humanities)

We are grateful that the AHRC and the DFG opted to fund our collaborative project between the Universities of Oxford and Erlangen-Nürnberg with roughly € 650,000. The project will run from February 2022 to January 2024. All details below:

Principal Investigators: Dr Anna Wilson and Dr Peter Uhrig

Co-Investigators: Dr Scott Hale, Dr Elinor Payne , Professor Philip Torr

Discussions of futures have never been more important and challenging than now when the world finds itself fighting COVID-19. People engage in conversations about what our lives will look like after the pandemic, and so do the media. The media often talk about futures to frame the way we think, and the media use viewpoint construction rooted in the depiction of futures as a subtle but powerful approach to manipulate and influence public opinion.

The project’s three main research questions are:

• How is effective disinformation constructed and communicated by media multimodally?
• How does the way people imagine futures enable the construction and communication of multimodal disinformation?
• How can a combination of human and machine approaches help us research multimodal disinformation effectively?

In the true spirit of the digital humanities, our 2-year project will fuse cognitive and corpus-driven/based analyses of language, prosody, and gesture with area studies (regional knowledge: culture, literature, history, and society), while leveraging latest developments in machine learning, natural language processing and computer vision to tackle the problem at scale. We combine multimodal analysis of Russian international media broadcasts (English and Russian) with the analysis of audiences’ comments on social media to open a window onto viewpoint construction in the audiences’ minds and thus provide additional validity to linguistic analysis of disinformation. Our project will break new ground in cognitive and corpus linguistics to significantly increase the overall reliability of large media data analysis currently used in linguistics as well as other humanities and the social sciences. Our approach will help answer questions posed by academic researchers and Western policymakers in relation to future information threats and mitigations. Ultimately, the project will go beyond the applied linguistics approach to disinformation analysis. It will enable researchers to test multimodal patterns found in a systematic way, thereby addressing one of the biggest challenges linguists face. Furthermore, the project will contribute to answering such big questions in multimodal research in theoretical linguistics (semantics, syntax, pragmatics, comparative) as:

• How are meanings constructed multimodally and how are multimodal meanings perceived?
• How is the construction of meanings distributed across gestural, prosodic, and verbal modes? Are the “redundant” modes of gesture and prosody really redundant?
• How do languages with quite distinct grammatical and phonetic properties, rooted in quite diverse historical and cultural backgrounds, realise the ideas of time and space multimodally while pursuing the same overarching communicative and pragmatic goals?