NSF/IIS-EPSRC: Longitudinal, Multilingual, and Multi-format Investigation and Detection of LLM-Generated Disinformation

Generative AI is now used by hundreds of millions of people worldwide. In the United States alone, 18% of adults have used ChatGPT. As the fluency and affordability of generative AI continue to grow, so does its wide-ranging misuse in cheap, but highly convincing large-scale disinformation campaigns. For example, NewsGuard has identified close to 800 AI-generated news and information sites, which operate with little human input and are propagating false narratives. Therefore new research is urgently needed to better understand how new AI models can be misused for the generation of false narratives in multiple languages and formats. To counter AI-generated disinformation in the wild, novel more effective, multilingual AI models for disinformation detection are also urgently needed. To this end, we will leverage on the two teams’ world leading research in Natural Language Processing (NLP), Machine Learning, and mis/disinformation detection. Thus this project will also strengthen the capacity of both UK and US in the key growth areas of NLP with machine learning methods and practical applications of AI. To accomplish this vision, the following objectives will be addressed: O1 GENERATE: Investigate different ways of bypassing the safeguards of Large Language Models (LLMs) to generate disinformation in multiple high/medium- and low-resource languages and formats (including news articles, social media content, and blog posts). We will leverage translation, paraphrasing, and perturbation methods with prefix-style prompts to create a total of 12 snapshots (quarterly across 3 years) of LLM-generated disinformation in 9 languages, 5 formats, and using multiple evolving state-of-the-art LLMs per snapshot. O2 DETECT: We will start by studying the ability of citizens, journalists, and fact-checkers to distinguish AI-generated disinformation from human-authored disinformation and true news. Next we will use these insights to develop novel multilingual LLM-based models for accurately detecting human-written and LLM-generated disinformation, taking into account the effects of multilinguality, text length, domain/topical diversity, and the different AI generators themselves. O3 INTEGRATE & VALIDATE: Test the real-world application of the detection models from O2 by integrating them within Agence France Presse (AFP)’s open source content verification browser plugin, currently used by over 100,000 professionals worldwide. Validation activities will involve all our fact-checking partners listed below. O4 IMPACT: Impact activities will target fact-checking and media organisations, policymakers, social media platforms, law enforcement agencies, and academics. Training and capacity building activities are aimed at academic beneficiaries, including students, early career researchers, and academics from social sciences and computer science. The outcome from O1, O2, and O3 will be an open-source benchmark dataset and novel methods for analysing AI-generated disinformation, useful both for the research and professional communities. O2 will also help improve our understanding of human capacity to unearth AI-generated disinformation, and help design better fact-checking and content verification tools in O3. For reproducibility, the codebase and models from O2 will be made available to vetted researchers, to prevent AI misuse by disinformation actors. The plugin integration code in O3 will be provided as a branch on AFP’s github repository. O4 will produce numerous peer-reviewed publications in top AI venues, across the UK and US teams, as well as online training materials, a summer school, and a policy-oriented white paper. The teams will ensure broader impact through their collaboration partners in Europe (FullFact, AFP, Deutsche Welle, the Journal, EDMO, FuJo, GLOBSEC, ATC) and worldwide (IFCN, SNUfactcheck, Washington Post).