Discriminating dialects: mitigating bias in Automatic Speech Recognition

In this project we focus on the societal implications of AI, and specifically how Automatic Speech Recognition (ASR) - the translation of spoken language into text - may impact on sustainability and resilience across linguistically marginalised communities. ASR has made remarkable advances in the 21st century, and is now ubiquitous across a range of services including policing and health care. However, research shows that most ASR systems perform significantly worse across particular types of language, including non-standard, often highly stigmatised dialects. Simply put, ASR systems do not recognise many voices worldwide. As such systems become commonplace in everyday life, these linguistic biases have the potential to further exacerbate existing inequalities in access in a society increasingly shaped by AI. Part of the problem with ASR systems is that much of the expertise to date in building the systems come from computer scientists and engineers. These experts have built systems which work for specific standard language varieties, and they concentrate on how the systems work - either successfully or not - but not why. A critical missing perspective in this human-computer interaction is the one at its heart - the everyday, highly variable language used by speakers worldwide. In this research, we bring a necessary humanities perspective to this technology. We leverage the US and UK's team's knowledge in sociolinguistic variation and speech technologies to assess ASR performance in two non-standard varieties of English divided by demography but united by bias: African American English (AAE) in the US and Scots in the UK. We significantly extend emergent research in this area through the analysis of ASR performance across multiple speakers and multiple dialect forms in multiple marginalised speech communities. This research will provide the most comprehensive sociolinguistic profile to date of ASR performance - which groups of speakers and which linguistic forms prove to be problematic in the translation of spoken language into text? More broadly, it will provide crucial knowledge of how ASR systems can be improved for these under-represented varieties, making them more equitable, answerable and accountable to the communities most heavily impacted by unequal access to this technology. The results of this project will be shared in workshop format with our existing industry partner, the Mozilla Foundation, who endeavour to democratise ASR systems through the inclusion of all types of speech data in future development. The specific research questions are: Assessing ASR performance overall: how do the different ASR systems compare with respect to error rates across the different speech communities? Do some dialect areas pose significantly more problems than others? 2. ASR and training: Is it possible to train ASR systems to improve their performance for these dialects, and if so, what sort of training data is necessary? 3. ASR and sociolinguistic variation: are certain specific dialect forms more prone to errors than others across different levels of the speech system? And how do these linguistic forms interact with the social system, including geographic region, age and gender? Are there universals of ‘crash points’ both within and between the varieties, or are the issues more local? 4. ASR and real-world applications: how can the knowledge gained in this research be applied in the further development of ASR systems?