Conversational AI and equity through assessing GPT-3's communication with diverse social groups on contentious topics.

ABSTRACT: Autoregressive language models, which use deep learning to produce human-like texts, have surged in prevalence. Despite advances in these models, concerns arise about their equity across diverse populations. While AI fairness is discussed widely, metrics to measure equity in dialogue systems are lacking. This paper presents a framework, rooted in deliberative democracy and science communication studies, to evaluate equity in human-AI communication. Using it, we conducted an algorithm auditing study to examine how GPT-3 responded to different populations who vary in sociodemographic backgrounds and viewpoints on crucial science and social issues: climate change and the Black Lives Matter (BLM) movement. We analyzed 20,000 dialogues with 3290 participants differing in gender, race, education

SUBMITTER: Chen K 

PROVIDER: S-EPMC10796352 | biostudies-literature | 2024 Jan

REPOSITORIES: biostudies-literature