A new study from Cornell University reveals that Amazon's AI shopping assistant Rufus provides vague or inaccurate responses to users speaking certain English dialects, such as African American English (AAE), particularly when prompts contain spelling errors.

The researchers developed an evaluation framework to analyze potential harms that may arise when chatbots perform poorly for users of different dialects. They emphasize that this study holds important implications for online platforms relying on large language model-powered chatbots. Emma Harvey, a Cornell Tech PhD student and lead author of the study, stated: “Currently, chatbots may deliver lower-quality responses to dialect users, but this is not inevitable. Training large language models to handle common dialect features beyond standard American English could lead to fairer outcomes.” The research won the Best Paper Award at the ACM Conference on Fairness, Accountability, and Transparency (FAccT 2025), held June 23–26. Co-authors include Associate Professor Rene F. Kizilcec from Cornell University's Ann S. Bowers College of Computing and Information Science. The paper is published in the Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency.

The team audited Amazon Rufus in the Amazon shopping app using the MultiVALUE tool to convert standard English prompts into five dialects and modified prompts to simulate real-world usage scenarios. Results showed a decline in response quality when using dialects, with spelling errors exacerbating the gap. For example, when asked in standard American English whether a jacket is machine-washable, Rufus answered correctly; however, when the question was phrased in AAE without conjunctions, Rufus often failed to answer accurately or directed users to irrelevant products. Overall, the research team advocates for dialect-aware AI auditing and urges developers to design systems that inclusively accommodate linguistic diversity.