“Are You Blacker Than ChatGPT?” is a new game highlighting racial bias in AI

As OpenAI breaks into new territory with the launch of its video generation model Sora, it is facing an accusation that has dogged it for years. Ad agency McKinney is behind Are You Blacker Than ChatGPT?, a simple browser-based quiz game that exposes racial blind spots in ChatGPT‘s training.

Each round of Are You Blacker Than ChatGPT? includes five questions pulled from a larger set created by McKinney. The questions seem somewhat subjective, but the answers, according to the game, are not. The first time I played, I only got two answers correct out of five, but ChatGPT whiffed on three questions as well. So I guess I’m equally as Black as ChatGPT, which does seem like a bit of an issue for OpenAI.

A disclaimer on the Are You Blacker Than ChatGPT? landing page reminds visitors that “the game you are about to play is for entertainment purposes only.” McKinney’s experiment may have the same irreverence and knowing in-jokes as an installment of Saturday Night Live‘s Black Jeopardy, but there is an important message underneath all the silliness. Black artists have spoken up about racial bias in AI, and Are You Blacker Than ChatGPT? is the latest evidence of that concerning trend.

Racial bias in AI takes multiple forms. The data sets used to train systems like ChatGPT can reflect the racism in our society, which leads to situations where discrimination is “baked into the outcomes the AI is asked to predict,” according to the ACLU

. AI also struggles to identify Black faces, and the lack of diversity in the tech industry makes it more difficult to address problems like these.

OpenAI has been criticized for the lack of diversity on its board and the racial bias exhibited by its technologies, including the image generator DALL-E. “Bias is an important, industrywide problem,” an OpenAI spokesperson told the New York Times last year. A few months later, documents released by OpenAI indicated that DALL-E has “a tendency toward a Western point-of-view” and produces images that “disproportionately represent individuals who appear White, female, and youthful.”

Copywriter Meghan Woods, who co-created Are You Blacker Than ChatGPT? at McKinney, added another theory that could explain racial blind spots in AI. As she told TechCrunch, many pieces of Black cultural knowledge are passed down in person and therefore could be missing from ChatGPT’s training data. “The blind spots can be pretty upsetting,” Woods said. “It’s pretty dangerous.”

Woods cited “hiring, retention,” and “making sure that people are in the room at the table” as three areas where the presence of people of color can “start to have an impact.” But until OpenAI’s chatbot studies up, Are You Blacker Than ChatGPT? will still be alarmingly easy for players.