Nerd City, a YouTube commentary channel with 700,000 subscribers that looks into online trends and crucial goings-on across the platform — and which previously made waves with an investigation into whether Jake Paul hawking merch to minors crosses legal boundaries and delved into YouTube’s alleged secret rating system — has conducted yet another study into a radioactive theory: that using LGBTQ+-related words in videos titles conclusively causes demonetization.
Nerd City (pictured above) partnered on the four-month study with InternetIsReal and Sealow, two creators with a smaller online presence but a clear affinity for the machinations of YouTube. Their conclusions were published yesterday evening in a video on the Nerd City channel titled “YouTube’s Biggest Lie.” This ‘lie’, they posit, refers to the fact YouTube appears to be outwardly accepting of the LGBTQ+ community while unfairly demonetizing videos that contain LGBTQ+-related terms in their titles.
In order to reach this conclusion, the crew conducted analysis on an undisclosed channel that they tell Tubefilter had no prior manual reviews or demonetization issues. Accordingly, they tested roughly 15,000 different terms as the titles of roughly two-second-long unlisted videos — with no other video or audio content that could sway monetization classifiers one way or another — to determine how YouTube’s bots would react to certain words.
Subscribe for daily Tubefilter Top Stories
Words used in the test ranged from innocuous to vicious hate speech (see below). Some words were always demonetized, some were always monetized, and others returned mixed results. While the overarching thesis of their video seems to suggest that YouTube demonetizes videos with LGBTQ+-related titles, more granular data that they provided in a Google Sheet (of all 15,296 terms that they tested) paints a far more complex picture with numerous exceptions.
In many instances, the results seem arbitrary and inconclusive. ‘Abortion’ was always demonetized, for instance, while ‘Abortions’ was always monetized. Other terms that were apparently not ad-friendly include ‘bold’ and ‘Colorado’, while terms like ‘d-cksucker’ and ‘f-dge p-cker’ were apparently able to run ads. While ‘gay’ was consistently demonetized in their research, they found terms like ‘lesbian’, ‘bisexual’, ‘transgender’, and ‘queer’ were all usually monetizable — though the team did note that they received “mixed results” on ‘bisexual’, ‘lesbian’, and ‘queer’.
Here are some funny/strange results from over the months. some have changed, while others are still like this… pic.twitter.com/jVtc7wwoTF
— YouTube Analyzed (@InternetsReal) September 30, 2019
Finally, Nerd City, InternetIsReal, and Sealow sought to drill down on the context of LGBTQ+ words in longer video titles — as they lay out in an accompanying White Paper. For this test, they replaced the word ‘gay’ with ‘happy’ in various made-up video titles — and the word ‘lesbian’ with ‘friend’. When they did this, demonetized videos were always re-monetized, they found.
The discrepancies noted in the study show that the inner-workings of YouTube’s machine-learning algorithm are impossible to understand. Even YouTube’s own engineers can’t 100% evaluate what the algorithm will do because it’s ever-changing as it processes clicks, decisions, watch time, and other behaviors on the billions of users who interact with the 500 hours of video uploaded to the site every minute. However, if there is a kind of algorithmic bias towards demonetizing LGBTQ+ content, Sealow opines it’s because LGBTQ+ topics are frequently raised in the context of non-ad-friendly sex talk or hate speech — which impacts how these terms are classed when being used innocently.
“I am personally confident the discriminatory demonetization of LGBTQ terms are the result of the probabilistic nature of the machine learning classifiers,” he told Tubefilter over Twitter DM. “Several LGBTQ creators often openly talk about sex and are manually confirmed to be demonetized. Likewise, anti-LGBTQ content is also manually confirmed to be demonetized for hate speech.”
Ultimately, the Nerd City video (below) calls on YouTube to be more transparent about the factors that trigger demonetization — a similar demand made by the recently-formed YouTuber Union, which Nerd City references extensively in its investigation. The community-based movement of 18,000 members joined forces with Europe’s largest trade union, IG Metall, in July to launch a campaign called FairTube. (Google said in late August that it would meet with the group, but declined to negotiate any of its demands).
In addition to YouTube’s bots, Nerd City, InternetIsReal, and Sealow also point out what they see as the detrimental impact of the video company’s vast team of 10,000 human reviewers, who could potentially harbor homophobic beliefs, they surmise. If these values are then bleeding into the demonetization appeal process, they infer, then this also impacts how YouTube’s bots are assessing comparable cases moving going forward.
YouTube, for its part, says that the team who handles manual reviews of appealed videos have been extensively trained.
“We’re proud of the incredible LGBTQ+ voices on our platform and take concerns like these very seriously,” a YouTube spokesperson told Tubefilter. “We do not have a list of LGBTQ+-related words that trigger demonetization, and we are constantly evaluating our systems to help ensure that they are reflecting our policies without unfair bias. We use machine learning to evaluate content against our advertiser guidelines. Sometimes our systems get it wrong, which is why we’ve encouraged creators to appeal. Successful appeals ensure that our systems are updated to get better and better.”
Given that this issue has cropped up before — dating back to Pride Month in 2018, when transgender creator Chase Ross brought similar allegations to light — YouTube says that it has actively evaluated its systems and taken substantive strides to ensure that LGBTQ+ creators and content aren’t being unfairly demonetized. Every time it launches a new monetization classifier, the company says, it tests samples of videos from LGBTQ+ creators against videos from the entire YouTube corpus to to ensure that there is no bias at play. Currently, the company says that demonetization is being applied fairly to videos with and without LGBTQ+-related themes.
In previous interviews, CEO Susan Wojcicki and chief product officer Neal Mohan have also categorically denied that YouTube has any policies in place that would automatically demonetize videos that have LGBTQ-related terms in the title. That said, Mohan has hinted that the company “may have made mistakes in the past” with respect to this issue. However, he claims that the company has been working on new ways to ensure “machine learning fairness” — in other words, to ensure that its bots aren’t unfairly targeting specific groups.
While Nerd City’s research poses compelling questions about the obtuse nature of demonetization, the issue was also brought to the fore last month when a fleet of eight creators (including Ross) filed a lawsuit against YouTube, alleging the platform discriminated against videos related to LGBTQ+ identities and issues.
