YouTube has removed part of its website’s publicly available source code after a group of creators discovered that code revealed vital internal information, including individual channels’ P-Scores, YouTube’s own ratings of videos, and details about the platform potentially throttling videos.
Let’s break down what a P-Score is before we dive in. The P-Score (once called a Preference Score) is an internal, proprietary measurement YouTube uses “to surface some of the most popular channels” on the platform. YouTube’s P-Score algorithm judges channels in five key areas: popularity, or how much watch time their content brings in; passion, or how much their content is engaged with (comments, likes, shares, etc.) by YouTube users; protection, or content’s advertiser-friendliness; platform, or how often content is watched on devices like TVs; and finally, videos’ quality of production.
A channel having a good P-Score is a significant factor in whether or not it gets into Google Preferred, a program that sells premium advertising against what YouTube considers its top 5% most brand-friendly channels. If a channel garners a high enough P-Score, it’s automatically enrolled in Google Preferred, and will likely begin generating more AdSense revenue thanks to running premier ads. If a channel has a P-Score lower than YouTube’s top 5%, it’s not getting in.
Subscribe to get the latest creator news
Like YouTube’s recommendation system, the P-Score algorithm has been criticized by creators who worry the platform’s algorithms are passing their content by, or even actively suppressing it, for reasons they don’t understand. Creators, including the fledgling YouTubers Union, are increasingly pushing for transparency from YouTube about how they can make their content more visible and/or friendly to its algorithms, and therefore potentially have it (a) not be demonetized, and (b) shown to more of the site’s two billion monthly active users.
A search for transparency is what prompted a group of creators, including reporter Josh Pescatore and YouTubers Bowblax, Nicholas DeOrio, and Optimus, to look at the publicly viewable code of YouTube’s webpages.
What they found: P-Scores, ratings, and throttling
“Creators who are doing great with popularity, platform, and passion might never have the opportunity to fix their protection scores because they’d never know it’s an issue,” the investigators wrote in their findings report. (News of said findings was first broken by FFWD.) They added that YouTube “does not alert creators or give them an opportunity to access this data without finding some sort of weird exploit to get to it.”
The “weird exploit” these creators used involves a common browser tool. Anyone can see the code they looked at by going to a YouTube video in their browser, right-clicking, and selecting the option to view the page’s source. (The option is View Page Source in Chrome, and Page Source in Firefox.) Doing so will open a separate window full of the page’s code.
The creators probed this code and found that every video on YouTube showed its uploader’s P-Score in a section called ContentLabelRating. Along with an exact P-Score number, the section contained a rating for the individual video, ranging from the most mild, family-friendliest rating, Y, to G, PG, Teen, Mature, and finally X. The investigating creators believe X-rated content is demonetized and age-restricted by default, and is “deemed problematic altogether.”
A number of videos they looked at also had one more thing in this ContentLabelRating section: a curious little bit of code (or sometimes several curious little bits of code) about the video being throttled. “REASON_BRAND_SAFETY_ADS_THROTTLING,” is one common label. Another is “FEATURE_VIDEO_IS_THROTTLED_FOR_OVERLAY.”
The investigating creators apparently noticed these pieces of code were present on nearly every video rated Teen or above. They suspect YouTube’s throttling is divided into brand safety throttling, where it restricts ads from videos after deeming them not brand-safe, and feature video throttling, which could refer to YouTube keeping the video from being recommended by its algorithms. One thing they did note about brand safety throttling is that “many” of the videos with that tag were still monetized.
“If the code here is to be taken literally, the webpage seems to quite literally be throttling advertisements on videos rated above PG mostly due to brand safety issues,” they wrote in their investigation report. “It will not fully demonetize, but it will strictly only advertise enough to make the video enough money to seem like it’s doing well.”
Investigating creators also noted that channels’ P-Scores differed between geographical locations, and that scores tended to be lower outside a user’s home country. Example: Optimus looked into his own channel and saw that in the U.S., where he’s based, his P-Score is 801.7. In Canada, it’s 771.56.
If your channel is also a late-night TV show, your P-Score is probably high
So, how does that P-Score compare to other channels?
To find out what kinds of channels YouTube thinks most highly of, the creators looked at P-Scores from 229 channels across the platform. Bowblax tells Tubefilter he and fellow investigators chose the channels semi-randomly, but did have an eye toward keeping the mix diverse, and ensured they chose channels making content across genres, channels with different subscriber counts, and channels targeting different age demographics.
As it turns out, YouTube likes channels featuring late-night hosts a lot. Like, a lot. Of the 229 channels investigators looked at, the top 10 highest P-Scored channels are nearly all late-night shows or news outlets. The top 10 are, in descending order: The Late Show with Stephen Colbert (U.S. P-Score: 1047.9419); Late Night with Seth Meyers (1046.9635); The Tonight Show Starring Jimmy Fallon (1039.356); The Daily Show with Trevor Noah (1027.056); Jimmy Kimmel Live (1022.8678); CBC News (1016.2958); Fox News (1014.6294); and Linus Tech Tips (1012.2891), the only creator-operated channel.
It’s also worth noting that Will Smith holds slot No. 12 with a score of 1007.8328, and PewDiePie, who we recently noted seemed to have regained YouTube’s favor, is at spot No. 15 with a score of 994.35492. The fact that Smith’s score is higher than PewDiePie’s, despite the latter getting between 400M and 600M views per month and the former getting between 30M and 60M, makes Bowblax speculate that YouTube may manually give higher P-Scores to creators it knows advertisers will like.
On that note, there are some seemingly odd anomalies in the data investigators gathered. For example, they found that McJuggerNuggets, a channel belonging to a creator who just quit YouTube after 13 years because the vast majority of his videos are deemed brand-unsafe and demonetized, apparently has a P-Score of 787.82306, putting him just slightly below popular creator Casey Neistat (789.95135) and higher than ostensibly brand-safe folks like newly crowned late-night host Lilly Singh (741.9342), and Hollywood actors Jack Black (778.95422) and The Rock (745.30951). Meanwhile, YouTube darling Liza Koshy has a P-Score of 826.08258, putting her below the likes of Logan Paul (857.46362), who’s run afoul of YouTube several times, and Steven Crowder (884.17609).
Bowblax and other investigating creators concluded that despite all their digging, they’re “completely in the dark like usual with how YouTube has developed all of this and implemented this.” They added, “We know that this information alone is extremely valuable to the YouTube community, and so figuring this out has been a potential revelation that can benefit creators for a long time to come.”
One hour after the creators’ report (which gives readers step-by-step instructions showing them how to use Page Source to find their own P-Scores) was published, YouTube removed the ContentLabelRating section from publicly available code.
Tubefilter reached out to YouTube to ask why it removed the section. We also asked why and when YouTube applies the “throttled” qualifiers to content, and what exactly those qualifiers do to videos.
YouTube declined to provide information about “throttled.” As for why it removed ContentLabelRating, a spokesperson tells us that YouTube does not share content ratings with creators on a video or channel basis, because those are part of its internal systems. They add that YouTube does not share details of internal systems because it wants to ensure the systems aren’t being abused.
YouTube also gave us its official statement about the creators’ investigation and findings. You can read that statement in full below:
“In order for videos to be eligible for advertising, they must comply with our Advertiser-Friendly Guidelines, which are public and published in detail. We have developed internal systems to mirror these guidelines so we can evaluate the millions of videos that are published to YouTube and determine which ones comply with these guidelines. To make sure we apply this process fairly, we also give creators the chance to appeal if they feel any of their videos have been unfairly demonetized. A set of monetizing channels are also eligible to be part of Google Preferred Lineups, which allows brands to appear among the most engaged and popular channels on YouTube.”
You can read the investigation report here, and see the investigators’ spreadsheet of 200 channels’ P-Scores here.
