Twitch will permaban streamers who create, share, or promote deepfakes

Twitch has updated its policies to explicitly ban “intentionally promoting, creating, or sharing” deepfake NSFW images.

“In January, a brief ‘deepfake porn’ incident was live streamed on Twitch,” the platform said in a blog post about the updates. “This moment caused immense distress to those whose images were used without their consent, and set off a wave of conversations about the dangers that AI-generated explicit imagery can pose.”

The “incident” they’re referring to is streamer Atrioc being caught with NSFW deepfakes of fellow content creators. Atrioc showed his screen during a January 2023 broadcast and inadvertently revealed he had a tab open to a website that hosts deepfaked, nonconsensual NSFW images and videos—including deepfakes of numerous female streamers.

Atrioc subsequently apologized, said he was taking a break from streaming, and stepped back from OFFBRAND, the creator event company he co-founded in September 2022 with Ludwig, Stanz, and Nick Allen.

In its policy update, Twitch says deepfakes aren’t a problem on its platform right now, “but it’s a terrible issue that some streamers (almost exclusively women) may face on the internet at large.”

It also says that deepfakes are banned by its current Community Guidelines, but it’s making the ban against them more explicit “for the sake of clarity and to stay up-to-date with the terminology around safety topics as they evolve.”

The updates are being made in two areas of Twitch’s Community Guidlines: its Adult Sexual Violence and Exploitation policy, and its Adult Nudity policy.

With the former, Twitch is updating guidelines “to make it more clear that intentionally promoting, creating, or sharing [synthetic non-consensual exploitative images, aka deepfakes] can result in an indefinite suspension on the first offense.”

And with its Adult Nudity policy, it’s updating to specifically include deepfakes in the policy language, and says that “even if [the deepfake] is shown only briefly, or, for example, shown to express your outrage or disapproval of the content, it will be removed and will result in an enforcement.”

These updates will roll out later this month, Twitch says.

In addition to changing its policies, Twitch is hosting a Creator Camp installment hosted by Zara Ward, a deepfake expert, manager at the Revenge Porn Helpline, and streamer. She and a member of Twitch’s safety team will talk about why NSFW deepfakes are so damaging, as well as resources available for those affected.

Their stream will air March 14 on the official Creator Camp channel.