Twitch to Detect People Evading Bans with Machine Learning

Twitch is advancing its efforts to reduce bullying with a new tool that makes use of machine learning to detect people who might be trying to evade bans. It is the latest addition to the company to combat hate forays, where streamer chats are full of trolls sending hate messages.

Suspicious User Detection, the new tool can identify users as "probable" or "possible" individuals who have evaded bans on a streamer's channel. The machine learning model that drives the tool identifies potential evaders by evaluating aspects such as their behavior and the characteristics of their account and compares that information with the accounts that have been banned from a transmitter channel.

Messages from "likely" evaders will not be sent to chat, but streamers and their mods will be able to see them. Streamers and mods can opt to monitor a likely ban evader, adding that user to a monitoring list and placing a message next to a user's name indicating monitoring (as shown in the GIF below) or ban it. Messages from "possible" evaders will appear in the chat, but streamers/mods can choose to have those messages blocked in chat as well.

Twitch says it will turn on Suspicious User Detection by default, but streamers can modify or disable the tool if they wish. Streamers and mods will also be able to manually select to monitor users they suspect.

"This tool was inspired in large part by community feedback around the need for better ways to curb ban evaders," Alison Huffman, Twitch's director of product for community health, said in a statement to The Verge. "When we were speaking with mods about their pain points, we heard that it can be hard to distinguish whether a user who chatted something that violated their channel's norms was a harmful, repeat harasser or just a newer viewer who hadn't learned that channel's customs yet. As such, we designed this tool to give mods and creators more information about potential ban evaders so they could make more efficient and informed decisions within their channel."

Suspicious user detection tool seems like it could make a difference in silencing haters, primarily when used in conjunction with recently introduced controls that allow a streamer to require verification by phone or email for accounts participating in the chat. But we need to wait and see how effective Suspicious User Detection will be in practice or if ban evaders will find ways to get around the tool.