How Facebook uses super-efficient AI models to detect hate speech
Building AI that can analyze complicated text isn’t enough to protect people from harmful content. We need systems that spot a slang-filled or intentionally misspelled piece of hate speech — and do it in a fraction of a second and at billion-person scale.
This has been a challenge when deploying systems to detect hate speech because the most powerful, cutting-edge language-understanding systems today use large-scale Transformer models with hundreds of millions or billions of parameters. New models, including Facebook AI’s RoBERTa and XLM-R, have repeatedly advanced the state of the art, but these gains have come from creating ever-larger models that require massive amounts of computation.
To unlock the capabilities of these powerful AI models, Facebook AI recently developed a new Transformer architecture called Linformer. It makes it possible to use them efficiently at scale. Linformer is the first theoretically proven linear-time Transformer architecture. With standard Transformers, the amount of required processing power increases at a geometric rate as the input length increases. With Linformer, however, the number of computations increases only at a linear rate. This makes it possible to use larger pieces of text to train models, and thereby achieve better performance.
We are now using Linformer to analyze billions of pieces of content on Facebook and Instagram in different regions around the world.
Along with other AI advances, Linformer has helped us make steady progress in catching hate speech and content that incites violence. A couple of years ago, very little of the hate speech on our platforms that we removed was done so before anyone reported it. As detailed in Facebook’s quarterly Community Standards Enforcement Report released today, AI proactively detected 94.7 percent of the hate speech we removed.
Earlier this year, we published our research on Linformer and released our code so other researchers and engineers could improve their models. Since our Facebook AI Research (FAIR) lab was founded in 2013, we’ve committed to an open science–based approach. Our research model revolves around publishing code and methodologies, collaborating with other researchers across industry and academia, and creating open benchmarks and challenges. We’re now sharing details here on how Linformer works and how we are using it to keep people safe on our platforms.
These are difficult problems, and our systems are still far from perfect. And even if we had perfect AI tools, there would still be difficult questions about what policies will serve people best. But progress in AI has made our platforms better and safer, and we are working hard to advance our technology further.
Transformer models have become ubiquitous in language modeling, machine translation, speech recognition, symbolic mathematics, computer vision, and reinforcement learning.
