AI moderation in social networks
Facebook isn’t the only platform massively leveraging AI moderation.
Let’s find out some recent solutions implemented by other social networks to maintain a pleasant and friendly environment.
Basically what I try to do when I bring Nutella cookies to our office.
AI moderation in LinkedIn
The professional platform has an automatic fake account and abuse detection system, which allows it to “take care” of more than 20 million fake profiles in the first 6 months of 2019.
The first approach followed by LinkedIn detected sets of inappropriate words and phrases known as blocklist. When they appeared in some account, it was marked and erased.
Unfortunately, it required a lot of manual work for the evaluation of words or phrases in relation to the context.
Lately, a new approach has been introduced to mitigate such issues. It’s based on a deep model that was trained with a set of accounts already labelled as appropriate or inappropriate (and consequently removed from the platform)
The system is powered by a Convolutional Neural Network (CNN), which is particularly efficient in managing images and texts classification tasks.
AI moderation in… Tinder!
I know that I just caught your attention. And it’s not a clickbait. I’m serious!
Tinder relies on machine learning to punish maniacs scan potentially offensive messages.
When the system detects a controversial message, Tinder asks the receiving user for a confirmation of the potential offense and directs him/her to its report form.
Tinder’s developers trained its machine-learning model on a wide set of messages already reported as inappropriate.
In this way, the algorithm recognized keywords and patterns that could help it recognize offensive texts.
AI moderation in Instagram
Instagram introduced several improvements to boost its text and image recognition technology.
This allowed, for example, to detect and delete 40% more content related to suicide and self-harm.
We need AI moderation now more than ever
I’ve already said so and I’ll repeat it.
The data flow shared between users is simply not manageable by humans anymore. At this point, the contribution of artificial tools to assist real moderators is essential.
But that’s not the only issue that is acting as a catalyst for the research of new solutions.
Another factor to consider is the growing use of fake accounts and other AI-based tricks to influence politics.
An example? Reports showed that fake accounts and bots represent 15% of Twitter users.
In 2018, Twitter declared to have canceled 50,000 fake accounts likely created to influence the 2016 presidential election.
According to estimates, their posts reached about 678,000 Americans.
More recently, the global health crisis has made it necessary to increase the use of AI moderation techniques.
YouTube has opted to rely more on AI to moderate videos during the pandemic because many human reviewers have been left home to limit the COVID spread.
Facebook has done the same, announcing in March its intention to send home its content moderators, many of whom were workers of newly quarantined Philippines-based outsourcing companies.
So, fewer people eliminating toxic content and more people sitting at home, willing to fight boredom by spamming nonsense on the internet.
When solutions bring more troubles
At the same time, this crisis also showed artificial intelligence flaws in handling tasks previously managed (at least in part) by humans.
In fact, the day after Facebook’s announcement on AI content moderation, users were already complaining about arbitrary errors and blocking of posts and links marked as spam.
Unfortunately, the list of problems related to AI moderation is still quite long. Let’s see some issues in this regard.
Flaws and biases of AI moderation
I started this article on artificial intelligence with the cool stuff. I’m gonna end it with bad things. Like a twisted fairy tale without a happy ending.
The point is that AI moderation is still far from perfection.
A first reason is that identifying harmful content often requires an understanding of the context around it, to determine whether or not it is really dangerous.
This can be a challenging task for both human and AI systems because it requires a general understanding of cultural, political and historical factors, which vary widely around the world.
Pasta with ketchup can be fine for unforgivable heretics some people. But it’s definitely controversial and harmful content for an Italian audience.
Rightly, I would say.
Moderating the “worst” content formats
Another issue concerns the fact that online content appears in many different formats, some of which are pretty difficult to analyse and moderate.
For example video content requires image analysis over multiple frames combined with audio analysis, while memes need a combination of text and image analysis, plus a bit of cultural and contextual understanding.
Even harder is the moderation of live content such as streaming and text chats, which can escalate quickly into a mess and must be scanned and moderated in real time.
The problem of AI biases
If humans are biased, machines made by humans are biased too
Systems can be biased based on who builds, develops and uses them. This issue is known as algorithmic bias and it’s really hard to figure out, since this technology usually operates as a black box.
In fact, we never know with absolute certainty how a specific algorithm was designed, what data helped train it, and how it works.
Regarding this, an interesting study has revealed that by training artificial intelligence with what humans have already written on the Internet, the system would produce bias against blacks and women.
Solving all these flaws will be the future challenge of AI developers.
Meanwhile, “enjoy” the flame!