You know those strange images that have been crowding your Facebook feed lately? Yes, I'm talking about those crazy posts with homeless veterans with poorly written signs or cliffs that look like Jesus. I was talking about this a couple of weeks ago, and now I've looked into the matter further. Welcome to the era of AI content on Facebook, a world where the absurd reigns supreme and, surprisingly, there are those who make big money from it. Even 400 euros per image.
Get ready for a journey into the dark side of artificial creativity, fueled by the distorted incentives of Facebook itself: yes sir, Zuckerberg has screwed up again.
The mechanism behind the invasion of deceptive AI content
A recent investigation of 404 Media shed light on the phenomenon of AI content on Facebook, revealing an entire ecosystem dedicated to the automated creation and distribution of images on the big "F" social network.
Creators from countries like Pakistan, India, Vietnam, Thailand and Indonesia use tools like Microsoft AI Image Creator to mass produce content, posting dozens of images every day, sometimes every hour. And this is the fault yet another drift that is infesting the timelines of all of us is always the same: the operator. Once again, Facebook is the cause.
Facebook's bonus program: a perverse incentive
The heart of the problem lies in Facebook's Creator Bonus Program. Accounts that post consistently can be invited to participate and start earning from viral content. This incentive has created yet another perverse mechanism: the more you publish, no matter what, the greater the potential earnings.
As a result, the goal became to post as much as possible, sometimes hundreds of times a day. And produce the easiest and most immediate reaction. Result? A monstrous bombing, which surpasses even the worst predictions.
The AI spam ecosystem
As in all spam circles, a parallel economy of tools and knowledge has developed to start this business. There are dedicated YouTube channels, guides on Fiverr and Gumroad, and Telegram channels filled with plain English prompts to help AI image generators create viral content.
I won't go into details, also so as not to contribute further to the phenomenon. I'll just say that with a combination of Vercel accounts, WordPress, and automated tools, spammers are able to bypass Facebook's (however ridiculous) spam detection systems, creating posts that look like photo albums but are actually disguised links.
Facebook's inadequate response
Facebook, or rather Meta, seems to be in an ambiguous position regarding this situation. On the one hand, the company says it encourages creators to use AI tools to produce high-quality content that meets community standards. On the other hand, Meta's content moderation teams can't keep up with the deluge of AI content.
“We know that bad actors adapt their tactics to evade our rules, so we are always working to improve our detection and enforcement,” a Meta spokesperson said.
A totally inadequate response given the scale of the problem. Facebook may not be motivated to completely stop this phenomenon, given that it benefits from the increase in engagement that this content generates. And then maybe the algorithm (it did it with us too) accidentally flags informative articles, instead letting authentic rubbish pass through.
The ethical and social implications, and our local "crafty people".
This phenomenon raises serious ethical questions. On the one hand, for some creators in developing countries, this practice represents a significant economic opportunity. On the other hand, behavior that is degrading the overall quality of content on Facebook and potentially misleading users.
And we put ours in too. And yes, because Facebook's incentives are also given to "unsuspected" European and Italian newspapers. And so, without naming names but remaining on the pages of our sector (scientific dissemination), we happen to see esteemed sites that suddenly start compulsively posting frivolous or increasingly off-topic topics. All to get extra likes and comments, handsomely paid for by Facebook.
Zuckerberg's responsibilities
It is clear that Facebook has a primary responsibility in this situation. The incentives created by the platform have actually encouraged this practice. While the company claims to be working to improve the detection and enforcement of rules, in reality there seems to be a lack of will to address the root of the problem.
Facebook could implement more stringent measures to identify and limit low-quality AI content. It could also overhaul its Creator Bonus Program to reward quality over quantity. The ideas are there, you just need to apply them. However, as long as this content continues to generate engagement and, consequently, profits for Facebook, there is a risk that the company will remain… How can I put it? Ambivalent.
In conclusion
The invasion of AI content on Facebook is the symptom of a broader problem: a business model that prioritizes engagement more than ever at the expense of content quality. And it ends up humiliating artificial intelligence itself, which would have the potential to enrich our online experience. Its improper use, encouraged by short-sighted corporate policies, is leading to a further deterioration of the social media ecosystem, already degraded by previous "mistakes" (casual or intentional). It's time for Facebook take responsibility to create a healthier and more authentic online environment, even if that might mean sacrificing some of its profits.
