Insider Q&A - Is the AI and content moderation combo safe?

Before departing Twitter in 2023, Alex Popken worked for a long time as a trust and safety executive, specialising in content moderation. When she began working there in 2013, she was the first employee specifically tasked with moderating Twitter's advertising business.

She currently works for WebPurify as vice president of trust and safety, a content moderation service provider that assists companies in making sure content posted on their websites complies with regulations.

There are other platforms besides social media that require oversight.

Any business that interacts with customers, be it a retailer, a dating app, or a news website, needs someone to filter out undesirable content, including hate speech, harassment, and illegal content. Artificial intelligence is being used by businesses more and more, but Popken points out that people are still necessary for the process.

Popken recently had an interview with The Associated Press. The length and clarity of the conversation have been adjusted.

During your ten years at Twitter, how did you observe changes in content moderation?

Content moderation on Twitter was very new when I joined. I believe that even the concepts of safety and trust were ones that people were only beginning to comprehend and struggle with. As platforms began to see themselves used as weapons in novel ways, the demand for content moderation increased. I can vaguely remember a few significant turning points from my time at Twitter. For instance, it was through Russian meddling in the 2016 US presidential election that we meaningfully and initially realised that bad actors could undermine democracy in the absence of content moderation.

Many large social media companies are using artificial intelligence (AI) to filter content. Do you believe that AI will ever reach a point where we can depend on it?

A: Content that works Machines and humans work together to moderate. Scale is solved by AI, which has been used sparingly for years. Thus, you have content-detection-capable machine learning models that have been trained on various policies. In the end, though, suppose you have a machine learning model that can identify the word "Nazi."

Many posts may be critiquing Nazis or offering informational resources regarding Nazis in contrast to, say, white supremacist ideology. Thus, it is unable to account for context and subtlety. In fact, that's where a human layer comes in.

Indeed, I believe that significant developments are beginning to emerge that will facilitate human labour. And one excellent example of that, in my opinion, is generative AI, which is far more adept than traditional AI models at understanding context and subtleties.

However, moderating generative AI outputs presents whole new use cases for our human moderators. Thus, I believe that human moderation will continue to be necessary for the foreseeable future.

Could you briefly discuss the non-social media businesses you deal with and the methods they employ for content moderation?

A: Well, everything from retail product customisation to, say, letting customers personalise T-shirts, you get the idea. Naturally, you want to stay away from situations where individuals take advantage of that and write offensive or dangerous things on the T-shirt.

In fact, you should be wary of anything that contains user-generated content, including online dating. There, you should make sure that users are who they say they are, avoid inappropriate photo uploads, and watch out for scams like catfishing. It does cover a variety of industries.

What about the issues you moderate? Does anything change there?

A constantly changing field is content moderation. And the world's events have an impact on it. Innovative and developing technologies have an impact on it. It is impacted by dishonest people who will try to use these platforms in creative and novel ways to gain access. As a result, your team responsible for content moderation is always trying to think ahead and identify potential threats.

I believe that there is a small amount of catastrophic thinking involved in this role, where you consider the worst possible outcomes. They undoubtedly change as well. I believe that misinformation is a prime example of something that has many facets and is difficult to moderate. Like boiling the ocean, that is. You can't, after all, verify the veracity of everything someone says? In order to avoid doing the greatest harm in the real world, platforms usually need to concentrate on disinformation. And that's constantly changing too.

Q: There are people who believe that generative AI will destroy the internet and leave it with just fake AI content. Do you think that could be taking place?

A: Artificial intelligence-generated disinformation worries me, particularly in this crucial election season for the entire world. It's true that harmful synthetic and manipulated media, including deepfakes, are becoming more prevalent online. This is concerning because, in my opinion, most people don't have it easy. deciding what is true and what is false.

I think medium to long term, if I can be properly regulated and if there are appropriate guardrails around it, I also think that it can create an opportunity for our trust and safety practitioners. I do. Imagine a world in which AI is an important tool in the tool belt of content moderation, for things like threat intelligence. You know, I think that it’s going to be extremely helpful tool, but it’s also going to be misused. And we’re we’re already seeing that.