Why Meta Sent Contractors to Test Rival Chatbots as Teens

The easy version of this story is too easy.

Some headlines have turned WIRED’s reporting into a claim that Meta “poisoned” ChatGPT and Gemini. That word does a lot of work, and most of it is not supported by the public record. What the reporting does support is still serious enough: contractors on a Meta project posed as under-18 users and tested rival chatbots on high-risk topics including self-harm, eating disorders, sex, drugs, slurs, and crisis scenarios. Meta did not deny the work. It framed it as safety benchmarking and said it does not use competitor benchmarking to train its own AI models.

That distinction matters. “Poisoning” suggests malicious data was injected into another company’s model-training pipeline. There is no public evidence for that. The real question is sharper and more useful: why would Meta, a direct competitor and not a regulator, run this kind of test against OpenAI, Google, and Character.AI without their authorization?

What appears to be solid

The original reporting came from WIRED on June 29. According to the story, the project was known internally as Cannes and was managed by Meta contractor Covalen. It was active as recently as April 21, 2026. Contractors were asked to create dummy accounts that appeared to belong to minors, send text and image prompts to rival chatbots, and copy the responses into spreadsheets.

WIRED said it reviewed internal documents, spoke with five people familiar with the project, and examined spreadsheets. One spreadsheet listed fake profiles, including names, email addresses, passwords, and birth dates. Another contained 3,748 prompts. WIRED also reported that one August 2025 testing round sent more than 45,000 prompts through rival systems.

The targets included OpenAI’s ChatGPT, Google’s Gemini, and Character.AI. None of those companies publicly described the work as authorized. Character.AI said the conduct violated its terms and policies. OpenAI said it was looking into the issue. Google said it had not approved the third-party testing and did not know its purpose, while adding that its internal review of WIRED’s samples showed Gemini responding in line with its policies.

Meta’s response is also important. It did not say the project did not exist. A spokesperson told WIRED that testing and benchmarking chatbot responses to ensure safe, age-appropriate experiences is a responsible, industry-standard practice. Meta also said it does not use competitor benchmarking to train its AI models.

The safest factual description, then, is this: Meta was tied to an undisclosed, contractor-run, large-scale safety and competitor-benchmarking effort against rival chatbots, using accounts that appeared to belong to minors. Whether that violated law, platform rules, or competition norms depends on facts that are not fully public.

Why “poisoning” is the wrong frame

Data poisoning usually means putting malicious or misleading data into a model’s training process so the model behaves badly later. Nothing currently public shows Meta had a way to place these chats inside ChatGPT’s or Gemini’s training data. WIRED was careful on this point: the documents it reviewed did not show how, or whether, Meta used the collected responses.

That does not make the conduct harmless. It just means we should describe the problem we can actually see. The visible problem is not model poisoning. It is undisclosed adversarial testing by a competitor, using fake child accounts, at scale, on topics where failure could affect children, lawsuits, regulation, and public trust.

That is more than enough to discuss without inflating the claim.

Why would Meta do it?

Meta’s official answer is youth safety benchmarking.

On its face, that answer is not absurd. AI companies do red-team testing. They benchmark competitors. They study how other products handle the same risk scenarios. Safety teams at OpenAI, Google, Meta, Anthropic, and others all test models against undesirable behavior. Google’s public policy for generative AI prohibits facilitation of self-harm, illegal substances, and attempts to circumvent safety filters. OpenAI’s usage policies prohibit unsolicited safety testing and circumventing safeguards. Meta’s own public materials discuss internal and external red teaming.

The trouble is not that testing exists. The trouble is the combination: rival products, no authorization, fake under-18 accounts, high-risk youth prompts, large scale, and contractors logging responses in spreadsheets.

If a neutral research lab ran a transparent study, disclosed the scope, minimized sensitive data, and followed a responsible disclosure process, the story would feel different. Meta is not neutral. It is one of the companies competing for users, developers, enterprise buyers, and regulatory trust in AI assistants. When Meta says it was doing safety work on rival products, the next question is obvious: safety for whom, and for what use?

Motive one: know where Meta stands under regulatory pressure

In September 2025, the U.S. Federal Trade Commission issued 6(b) orders to seven companies offering consumer-facing AI chatbots: Alphabet, Character Technologies, Instagram, Meta Platforms, OpenAI, Snap, and xAI. The FTC said it wanted information on how these companies measure, test, and monitor potentially negative effects on children and teens.

That turned youth chatbot safety into a regulatory file, not just a PR concern.

For Meta, competitor data would be useful in several ways. It could help the company understand whether its own systems are worse than, comparable to, or better than rivals on the same youth-safety prompts. It could help prepare internal risk assessments. It could help frame the issue as an industry-wide challenge rather than a Meta-specific failure.

There is no public proof that Meta planned to use the data with regulators. But the value of such data in a regulatory environment is easy to see.

Motive two: Meta had its own youth-safety problem

This is where the timeline matters.

In August 2025, Reuters reported on a Meta internal document called “GenAI: Content Risk Standards.” Reuters said the document had allowed Meta chatbots to engage in “romantic or sensual” conversations with children. Meta confirmed the document was authentic but said the examples and notes in question were erroneous, inconsistent with its policies, and had been removed. It also acknowledged inconsistent enforcement.

Later that month, Meta said it would add teen safeguards to its AI products: training systems not to engage with teens on self-harm, suicide, disordered eating, or inappropriate romantic conversations, and limiting teen access to certain AI characters. In October, Meta announced more parental controls and teen AI safety tools.

Against that backdrop, testing rivals becomes easier to understand. Meta needed to know whether it was alone in failing, behind its peers, or facing a problem that every chatbot company struggled with. If rivals failed similar tests, Meta could argue that the problem was industry-wide. If rivals performed better, Meta could learn where its own safeguards lagged.

That is a practical corporate motive. It is not necessarily noble, but it is very believable.

Motive three: product teams benchmark competitors, but AI makes that messy

Every product company studies competitors. A payments company checks checkout flows. A social app studies onboarding. A search company compares answer quality. In AI, the product is not just the interface. The product is model behavior.

To compare model behavior, you ask the same kinds of questions and observe the answers. That turns ordinary competitor research into something closer to adversarial evaluation. When the questions involve minors, self-harm, drugs, or sexual content, the stakes change.

There is also a terms-of-service problem. OpenAI’s policies prohibit unsolicited safety testing and circumventing safeguards. Google’s generative AI policy prohibits attempts to circumvent safety filters and content facilitating self-harm or illegal activity. Character.AI has gone further on youth safety, announcing in late 2025 that users under 18 would no longer have open-ended chat with AI characters.

A few product managers casually trying a competitor’s chatbot would not have drawn this level of scrutiny. Hundreds of contractors, fake minor profiles, tens of thousands of prompts, and structured response collection are different. That looks less like market research and more like a systematic external probe.

Motive four: safety is now competitive intelligence

AI companies are competing not only on capability, price, and latency. They are competing on trust. For chatbots used by families, schools, and teenagers, safety is part of the product.

A dataset showing how often competitors fail on youth-safety prompts could be valuable. It could inform product strategy. It could help sales teams position a product. It could help legal or policy teams understand where rivals are vulnerable. It could shape internal benchmark dashboards.

That is why the Cannes project feels uncomfortable even if one accepts Meta’s safety framing. Safety testing is necessary. But when a competitor secretly tests other companies’ systems through fake child accounts, safety can become a convenient label for competitive intelligence.

Rumman Chowdhury of Humane Intelligence made that point to WIRED, calling the setup a governance gray zone where safety becomes a cover for anticompetitive practices. That is not a legal verdict. It is a useful description of the ambiguity.

This is not only a Meta problem

The line between evaluation and model improvement is becoming harder to police across the whole AI industry.

Business Insider reported in 2025 that Google contractors at Scale AI had used ChatGPT while working to improve Bard. Scale AI described that work as standard side-by-side evaluation, not training on ChatGPT outputs. The language sounds familiar: evaluation, not training.

Reports about Meta restricting engineers’ use of Claude Code and OpenAI Codex point in the same direction. The concern there was that outputs from rival AI tools might seep into Meta’s own training data or evaluations and create distillation or contractual problems. If those reports are accurate, they show that major labs understand rival model outputs are not just ordinary reference material.

The Cannes reporting lands in that larger gray area. AI companies need to test and compare systems. They also know that model outputs can carry competitive value. The industry has not drawn a clean public line between benchmarking, red teaming, distillation, and training assistance.

The best reading is not the loudest one

The strongest reading is not “Meta poisoned ChatGPT.” The evidence does not show that.

The strongest reading is also not Meta’s neat version: routine, responsible, industry-standard safety work. Too many details make that hard to accept without qualification: the fake minor accounts, the lack of authorization, the rival targets, the scale, the sensitive topics, and the opaque use of collected responses.

A more grounded conclusion is this: Meta appears to have run a large, undisclosed competitor test in a high-stakes safety area where it had its own regulatory and reputational exposure. The motives probably overlapped — safety benchmarking, product comparison, regulatory preparation, and competitive intelligence. That overlap is exactly the problem.

If AI companies want youth-safety benchmarks to be trusted, they cannot be built like private opposition research. They need clear rules: who may test, under what disclosure terms, how sensitive prompts and responses are stored, whether outputs can be used for product improvement, and how results are shared with the companies being tested.

Until that exists, the next version of this story will not need a new plot. It will only need a new company name.

Main sources

• WIRED: “Meta Contractors Posed as Teens to Prompt Rival Chatbots About Suicide, Sex, and Drugs”
• FTC: “FTC Launches Inquiry into AI Chatbots Acting as Companions”
• Reuters: “Meta’s AI rules have let bots hold ‘sensual’ chats with children”
• Reuters: “Meta to add new AI safeguards after Reuters report raises teen safety concerns”
• Meta: “Our Approach to Teen AI Safety: Empowering Parents, Protecting Teens”
• OpenAI: Usage Policies
• Google: Generative AI Prohibited Use Policy
• Character.AI: “Taking Bold Steps to Keep Teen Users Safe on Character.AI”
• Business Insider: “Google Used ChatGPT to Improve Bard, Scale AI Documents Reveal”
• Bloomberg Law / The Information reporting on Meta limits around Claude Code and Codex