There’s something deeply ironic about getting fired from an AI company for using AI, but that’s reportedly exactly what happened to multiple contractors helping train OpenAI’s models.
Contractors hired to review and improve the responses generated by OpenAI’s AI models have been fired or removed from projects after using AI to do their own work, according to a new report from 404 Media.
The rule makes more sense than it might sound. These workers are being hired specifically for their human judgment — something that becomes considerably less useful if ChatGPT or another AI chatbot is doing the judging for them.
One contractor told 404 Media that they see people using AI “all the time” and that workers are regularly removed for doing it. Another contractor shared what they said was a termination letter citing problems with the “authenticity” of their work.
Keeping humans in the loop
AI models don't simply learn everything from scraping enormous amounts of text from books or the internet.
OpenAI says information provided or generated by human trainers and researchers is one of the sources used to develop its foundation models. Human feedback has also long played a role in teaching models which responses people prefer and how those models should behave.
That makes the quality of the human feedback truly important. OpenAI has previously acknowledged in its research that models trained using human feedback can be influenced by the people labeling that data.
The contractors described by 404 Media work across various OpenAI projects, some of which can involve thousands of workers. Their jobs can include reading AI responses, rating them and deciding which answers are better.
But according to internal documents obtained by 404 Media, reviewers aren't allowed to outsource that judgment back to an AI. One set of instructions reportedly prohibits workers from using AI to write feedback or comments and specifically names tools including Grammarly and AI translation.
Mercor, an AI-training company that employs contractors working on OpenAI projects, confirmed to 404 Media that its contracts prohibit workers from using large language models to complete projects.
AI detectors didn't catch the cheating
You might assume an AI company would catch workers using AI with an AI detector. Apparently not.
404 Media doesn't reveal exactly how the contractors who were fired were caught. But internal instructions obtained by the publication offer a fascinating look at how reviewers are told to spot suspected AI use.
And AI detectors aren't part of the process.
The instructions reportedly tell reviewers not to use tools such as GPTZero because they aren't considered reliable. Instead, reviewers are told to judge the overall pattern of a worker's submissions and look for clues including repetitive wording, unusually fast completion times and writing patterns associated with AI. Even excessive use of em dashes can raise suspicion.
Reviewers are also reportedly instructed not to tell workers exactly what made them suspect AI use, because doing so could make it easier to hide in the future.
This is surprisingly revealing about the process: the people checking whether humans are secretly using AI to help train ChatGPT are explicitly told not to trust AI-detection software to do it for them.
Bottom line: humans are irreplaceable
OpenAI uses a lot of synthetic data — information generated with the help of AI models — as part of some training processes. The company says synthetic data can improve model performance and fill gaps where other training data is scarce.
But that's different from secretly substituting an AI model for a person who has specifically been hired to provide human feedback. If an evaluator is supposed to tell OpenAI which response a human prefers, asking another AI to make that decision defeats much of the point of hiring the evaluator.
It doesn't mean these incidents have made ChatGPT worse, and there's no evidence in 404 Media's reporting that AI-generated contractor work caused measurable damage to an OpenAI model.
But the situation does expose one of the stranger realities of the AI boom: even the companies building the AI systems still need humans to tell those systems when they've gotten something right.
Follow Tom's Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds. Subscribe to Tom's Guide on YouTube and follow us on TikTok.
More from Tom's Guide
