OpenAI Cracks Down on Contractors Using AI for Model Training

OpenAI is reportedly removing contract workers from AI model-training projects after finding that some used artificial intelligence tools to complete work intended to improve the company’s own AI systems.

Why Are Contractors Being Removed?

The workers were part of model-training projects where they were expected to review and improve AI outputs. According to the information cited in the report, some contractors instead used AI tools to carry out the work themselves.

The projects can involve tens of thousands of contract workers. Contractors may earn as much as $50 an hour, roughly ₹4,800, according to the report.

Using AI for these assignments is prohibited under the guidelines described in the report. One contractor said the practice was common and that workers could be removed quickly if they were found using AI.

What Work Were They Doing?

Some workers reportedly reviewed real ChatGPT user prompts and conversations to assess how the chatbot responded. Their job included analysing responses and checking that the system did not become overly sycophantic, meaning it simply agreed with users or told them what they wanted to hear instead of providing accurate information.

The work was intended to help improve AI models by bringing human assessment into the training process.

Reviewers examining contractors’ submissions were also given instructions restricting the use of AI. They were told not to use tools including Grammarly and AI translation when reviewing work, writing feedback or preparing comments.

Proposal for Conducting Cyber Crisis Drill, Tabletop Exercise (TTEx) & CCMP Readiness Exercise

Why Is AI-Generated Training Data a Concern?

One concern highlighted in the report is the possibility of AI models being trained on AI-generated material.

Training models repeatedly on such output can contribute to what is described as “model collapse”, where a model may become less accurate and less diverse over time as it learns from material produced by other AI systems rather than the intended human-generated work.

This makes the authenticity of human input important in projects specifically designed to use people to assess and improve AI behaviour.

What Happened to Workers Who Used AI?

One contractor who had used AI shared what was described as a termination letter that raised concerns about the “authenticity” of the person’s work.

The contractor said they had initially used AI for a small productivity boost, but that reliance on the technology increased. The worker said this eventually contributed to their removal from the project.

Another contractor said people used AI “all the time” and were also regularly removed for doing so.

What Rules Apply to the Work?

The guidelines described in the report prohibit workers from using AI tools for the assignments. They also restrict the use of AI-detection tools.

Mercor, a company that hires contract workers for AI labs including OpenAI, said its contracts strictly prohibit the use of large language models to complete projects.

A Mercor spokesperson said the company uses tools and systems to detect misuse and ensure experts follow project rules and contract terms. When it confirms that an expert has used AI to complete a task, the person is immediately removed from the project.

How Are User Prompts Handled?

The report also said OpenAI routes user prompts through a Privacy Filter model before they reach contractors.

However, documentation cited in the report said sensitive information can still pass through, including information from user memory summaries.

For Free, Plus and Pro users, training participation is enabled by default and can be turned off through ChatGPT’s data controls, according to the report.

The unusual part of the case is that human reviewers hired to improve AI systems are themselves being scrutinised for relying on AI. The concern goes beyond workplace rules. If a task specifically requires human judgement, replacing that judgement with machine-generated output can undermine the purpose of the training process itself.