TikTok faces fresh scrutiny after a New York lawsuit alleged that safety tests misled young users and left harmful content accessible.
Thousands of TikTok users, including children and teenagers, were allegedly given a non-functional version of the platform’s ‘Algo Refresh’ feature, according to newly unsealed court documents in a lawsuit filed by New York Attorney General Letitia James. The complaint claims that users who believed they had reset their video recommendations continued receiving feeds that had not undergone the intended changes.
How did the Algo Refresh feature allegedly fail users?
TikTok designed Algo Refresh to reorganise video recommendations around users’ newer interests, potentially reducing the appearance of unwanted or harmful material. However, the lawsuit alleges that the company repeatedly conducted experiments in 2023 in which some users received a placebo version that did not actually change their feeds.
According to the complaint, users who activated the feature believed they had taken steps to influence the content recommended to them. In reality, those assigned to the non-functional version did not experience the same changes as users who received the working feature.
The lawsuit alleges that TikTok compared the two groups to assess how the feature affected the time users spent on the platform and the resulting advertising revenue. The tests have raised questions about whether users were given a meaningful choice when they attempted to change their viewing experience.
https://the420.in/fcrf-cp-frm-certified-professional-fraud-risk-management-india/
Why did an employee reportedly object to the tests?
The court documents describe concerns raised internally by a TikTok product manager who worked on digital well-being. The employee allegedly warned that placebo testing could mislead users who believed the feature would give them greater control over their recommendations.
According to the complaint, the concern centred on the purpose of Algo Refresh. The feature was intended to offer users greater transparency and control over the recommendation algorithm, making a non-functional version potentially inconsistent with that objective.
The lawsuit also challenges TikTok’s descriptions of how the feature worked. It alleges that the company presented Algo Refresh as a way to reset the algorithm, although its effects were limited to a temporary change. TikTok has disputed this characterisation, arguing that the lawsuit misrepresents the feature’s operation.
Were other safeguards against harmful content disabled?
The unsealed documents describe a separate experiment in which TikTok allegedly disabled safety systems on selected accounts. These safeguards were designed to limit users’ exposure to harmful content, according to the complaint.
The lawsuit identifies a teenage boy in New York who was allegedly included in one such experiment. He later died by suicide. The complaint states that an internal review found he had been shown a substantial amount of content related to suicide and self-harm.
The allegations raise questions about how the company tested its protective systems and whether vulnerable young users were exposed to risks that those safeguards were intended to reduce. However, the claims in the lawsuit do not, by themselves, establish that the experiments caused the teenager’s death.
TikTok has broadly denied the allegations. The company said it regularly improves its products and tests new features under real-world conditions to understand their performance and effects on the user experience. It has also maintained that protecting young users is a priority.
What does the lawsuit mean for TikTok’s safety practices?
James filed the lawsuit against TikTok in 2024, as part of wider legal action by US states over the platform’s effects on children and teenagers. More than two dozen states have sued the company, alleging that its design encourages prolonged engagement and that users have been misled about safety measures.
The latest allegations were included in an amended complaint filed at the end of August. The documents were initially sealed before parts were made public following a court order.
The case places renewed attention on the difference between safety features as presented to users and how they operate in practice. It also raises broader questions about the use of experiments involving children and teenagers, particularly when tests concern tools intended to help them avoid harmful content.
Follow for daily updates on cybercrime, corporate fraud, DFIR, hacking, investigations, and digital forensics
