A new study published this week in JAMA Network Open has found that leading artificial intelligence chatbots perform poorly when asked to provide medical advice, raising fresh concerns about the growing number of Americans turning to such tools for health guidance. Researchers tested 21 frontier large language models with realistic patient symptoms and found failure rates exceeded 80 percent in ambiguous cases that could match more than one condition. Even in more straightforward cases that included physical exam findings and lab results, the systems still failed 40 percent of the time.
Clinical Reasoning Falls Short
The researchers said the models showed weak performance across the board and, unlike human clinicians, tended to collapse prematurely onto a single answer rather than working through a differential diagnosis. Marc Succi, corresponding author of the study and associate chair of innovation and commercialization at Massachusetts General Hospital, said off the shelf large language models are not ready for unsupervised clinical grade deployment.
The findings point to serious risks when people rely on chatbots for medical guidance without a full clinical picture. A person asking about symptoms such as a rash or sudden cough could be given misleading information and potentially dangerous advice, the study suggested.
FCRF Returns With CDPO, Its Premier Data Protection Certification for Privacy Professionals
Growing Reliance on AI for Health Advice
The study’s findings come alongside survey data from the West Health-Gallup Center on Healthcare in America showing that one in four American adults, or the equivalent of 66 million people, are already asking ChatGPT and similar chatbots for medical advice. Respondents said they often used the tools both before and after seeing a healthcare professional, but in some cases they skipped real world medical care altogether after consulting a chatbot.
Among those who had asked AI for health advice, 14 percent, representing more than nine million Americans, said they did not see a provider they otherwise would have visited because of the technology. The survey also found that 27 percent cited the cost of a doctor’s visit as a reason for consulting AI, while 14 percent said they were unable to pay for one. Others said they did not have the time or ability to visit a doctor.
False Confidence and Calls for Oversight
The survey found that AI’s influence extended beyond access and cost. Nearly half of respondents said talking to a chatbot about medical problems made them feel more confident when speaking to a provider. Another 22 percent said it helped them identify issues earlier, while 19 percent said it allowed them to avoid unnecessary tests or procedures.
At the same time, skepticism remained high. Roughly a third of respondents who used AI for health issues said they distrusted the tool, while one in ten said it had provided potentially unsafe advice. Tim Lash, president of the West Health Policy Center, said artificial intelligence is already reshaping how Americans seek health information, make decisions and engage with providers, and that health systems must keep pace. Taken together, the study and survey present a stark picture of a healthcare environment in which flawed chatbot advice is influencing decisions, even as concerns grow over safety and the need for stronger regulatory oversight.
