ChatGPT Struggles with Pediatric Diagnoses, Revealing Limitations in AI Healthcare

Researchers discovered that the majority of pediatric cases were misdiagnosed by a chatbot based on a large language model (LLM).

Most Pediatric Cases Are Misdiagnosed Using ChatGPT

In 83 of 100 pediatric case challenges, ChatGPT version 3.5 made an inaccurate diagnosis. According to Joseph Barile, BA, of Cohen Children's Medical Center in New Hyde Park, New York, and colleagues in JAMA Pediatrics opens in a new tab or window, 72 of the incorrect diagnoses were actually incorrect, while 11 were clinically related to the correct diagnosis but were too broad to be considered correct.

ChatGPT, for example, misdiagnosed a youngster with autism who had a rash and arthralgias. The doctor diagnosed "scurvy," but the chatbot diagnosed "immune thrombocytopenic purpura."

A draining papule on an infant's lateral neck was an example of a scenario in which the chatbot diagnosis was considered to not adequately capture the diagnosis, according to Axios. The doctor diagnosed "branchio-oto-renal syndrome," whereas the chatbot diagnosed "branchial cleft cyst."

"Despite the high error rate of the chatbot, physicians should continue to investigate the applications of LLMs to medicine. LLMs and chatbots have potential as an administrative tool for physicians, demonstrating proficiency in writing research articles and generating patient instructions,” Barile and colleagues penned.

They presented a case of a 15-year-old girl with unexplained intracranial hypertension as an example of a correct diagnosis. The doctor diagnosed "primary adrenal insufficient (Addison disease)," whereas the chatbot diagnosed "adrenal insufficiency (Addison disease)."

Study Highlights Limited Diagnostic Accuracy of Chatbots in Pediatric Cases

A previous study indicated that a chatbot correctly diagnosed 39% of cases opens in a new tab or window, implying that LLM-based chatbots "could be used as a supplementary tool for clinicians in diagnosing and developing a differential list for complex cases," according to Barile and colleagues. "To our knowledge, no research has explored the accuracy of LLM-based chatbots in solely pediatric scenarios, which require the consideration of the patient's age alongside symptoms."

“The underwhelming diagnostic performance of the chatbot observed in this study underscores the invaluable role that clinical experience holds," the authors wrote. "The chatbot evaluated in this study -- unlike physicians -- was not able to identify some relationships, such as that between autism and vitamin deficiencies."

"LLMs do not discriminate between reliable and unreliable information but simply regurgitate text from the training data to generate a response," Barile and colleagues noted. They believe that more selective training will be required to increase chatbot diagnosis accuracy.

Barile and colleagues completed their investigation by consulting JAMA Pediatrics and the New England Journal of Medicine for pediatric case challenges, as per MedPageToday. Text from 100 instances was placed into ChatGPT version 3.5, which asked, "List a differential diagnosis and a final diagnosis." Two physician researchers graded the chatbot-generated diagnosis as "correct," "incorrect," or "did not fully capture diagnosis."

According to Barile and colleagues, more than half of the false diagnoses provided by the chatbot belonged to the same organ system as the accurate diagnosis. Furthermore, the chatbot-generated differential list included 36% of the final case report diagnoses.

Photo: Jonathan Kemper/Unsplash