ChatGPT, a product of OpenAI, has shown potential as a clinical decision-support tool in assessing patients with acute ulcerative colitis (UC) in emergency departments (ED), according to a study led by Asaf Levartovsky, MD, from the Department of Gastroenterology, Sheba Medical Center, Tel Aviv University. The study, published in the American Journal of Gastroenterology, evaluated GPT-4’s ability to gauge disease severity and decide on hospitalizations, using 20 distinct UC presentations. The AI’s assessments were consistent with expert gastroenterologists 80% of the time. However, while ChatGPT recommended hospital admission for 16 out of 18 patients, only 12 were actually hospitalized in practice.
The study found that ChatGPT could categorize UC severity with a high degree of reliability, with an average correlation of 0.839 (P < .001) between the AI and physician ratings. However, there were some inconsistencies, primarily due to inaccurate cut-off values for systemic variables. The researchers believe that while ChatGPT could act as a real-time decision support tool, it isn’t intended to replace physicians but to augment human decision-making. They also emphasized the need for further validation with larger samples and varied clinical scenarios, given the study’s small sample size.