ChatGPT 4.0, a chatbot, has demonstrated an overall accuracy of 80% in answering questions related to irritable bowel syndrome (IBS). This assessment was based on a study conducted by Dr. Anthony Lembo and his team at the Cleveland Clinic’s Digestive Disease Institute. The researchers used Google Trends to identify IBS-related searches from the past five years and formulated 15 questions based on these queries. These questions were then input into ChatGPT 4.0, which was also asked to provide references for its answers.
Key Findings:
- Three independent gastroenterologists evaluated the answers provided by ChatGPT. The results showed that 53% of the answers were accurate, 33% were partially inaccurate, and 13% were accurate but lacked some details. None of the answers were entirely inaccurate.
- In terms of references, ChatGPT provided suitable references for 33% of its answers, unsuitable references for 53%, and no references for 13%.
- Dr. Lembo highlighted that while ChatGPT generally provided accurate information on IBS, it occasionally missed some details or offered outdated information. However, no fully inaccurate information was given.
- Despite its current limitations, especially in literature research and referencing, ChatGPT has the potential to be a valuable resource for patients and may assist physicians in answering patient queries.