News

ELHS Institute Publishes Novel ChatGPT Benchmarking Study in JAMIA

 

📅 December 12, 2023

 

 

We are excited to announce the publication of our groundbreaking ChatGPT benchmarking study in the Journal of the American Medical Informatics Association (JAMIA).

 

In this study, we pioneered the use of live symptom-checking services, such as the Mayo Clinic Symptom Checker, to evaluate ChatGPT’s diagnostic accuracy. Testing across 194 diseases, we found that GPT-4 achieved nearly 80% accuracy, demonstrating its potential for clinical applications.

 

This novel approach sets a new standard for assessing AI-driven medical tools and paves the way for further research on integrating ChatGPT into real-world clinical decision-making. 🚀

 

Reference:

Chen A, Chen DO, Tian L. Benchmarking the symptom-checking capabilities of ChatGPT for a broad range of diseases. J Am Med Inform Assoc. 2023 Dec 18:ocad245. doi: 10.1093/jamia/ocad245.