News

Our new ChatGPT benchmarking approach published by JAMIA
2023-12-20 00:00 2023/12/20 We published a novel approach for ChatGPT benchmarking study in JAMIA.   We pioneered the use of live symptom checking services, such as Mayo Clinic Symptom Checker, for benchmarking ChatGPT’s symptom checking capabilities. Across a broad range of 194 diseases, the symptom checking accuracy approached 80% for GPT-4 model, which qualifies ChatGPT for clinical studies.     Chen A, Ch...
Our ChatGPT co-pilot for LHS training published by JAMA
2023-08-01 00:00 2023/8/1 We published a paper on LHS training with ChatGPT co-pilot in the JAMA Network.   For the first time, ChatGPT was utilized as a co-pilot to provide LHS training, and we documented the challenges associated with the ChatGPT co-pilot. The scientific journal references cited in GPT-4 are mostly correct, although nearly all are fictitious in GPT-3.5.   Chen A, Chen DO. Accuracy of Chatbo...
Our ML-LHS simulation breakthrough published by Nature
2022-10-26 00:00 2022/10/26 Our pioneering work of ML-LHS simulation published by Nature   Our pioneering work on using synthetic patient data to simulate Learning Health Systems (LHS) has been published by Nature. More than a decade after the National Academy of Medicine (NAM) released its vision for Learning Health Systems, there remains a lack of reported examples of machine learning-driven LHS in routine cl...