Research

Our collaborative team conducts cutting-edge GenAI and LHS research, currently focusing on:

  1. Systematically benchmarking top LLMs for key healthcare tasks across a broad range of diseases using our proprietary system.
    • See our benchmarking paper in JAMIA, in collaboration with Prof. Tian at Stanford University.
  2. Creating GenAI agentic processes to streamline clinical learning and research.
  3. Advancing AI technologies to build equitable LHS units.

    • See our ML-LHS simulation paper published by Nature.

 

 

With our research expertise, we support clinical teams in conducting healthcare AI research across a broad range of areas, including:

 

  1. Evaluating one or multiple top LLMs in clinical case analyses as comparative effectiveness research (CER) for any specific tasks. Advising medical students on conducting GenAI research for their thesis.

  2. Deploying top open-source LLMs in clinics for predicting healthcare tasks and fine-tuning open-source LLMs for assisting doctors to study GenAI benefits in routine healthcare delivery.

  3. Converting clinical teams’ unique expertise into their own GenAI based on open-source LLM finetuning, assisting experts to share their best-performing task-specific LLMs in clinical research networks for broader impact.

  4. Leveraging EHR data to create patient graphs and ML models for predictive healthcare, disease early detection, and prevention.

  5. Generating and using synthetic patient data to expedite healthcare AI research and training.

  6. Development of ML-LHS units for equitable healthcare.

 

Some Representative Research Results:

ChatGPT as copilot in Healthcare

We have introduced the concept of using ChatGPT as a copilot for LHS training and measured ChatGPT’s accuracy in providing scientific references (published by JAMA). A comprehensive benchmarking study of ChatGPT in symptom checking for a wide array of diseases has also been conducted (JAMIA paper). Additionally, we have developed a free new GenAI learning and research copilot tool.

Inclusive ML Models for ML-LHS

Our collaborators have applied the data-driven EHR ML approach proposed by us to develop both inclusive and practical ML models for risk prediction of two cancers and TIA, initializing ML-LHS units (see ML papers below). New quantitative distributions of health factors for two cancers and TIA were also generated from patient graphs using EHR data (see patient graph papers below).

First Synthetic ML-LHS Simulation

Our pioneering study simulating an ML-enabled LHS unit for lung cancer risk prediction using synthetic patient data demonstrated the benefits of continuous ML cycles on predictive care (published by Nature). We hypothesize that the ML-LHS approach will ensure the system-level changes required for successful deployment of AI models in most healthcare settings.   

 

 

Recent Publications:

 

  1. Anjun Chen, Erman Wu, Ran Huang, et al. Development of Lung Cancer Risk Prediction Machine Learning Models for Equitable Learning Health System: Retrospective Study. JMIR AI. 2024. (Accepted)

  2. Chen A, Liu L, Zhu T. Advancing the democratization of generative artificial intelligence in healthcare: a narrative review. J Hosp Manag Health Policy 2024;8:12. doi:10.21037/jhmhp-24-54

  3. Chen A, Liu Y, Chen W. Impact of Democratizing Artificial Intelligence: Using ChatGPT in Medical Education and Training. Academic Medicine. 2024:10.1097/ACM.0000000000005672. DOI:10.1097/ACM.0000000000005672

  4. Chen A, Chen DO, Tian L. Benchmarking the symptom-checking capabilities of ChatGPT for a broad range of diseases. J Am Med Inform Assoc. 2023:ocad245. doi:10.1093/jamia/ocad245.

  5. Chen A, Chen DO. Accuracy of Chatbots in Citing Journal Articles. JAMA Netw Open. 2023;6(8):e2327647. doi:10.1001/jamanetworkopen.2023.27647

  6. Wen J, Zhang T, Ye S, et al. Quantitative patient graph analysis for transient ischemic attack risk factor distribution based on electronic medical records. Heliyon. 2023;10(1):e22766. doi: 10.1016/j.heliyon.2023.e22766.

  7. Chen A. et al. Development of Inclusive and Practical Disease Risk Prediction Machine Learning Models for Implementation of Learning Health Systems. Poster presentation, AcademyHealth ARM 2023.

  8. Chen A, Lu R, Han R, et al. Building practical risk prediction models for nasopharyngeal carcinoma screening with patient graph analysis and machine learning. Cancer Epidemiol Biomarkers Prev. 2022:EPI-22-0792. doi:10.1158/1055-9965.EPI-22-0792.

  9. Chen A, Huang R, Wu E, et al. The Generation of a Lung Cancer Health Factor Distribution Using Patient Graphs Constructed From Electronic Medical Records: Retrospective Study. J Med Internet Res. 2022;24(11):e40361. doi:10.2196/40361.

  10. Chen A, Chen DO. Simulation of a machine learning enabled learning health system for risk prediction using synthetic patient data. Nature Sci. Rep. 2022;12(1):17917. doi:10.1038/s41598-022-23011-4.

  11. Chen A. A novel graph methodology for analyzing disease risk factor distribution using synthetic patient data. Healthcare Analytics. 2022;2:100084. doi:10.1016/j.health.2022.100084.

  12. Haendel MA, Chute CG, Bennett TD, et al; N3C Consortium. The National COVID Cohort Collaborative (N3C): Rationale, design, infrastructure, and deployment. J Am Med Inform Assoc. 2021;28(3):427–443. doi:10.1093/jamia/ocaa196. 

  13. Chen A, Shen B, Zhang R, et al. Developing new event-driven CBK network for global real-time clinical collaborations. Learning Health Systems. 2021;5(2):e10256. doi:10.1002/lrh2.10256.

  14. Chen A, et al. Feasibility study for implementation of the AI-powered Internet+ Primary Care Model (AiPCM) across hospitals and clinics in Gongcheng county, Guangxi, China. The Lancet. 2019;394(Supplement 1):S44. doi:10.1016/S0140-6736(19)32380-3.