‹ Back

ChatGPT passed medical licensing exams

 

ELHS NEWSLETTER 2023-10-15

Paving the Way for Global Health Equity with GenAI, ML, and LHS (Learning Health Systems)

| Subscribe | All Issues |

 

 

MAIN TOPIC:

GENAI IN MEDICAL EDUCATION AND TRAINING

• A Generative AI (GenAI) chatbot, ChatGPT, has passed medical licensing exams.

• JMIR Medical Education has issued a call for papers to explore the role of GenAI in medical education.

• GenAI is transforming medical education.

 ~

 

 

 

From Page Mill

^{Published papers, recent news, and significant events in a coherent narrative for the main topic.} 

 

Eysenbach G. The Role of ChatGPT, Generative Language Models, and Artificial Intelligence in Medical Education: A Conversation With ChatGPT and a Call for Papers.

JMIR Med Educ 2023;9:e46885. doi: 10.2196/46885

[2023/3] The publisher of JMIR Publications interviewed ChatGPT about how to use chatbots in medical education. Due to the impact of this new technology on medical education, JMIR Medical Education is launching a call for papers for a new e-collection and theme issue. 

 

Gilson A, Safranek CW, Huang T, et al. How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment.

JMIR Med Educ 2023;9:e45312. doi: 10.2196/45312

[2023/2] Yale researchers showed: “By performing at a greater than 60% threshold on the NBME-Free-Step-1 data set, the model achieves the equivalent of a passing score for a third-year medical student. Additionally, we highlight ChatGPT’s capacity to provide logic and informational context across the majority of answers. These facts taken together make a compelling case for the potential applications of ChatGPT as an interactive medical education tool to support learning.”

 

Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models.

PLOS Digit Health. 2023;2(2):e0000198. doi: 10.1371/journal.pdig.0000198.

[2023/2] “We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.”

 ~

 

 

In the Eyes of GPTs

^{GPTs are becoming an integral part of our society. AJ talks with GenAI to understand how machines view healthcare.}

 

  Generative AI, like ChatGPT, is quite distinct from traditional AI that requires structural data input. With the rise of GenAI in 2023, it seems a new realm of applications for GenAI has opened up. What novel impacts or integration methods will ChatGPT introduce in medical education?

 

  Generative AI models, such as ChatGPT and its subsequent iterations, represent a major shift in AI capabilities. Unlike traditional AI that operates within narrow confines and requires structured data, Generative AI can process and generate human-like text, enabling more natural and nuanced interactions. In the context of medical education, this opens up several unique opportunities:

 

1. Interactive Case-Based Learning:

ChatGPT can simulate patient-doctor interactions in real-time, allowing students to "interview" a virtual patient, obtain a history, and make diagnostic decisions based on the conversation.

 

2. 24/7 Virtual Tutor:

Students can pose complex medical questions to ChatGPT at any time, facilitating continuous learning. The AI can provide explanations, clarify doubts, or even suggest further reading on the topic.

 

3. Role-Playing and Ethical Scenarios:

Generative AI can simulate various characters in medical scenarios, such as patients, family members, or healthcare professionals. This can be used for training in communication skills, breaking bad news, or navigating ethical dilemmas.

 

4. Multimodal Learning Platforms:

Generative AI can be integrated into platforms that combine text, visuals, and audio, catering to various learning styles. For example, explaining a concept verbally while also generating relevant diagrams or animations.

 

5. Continuous Knowledge Updates:

ChatGPT can be programmed to stay updated with the latest medical research and guidelines. It can thus offer current knowledge without the need for manual updates, ensuring students have access to the most recent information.

 

6. Customized Learning Trajectories:

By analyzing a student's interactions, questions, and responses over time, the AI can identify areas of strength and weakness, adapting content delivery to individual needs.

 

7. Peer Interaction Simulations:

ChatGPT can simulate the role of another medical student or a professional, allowing for peer discussion simulations on case studies, research topics, or clinical decision-making.

 

8. Language and Cultural Training:

Given its language capabilities, Generative AI can help medical students learn medical terminologies in various languages or understand cultural nuances in patient interactions, crucial for doctors planning to work in diverse environments.

 

9. Exam Preparation and Mock Tests:

ChatGPT can generate questions and case scenarios for students to practice, offering instant feedback on their responses. It can simulate various exam formats, from multiple-choice questions to open-ended clinical scenarios.

 

10. Research Assistance:

For students engaged in research, Generative AI can assist in literature reviews, summarizing articles, or even suggesting research methodologies or statistical approaches.

 

11. Integrating Patient Voices:

Generative models can be trained on real patient stories, experiences, and feedback, ensuring that the "patient voice" is integrated into medical education. This can be particularly valuable in understanding patient perspectives on diseases, treatments, and healthcare experiences.

 

It's worth noting that while Generative AI offers these potential benefits, its integration should be approached with caution. Medical educators need to ensure that the information provided by the AI is accurate and relevant. Additionally, an over-reliance on AI could risk sidelining essential human aspects of medical training, such as hands-on clinical experience and human mentorship. As with any tool, its effectiveness will depend on how it's used within the broader educational framework.

 

 Thanks for the extensive list of ChatGPT applications in medical education. You are right; with the incredible capability of genuine natural language interactions with humans come both advantages and drawbacks. What would you recommend medical educators proactively do to prevent potential pitfalls associated with GenAI?

 

 Absolutely, like any powerful technology, Generative AI's potential benefits come with associated risks. Medical educators can take several steps to harness the positives while minimizing potential dangers:

 

1. Training & Familiarization:

Educators themselves should be trained on the capabilities and limitations of Generative AI. Understanding the system ensures they can guide students on effective and safe usage.

 

2. Continuous Validation:

Regularly validate the information provided by the AI system against trusted sources. Medical knowledge evolves, and it's crucial to ensure that what the AI provides remains accurate and up-to-date.

 

3. Emphasize Human Judgement:

Reinforce the idea that AI is a supplementary tool. Critical thinking, clinical judgment, and the human touch are irreplaceable. Encourage students to use AI as a reference or guide, not a definitive answer.

 

4. Address Ethical Concerns:

Discuss the ethical implications of using AI in healthcare with students. Topics might include data privacy, potential biases in AI algorithms, and the dehumanization of medicine.

 

5. Implement Feedback Mechanisms:

Allow students and educators to provide feedback on AI interactions. This feedback can be used to identify areas where the AI may be misleading or incorrect.

 

6. Maintain a Multi-Modal Learning Approach:

Ensure that AI tools are just one component of a diverse learning strategy. Traditional methods, hands-on experience, and human interaction should remain central.

 

7. Encourage Skepticism:

Just as medical students are taught to critically appraise medical literature, they should be trained to question and verify AI-generated content.

 

8. Set Boundaries:

Define clear use-case scenarios for students. For instance, while the AI might be great for explaining pathophysiology, it might not be the best tool for teaching empathy or breaking bad news.

 

9. Monitor Dependence:

It's essential to ensure that students don't become overly reliant on AI for their learning or decision-making. Periodic assessments without the aid of AI can ensure that foundational knowledge and skills are retained.

 

10. Privacy Measures:

Ensure that any AI tools used respect student data privacy. This is particularly vital if AI systems are used to assess or provide feedback on student performance.

 

11. Diversify AI Training Data:

If customizing AI for specific educational purposes, ensure the data it's trained on is diverse and comprehensive to prevent biases and ensure a broad knowledge base.

 

12. Foster Open Discussions:

Creating an open environment where students can discuss their experiences, reservations, and concerns about using AI in their education can provide educators with valuable insights into potential pitfalls.

 

Incorporating these measures and maintaining a proactive approach can help medical educators integrate Generative AI into their curricula safely and effectively. The goal is to use AI to enhance learning and not to replace the foundational tenets of medical education.

 

~~~