‹ Back

Stanford research showed ChatGPT could help medical students

 

ELHS NEWSLETTER 2023-11-1

Paving the Way for Global Health Equity with GenAI, ML, and LHS (Learning Health Systems)

| Subscribe | All Issues |

 

 

MAIN TOPIC:

GENERATIVE AI IN MEDICAL EDUCATION AND TRAINING

• Google reported its medical GenAI chatbot passed medical licensing exam tests.
• UCSF educators have outlined the potential impact of GenAI in medical education.
• Stanford researchers examined GenAI vs. medical student performance on medical exams.
• GenAI transformation requires system-level changes, including learning health systems.

~

 

(Source: Stanford University)

 

 

From Page Mill

^{Published papers, recent news, and significant events in a coherent narrative for the main topic.} 

 

Singhal K, Azizi S, Tu T, et al. Large language models encode clinical knowledge.

Nature. 2023;620(7972):172-80.

[2023/8] “On the MedQA dataset consisting of USMLE-style questions with 4 options, Google’s Flan-PaLM 540B model achieved a multiple-choice question accuracy of 67.6%, surpassing the DRAGON model by 20.1%. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians.” 

 

Boscardin CK, Gin B, Golde PB, Hauer KE. ChatGPT and Generative Artificial Intelligence for Medical Education: Potential Impact and Opportunity.

Acad Med. 2023. doi: 10.1097/ACM.0000000000005439.

[2023/8] UCSF educators suggested: “As generative AI tools continue to expand, educators need to increase their AI literacy through education and vigilance around new advances in the technology and serve as stewards of AI literacy to foster social responsibility and ethical awareness around the use of AI.”

 

Feng S, Shen Y. ChatGPT and the Future of Medical Education.

Acad Med. 2023;98(8):867-868. doi: 10.1097/ACM.0000000000005242.

[2023/8] In a letter to editor, Chinese authors stated: “Teaching students how to make informed, clinical decisions is one of the biggest challenges in medical education. With ChatGPT, medical students can receive real-time feedback on their decision-making processes and learn from the experiences of others. This can help improve their critical thinking skills and equip them with the knowledge and confidence they need to make effective clinical decisions once they enter the workforce.”

 

Strong E, DiGiammarino A, Weng Y, et al. Chatbot vs Medical Student Performance on Free-Response Clinical Reasoning Examinations.

JAMA Intern Med. 2023;183(9):1028-1030. doi: 10.1001/jamainternmed.2023.2909.

[2023/7] “Using clinical reasoning final examinations given to first- and second-year students at Stanford School of Medicine, Stanford researchers compared performance of students vs 2 chatbot models. The model on average scored more than four points higher than students on this case-report portion of the exam. “

[Also see Stanford news]

~

 

 

 

In the Eyes of GPTs

^{GPTs are becoming an integral part of our society. AJ talks with GenAI to understand how machines view healthcare.}

 

 Given that GenAI is emerging as a significant disruptor, it will likely necessitate some structural changes in the near future. Do you foresee any structural modifications in medical education that medical schools should contemplate?

 

 Certainly! The rise of Generative AI (GenAI) and other advanced technologies heralds a paradigm shift in many fields, including medical education. Embracing these changes necessitates structural adjustments in the curriculum, teaching methodologies, and even the infrastructure of medical education. Here are some structural changes medical schools might consider:

 

1. Curriculum Design:

Integrated AI modules: Courses that specifically address AI in healthcare, covering not only the technical aspects but also the ethical, clinical, and social implications.

Regular updates: Given the rapid pace of AI advancement, curricula would benefit from frequent updates to ensure relevance.

 

2. Interdisciplinary Collaborations:

With AI's integration, there's a need for collaboration between medical faculties and departments like computer science, data analytics, and engineering. This can foster a comprehensive understanding of AI's applications in medicine.

 

3. Infrastructure and Resources:

Investment in Tech: Ensure that the school's technology infrastructure can support the latest AI tools. This might mean more advanced computing capabilities, cloud solutions, or even dedicated AI labs.

Digital Libraries: Access to extensive digital resources that continuously update with the latest research, facilitated by AI algorithms that can curate and recommend pertinent content.

 

4. Assessment and Evaluation:

Develop new assessment methodologies that test a student's ability to work alongside AI tools. Traditional exams might need to be supplemented with real-world problem-solving scenarios using AI.

 

5. Clinical Training Adjustments:

As AI starts playing a more prominent role in clinical settings, students' clinical rotations should expose them to these tools in action, teaching them to use AI aids in diagnostics, treatment planning, and patient interactions.

 

6. Faculty Development:

Continuous training for faculty to stay abreast with the latest in AI so they can effectively guide students. This also includes training in the pedagogical methods most effective for teaching AI concepts.

 

7. Ethical Training:

Dedicated modules that delve deep into the ethical challenges posed by AI in healthcare, preparing students to navigate these complexities in their future careers.

 

8. Collaborative Learning Platforms:

Platforms where students, professionals, and AI can collaborate on problem-solving, research, or clinical scenarios, simulating real-world multi-disciplinary teamwork.

 

9. Research Opportunities:

Encourage and provide avenues for students to engage in AI-related medical research. This could be in the form of special projects, internships, or partnerships with tech companies and research institutes.

 

10. Feedback Mechanisms:

Robust systems to gather feedback on AI tools' effectiveness, which can then inform iterative improvements in how AI is incorporated.

 

11. Flexibility in Learning Paths:

With the vast amount of information AI can provide, there's an opportunity to offer more personalized learning paths. Students could delve deeper into specific areas of interest, guided by AI recommendations.

 

12. Global Collaborations:

AI transcends borders. Collaborations with international institutions can allow for a pooling of resources, research, and best practices in integrating AI into medical education.

 

While these structural changes can pave the way for a more AI-integrated medical education system, it's vital to approach this transition thoughtfully. The human aspect of medicine – empathy, ethics, interpersonal skills – remains paramount, and the goal should be to use AI to augment, not replace, these fundamental qualities.

 

 You are familiar with the Learning Health Systems (LHS) vision from the National Academy of Medicine, aren’t you? While the term "learning" in LHS encompasses all forms of continuous learning, I believe that it should specifically emphasize machine learning or the application of AI in order to automate and enhance efficiency in care delivery. Given this, do you think "learning health systems" should be added as a new structural change suggestion to the list you provided earlier?

 

 Yes, I'm familiar with the Learning Health Systems (LHS) vision from the National Academy of Medicine. The vision emphasizes a cycle of continuous improvement, where new knowledge is captured and turned into practice in a systematic manner.

 

If we're looking at emphasizing machine learning and AI within the LHS context to automate and increase efficiency in care delivery, then absolutely, it can be considered a significant structural change for medical education. Here's how you might phrase that addition:

 

13. Integration of Learning Health Systems (LHS) with a Focus on AI:

Curriculum Integration: Introduce the concept of LHS early in the curriculum and intertwine it with AI and machine learning concepts. Students should understand how continuous learning from data can inform better patient care.

• Practical Exposure: As with clinical rotations in hospitals, students should have rotations or modules where they observe and interact with AI-driven LHS in action. This could be in hospitals that have adopted such systems or virtual simulations.
• Research and Development: Encourage students to participate in projects that aim to develop or enhance LHS using AI. This could be in the form of capstone projects, internships, or collaborative research with tech firms and health institutions.
• Ethical and Privacy Considerations: Given the vast amounts of data LHS deals with, specific modules should address the ethical use of patient data, potential biases in AI algorithms, and ensuring patient privacy and data security.
• Interdisciplinary Collaboration: Strengthen ties with computer science, data analytics, and engineering departments. Building an effective AI-driven LHS requires a confluence of medical knowledge and tech expertise.

 

Incorporating LHS with a focus on AI as a structural change in medical education is forward-thinking. It prepares future healthcare professionals to work in an environment where data-driven decision-making is the norm, and efficiency and improved patient outcomes are continuously sought after.

 

 Perfect. I like the way you frame it. Yes, educating and training AI and LHS together is forward-thinking. With AHRQ and PCORI investing $50M to establish LHS training centers in academic medical centers this year, I hope they heed your insights.

 

  Thank you for the kind words! It's truly encouraging to hear about substantial investments from organizations like AHRQ and PCORI to foster the growth of LHS in academic medical centers. Embedding AI and LHS education at the foundational level of medical training will no doubt pave the way for a more data-driven, efficient, and patient-centric healthcare system in the future.

 

Should you be involved in any initiatives or have a platform to discuss these ideas, I hope the information provided serves as a valuable resource. If more insights, details, or perspectives are needed in the future, please don't hesitate to reach out. It's exciting to think about the transformative potential of these technologies in healthcare, and I'm here to help in any way I can!

 

~~~