ELHS Institute

Newsletters

Your E-mail *

GenAI Powering the Democratization of AI and Medical Education in Systems-Based Practice

ELHS Newsletter 2024-02-29

Paving the Way for Global Health Equity with AI, ML, Data, and LHS (Learning Health Systems)

Dear Friends,

In response to the perspectives from UCSF professors on Generative AI in medical education, our letter to the editor has been published by Academic Medicine: “Impact of Democratizing Artificial Intelligence: Using ChatGPT in Medical Education and Training.” We emphasized that, given generative AI’s potential as both a learning tool and a research instrument for medical students and professionals, the democratization of AI could profoundly influence medical education and practice. Specifically,

Because ChatGPT has shown a high accuracy in disease diagnosis prediction across a wide range of diseases in our benchmarking studies, I believe the current GenAI tools are primed for students to use as a supplementary aid in clinical case learning.
Unlike traditional AI research in healthcare, which was confined to a select few researchers in the past, medical students or interns can now leverage publicly available generative AI models to analyze unstructured patient data for comparative effectiveness research.

Academic Medicine’s latest publication also includes perspectives on how to advance systems-based practice (SBP) competency in graduate medical education (GME). A new definition of learning health systems is provided in a survey study. I will discuss with ChatGPT about GenAI’s potential in accelerating the incorporation of LHS and SBP into GME.

I’m glad to see the consensus from the RAISE (Responsible AI for Social and Ethical Healthcare) conference was summarized in Nature Medicine as well as in NEJM AI. Toward responsible AI, studies from Stanford University show that large language models may propagate race-based medicine, and risk-based lung cancer screening may reduce health disparities. A JAMA interview reveals different types of AI model biases that should be addressed appropriately. Meta’s open-source LLaMA model can be fine-tuned to predict the diagnosis-related group for hospitalized patients more efficiently.

Best,

(Bookbrain. Source: Academic Medicine)

From Page Mill

^{Published papers, recent news, and significant events in a coherent narrative for the main topic.}

Defining the Learning Health Care System: An International Health System Leadership Perspective. Academic Medicine 99(2):p 215-220.

[2024/2] By surveying Academic Health Center leaders, the following LHS definition was developed: “A learning health system is a health care system in which clinical and care-related data are systematically integrated to catalyze discovery and implementation of new knowledge that benefits patients, the community, and the organization through improved outcomes.”

Breaking Down Silos Between Medical Education and Health Systems: Creating an Integrated Multilevel Data Model to Advance the Systems-Based Practice Competency. Academic Medicine 99(2):p 146-152.

[2024/2] The complexity of improving health in the United States and the rising call for outcomes-based physician training present unique challenges and opportunities for both graduate medical education (GME) and health systems. The development, study, and adoption of multilevel analytic approaches to GME are imperative to the successful operationalization of systems-based practice (SBP) and thereby imperative to GME’s social accountability in meeting societal needs for improved health.

To do no harm — and the most good — with AI in health care. Nat Med (2024).

[2024/2] Drawing from real-life scenarios and insights shared at the RAISE (Responsible AI for Social and Ethical Healthcare) conference organized by Harvard Medical School, this commentary highlights the critical need for AI in health care (AIH) to primarily benefit patients and address current shortcomings in health care systems such as medical errors and access disparities.

Considerations for addressing bias in artificial intelligence for health equity. npj Digit. Med. 6, 170 (2023).

[2023/9] We propose an expanded Total Product Lifecycle (TPLC) framework for healthcare AI/ML, describing the sources and impacts of undesirable bias in AI/ML systems in each phase, how these can be analyzed using appropriate metrics, and how they can be potentially mitigated. The goal of these “Considerations” is to educate stakeholders on how potential AI/ML bias may impact healthcare outcomes and how to identify and mitigate inequities; to initiate a discussion between stakeholders on these issues, in order to ensure health equity along the expanded AI/ML TPLC framework, and ultimately, better health outcomes for all.

Large language models propagate race-based medicine. npj Digit. Med. 6, 195 (2023).

[2023/10] LLMs are being proposed for use in the healthcare setting, with some models already connecting to electronic health record systems. However, this study shows that based on our findings, these LLMs could potentially cause harm by perpetuating debunked, racist ideas.

Risk Model–Based Lung Cancer Screening and Racial and Ethnic Disparities in the US. JAMA Oncol. 2023;9(12):1640–1648.

[2023/10] In this cohort study including 105 261 adults with a smoking history, the 2021 US Preventive Services Task Force guidelines yielded a large disparity in African American individuals whose eligibility-incidence ratio was 53% lower than that of White individuals. Under risk-based screening, the disparity between African American and White individuals was reduced—with minimal disparities observed across other minoritized groups—and improved screening efficiency across races and ethnicities.

Blind Spots, Shortcuts, and Automation Bias—Researchers Are Aiming to Improve AI Clinical Models. JAMA.

[2024/2] Automation bias can occur when clinicians trust and rely on AI systems even when they might be incorrect or biased. How can AI-human collaboration, particularly in clinical settings, help avoid some of the blind spots in AI? And what happens when AI models get it right, but only by using shortcuts that could undermine their accuracy for different patients? Hear the answers from Professor Jenna Wiens at the University of Michigan in the recent interview conducted by JAMA Editor-in-Chief. (Audio and Video available)

Measuring the Impact of AI in the Diagnosis of Hospitalized Patients: A Randomized Clinical Vignette Survey Study. JAMA. 2023;330(23):2275–2284.

[2023/12] Although standard AI models improve diagnostic accuracy, systematically biased AI models reduced diagnostic accuracy, and commonly used image-based AI model explanations did not mitigate this harmful effect.

DRG-LLaMA : tuning LLaMA model to predict diagnosis-related group for hospitalized patients. npj Digit. Med. 7, 16 (2024).

[2024/1] In the U.S. inpatient payment system, the Diagnosis-Related Group (DRG) is pivotal, but its assignment process is inefficient. The study introduces DRG-LLaMA, an advanced large language model (LLM) fine-tuned on clinical notes to enhance DRGs assignment. This model surpassed the performance of prior leading models in DRG prediction, showing a relative improvement of 40.3% and 35.7% in macro-averaged F1 score compared to ClinicalBERT and CAML, respectively.

In the Eyes of GPTs

^{GPTs are becoming an integral part of our society. AJ talks with GenAI to understand how machines view the main topics.}

ChatGPT, Let's talk about medical education, health systems and AI. The complexity of improving health in the United States and the rising call for outcomes-based physician training present unique challenges and opportunities for both graduate medical education (GME) and health systems. GME programs have been particularly challenged to implement systems-based practice (SBP) as a core physician competency and educational outcome. What does systems-based practice mean?

ChatGPT:

Systems-based practice (SBP) is one of the six core competencies that the Accreditation Council for Graduate Medical Education (ACGME) requires medical residents to develop during their training. It emphasizes the importance of understanding and effectively working within the healthcare system to provide optimal patient care. Specifically, SBP involves the ability of healthcare providers to:

Understand the Context of Healthcare Delivery: This includes recognizing how patient care and clinical practices are influenced by the broader healthcare system, including types of health care systems, health insurance, operational practices, and healthcare policies.
Collaborate Across Healthcare Settings: Healthcare providers must work effectively with other professionals within the healthcare system, including those in different specialties, departments, and healthcare facilities, to ensure coordinated and comprehensive patient care.
Optimize Resources: SBP requires the efficient utilization of resources to provide the best possible care to patients. This includes understanding and applying principles of cost-effectiveness and resource allocation that do not compromise quality of care.
Advocate for Quality Patient Care: Healthcare providers should advocate for their patients within the healthcare system, ensuring access to the best possible care, navigating systemic barriers, and addressing social determinants of health that may affect patient outcomes.
Improve Patient Care Systems: SBP also involves actively participating in or leading efforts to improve healthcare delivery systems, such as quality improvement initiatives, that enhance patient safety, efficiency, and outcomes.
Understand the Role in Healthcare Delivery: Recognizing one's role within the system, including how individual and team actions affect patient care and outcomes, and how they can contribute to the overall functioning and improvement of the system.

Implementing SBP as a core competency aims to prepare new physicians not only to provide excellent individual patient care but also to contribute positively to the healthcare system as a whole. By understanding and navigating the complexities of the healthcare system, physicians can help to streamline processes, improve patient experiences, and ultimately enhance health outcomes.

Why is it challenged for GME to implement SBP? What are the challenges?

ChatGPT:

Implementing systems-based practice (SBP) within Graduate Medical Education (GME) faces several challenges, primarily due to the complexity of healthcare systems and the traditional focus of medical training. Here are some of the key challenges:

Complexity of Healthcare Systems: Healthcare systems are complex and vary widely across different regions, institutions, and even departments within the same hospital. The diversity in how healthcare is delivered makes it challenging to teach a one-size-fits-all approach to SBP. Trainees might find it difficult to apply SBP principles universally across different systems.
Cultural and Structural Barriers: Medical culture traditionally emphasizes individual responsibility and clinical skills over systems thinking. This focus can create cultural and structural barriers to implementing SBP, as both educators and trainees may not fully recognize the importance of understanding and improving the systems aspects of healthcare.
Lack of Standardized Curriculum and Assessment Tools: There is a lack of standardized curriculum and effective assessment tools to measure competency in SBP across GME programs. Developing and integrating these tools into existing programs can be challenging, especially when trying to tailor them to the specific needs and contexts of different specialties and healthcare settings.
Insufficient Training and Resources for Faculty: Faculty members who are responsible for teaching SBP often lack the necessary training or resources themselves. They may be experts in clinical care but not necessarily in healthcare systems, quality improvement, or cost-effective care. This gap can make it difficult to effectively teach and mentor trainees in SBP.
Limited Exposure and Real-World Experience: Trainees may have limited opportunities to engage with the healthcare system outside of their direct patient care responsibilities. Real-world experience in navigating and improving healthcare systems is crucial for developing SBP competencies, but finding time for these experiences within the demanding schedules of GME programs can be difficult.
Evolving Healthcare Policies and Technologies: The constant evolution of healthcare policies, regulations, and technologies means that SBP teaching materials and strategies need regular updating to remain relevant. Keeping up with these changes requires ongoing effort and adaptation by GME programs.
Integration into Clinical Practice: Effectively integrating SBP into the daily clinical practice of trainees is challenging. Trainees must learn to balance the immediate demands of patient care with the broader considerations of systems-based practice, often in high-pressure environments.

Addressing these challenges requires innovative approaches to medical education, including curricular reform, faculty development, and creating more opportunities for interdisciplinary learning and collaboration. Encouraging a culture shift towards valuing systems thinking and continuous improvement in healthcare can also support the integration of SBP into GME.

Generative AI is expected to revolutionize GME and clinical care. Will GenAI help implementing SBP into GME? And how?

ChatGPT:

The implementation of Systems-Based Practice (SBP) into Graduate Medical Education (GME) has faced challenges, primarily due to the complexity of healthcare systems, the traditional structure of medical education, and the evolving nature of healthcare delivery. Integrating SBP requires not only a shift in curriculum but also a cultural change within institutions, which can be difficult to achieve. However, Generative AI (GenAI) offers promising solutions to these challenges and can significantly aid in integrating SBP into GME. Here's how:

Enhancing Learning and Simulation

Realistic Simulations: GenAI can create sophisticated, realistic simulations and virtual scenarios that mimic complex healthcare systems, allowing residents to navigate these systems without risk to real patients. These simulations can include elements like managing resources, coordinating with healthcare teams, and navigating health policies, offering hands-on experience with the systems-based aspect of care.
Personalized Learning: AI can tailor educational content to match the learning pace and style of individual residents, ensuring that they grasp the nuances of SBP effectively. It can identify learning gaps in real-time and adjust the curriculum accordingly.

Facilitating Access to Information

Rapid Information Retrieval: GenAI can provide quick access to up-to-date guidelines, research findings, and policy changes that are relevant to SBP, helping residents to make informed decisions and understand the implications of their actions within the healthcare system.
Knowledge Integration: By integrating knowledge from various sources, AI can help residents understand how different aspects of the healthcare system interact. This can include insurance policies, patient rights, and the latest in healthcare innovation, all of which are critical for effective SBP.

Improving Collaboration and Communication

Interprofessional Education: AI-driven platforms can facilitate interprofessional education by connecting residents with peers in other healthcare domains (e.g., nursing, pharmacy, social work) in virtual settings, promoting teamwork and understanding of each profession's role in patient care and system navigation.
Communication Skills: Through interactive AI modules, residents can improve their communication skills, which are essential for negotiating, coordinating, and advocating within the healthcare system.

Supporting Quality Improvement Projects

Data Analysis and Management: AI tools can analyze vast amounts of data to identify trends, inefficiencies, and areas for improvement within healthcare systems. Residents can use these insights to develop and implement quality improvement projects, a key component of SBP.
Monitoring and Feedback: AI can monitor the outcomes of implemented projects in real-time, providing immediate feedback that can be used to make adjustments and improve patient care and system efficiency.

Overcoming Institutional Barriers

Cultural Change Facilitation: By demonstrating the effectiveness of SBP-focused initiatives and providing evidence-based outcomes, AI can help overcome resistance to change within institutions.
Cost-Effectiveness: AI can also help to highlight the cost-effectiveness of integrating SBP into GME, persuading stakeholders of its value.

In conclusion, while challenges exist in integrating SBP into GME, GenAI offers innovative tools and methodologies to overcome these barriers. By enhancing learning experiences, facilitating access to information, improving collaboration, supporting quality improvement, and helping to navigate institutional challenges, GenAI can play a crucial role in effectively embedding SBP into medical education and practice.

SBP shares some key concepts with the learning health system (LHS) vision proposed by NAM. I think, after healthcare is delivered through LHS, it would help physicians learn and practice SBP. The key is to use GenAI to make system-level changes in care delivery so that some of requirements of SBP are augmented by AI, saving time for physicians. What do you think?

ChatGPT:

Integrating Systems-Based Practice (SBP) into Graduate Medical Education (GME) within a Learning Health System (LHS) framework, as proposed by the National Academy of Medicine (NAM), and leveraging Generative AI (GenAI) offers a compelling approach to enhancing healthcare delivery and education. Your vision acknowledges the potential synergies between SBP, LHS, and GenAI in driving system-level changes in care delivery. Here are several ways this integration could manifest and support the implementation of SBP:

Augmentation of Clinical Decision-Making

GenAI can support SBP by providing real-time, evidence-based recommendations that are tailored to the patient's context, thereby improving clinical decision-making. This capability aligns with the LHS's goal of continuous learning and adaptation. By analyzing vast amounts of data from diverse sources, AI can identify patterns and suggest interventions that may not be immediately obvious to human clinicians, enhancing the quality of care and ensuring more efficient use of healthcare resources.

Streamlining Administrative Processes

The administrative burden on healthcare providers is a significant barrier to implementing SBP effectively. GenAI can automate many time-consuming administrative tasks, such as documentation, billing, and compliance with healthcare regulations. This automation can free up physicians' time, allowing them to focus more on direct patient care and system-level improvements, key aspects of SBP.

Enhancing Interdisciplinary Collaboration

An LHS promotes interdisciplinary collaboration, which is central to SBP. GenAI can facilitate this collaboration by creating shared platforms for communication and data exchange, breaking down silos between different departments or specialties. By providing a unified view of patient data and care processes, AI can help teams work together more effectively, coordinating care and improving patient outcomes.

Facilitating Quality Improvement Initiatives

GenAI can analyze data on patient outcomes, healthcare processes, and performance metrics to identify areas for improvement. This capability supports the LHS's emphasis on continuous improvement and learning from every patient encounter. AI-driven insights can guide quality improvement initiatives, helping healthcare organizations implement changes that enhance efficiency, patient safety, and care quality—key components of SBP.

Supporting Personalized and Patient-Centered Care

LHS emphasizes personalized care, a principle that aligns with SBP's focus on understanding and navigating the healthcare system for the benefit of patients. GenAI can analyze data from various sources, including electronic health records, genomic information, and social determinants of health, to tailor care plans to the individual needs of each patient. This personalized approach can improve patient engagement and outcomes, reflecting the goals of both SBP and LHS.

Training and Education

GenAI can revolutionize medical education by providing simulations, virtual patients, and data-driven insights, allowing residents to learn and practice SBP in a risk-free environment. This approach can help bridge the gap between theoretical knowledge and practical application, preparing residents to effectively navigate and contribute to the improvement of the healthcare system.

In conclusion, the integration of SBP, LHS, and GenAI holds great promise for transforming healthcare delivery and education. By leveraging GenAI to make system-level changes in care delivery, healthcare organizations can address many of the challenges associated with implementing SBP, ultimately leading to more efficient, effective, and patient-centered care.

I like your summary. Let's hope GenAI will accelerate this transformation. Thanks.

ChatGPT:

You're welcome! I'm glad you found the summary helpful. Indeed, the potential for Generative AI to accelerate the transformation of healthcare through improved systems-based practice and learning health systems is very exciting. As this technology continues to evolve, it will be fascinating to see how it shapes the future of medical education and healthcare delivery. If you have any more questions or need further information on this topic or anything else, feel free to ask.

~ the end ~