Center for Medical Ethics and Health Policy Staff Publications
Publication Date
1-1-2024
Journal
Frontiers in Digital Health
DOI
10.3389/fdgth.2024.1426057
PMID
39512857
PMCID
PMC11540825
PubMedCentral® Posted Date
10-24-2024
PubMedCentral® Full Text Version
Post-print
Published Open-Access
yes
Keywords
artificial intelligence, large language models, expert panel discussion, electronic health records, implementation science
Abstract
Objective: Artificial intelligence (AI) is revolutionizing healthcare, but less is known about how it may facilitate methodological innovations in research settings. In this manuscript, we describe a novel use of AI in summarizing and reporting qualitative data generated from an expert panel discussion about the role of electronic health records (EHRs) in implementation science.
Materials and methods: 15 implementation scientists participated in an hour-long expert panel discussion addressing how EHRs can support implementation strategies, measure implementation outcomes, and influence implementation science. Notes from the discussion were synthesized by ChatGPT (a large language model-LLM) to generate a manuscript summarizing the discussion, which was later revised by participants. We also surveyed participants on their experience with the process.
Results: Panelists identified implementation strategies and outcome measures that can be readily supported by EHRs and noted that implementation science will need to evolve to assess future EHR advancements. The ChatGPT-generated summary of the panel discussion was generally regarded as an efficient means to offer a high-level overview of the discussion, although participants felt it lacked nuance and context. Extensive editing was required to contextualize the LLM-generated text and situate it in relevant literature.
Discussion and conclusions: Our qualitative findings highlight the central role EHRs can play in supporting implementation science, which may require additional informatics and implementation expertise and a different way to think about the combined fields. Our experience using ChatGPT as a research methods innovation was mixed and underscores the need for close supervision and attentive human involvement.
Included in
Artificial Intelligence and Robotics Commons, Bioethics and Medical Ethics Commons, Health Policy Commons, Medical Sciences Commons