Faculty, Staff and Student Publications
LLM-IE: A Python Package for Biomedical Generative Information Extraction With Large Language Models
Publication Date
4-1-2025
Journal
JAMIA Open
Abstract
Objectives: Despite the recent adoption of large language models (LLMs) for biomedical information extraction (IE), challenges in prompt engineering and algorithms persist, with no dedicated software available. To address this, we developed LLM-IE: a Python package for building complete IE pipelines.
Materials and methods: The LLM-IE supports named entity recognition, entity attribute extraction, and relation extraction tasks. We benchmarked it on the i2b2 clinical datasets.
Results: The sentence-based prompting algorithm resulted in the best 8-shot performance of over 70% strict F1 for entity extraction and about 60% F1 for entity attribute extraction.
Discussion: We developed a Python package, LLM-IE, highlighting (1) an interactive LLM agent to support schema definition and prompt design, (2) state-of-the-art prompting algorithms, and (3) visualization features.
Conclusion: The LLM-IE provides essential building blocks for developing robust information extraction pipelines. Future work will aim to expand its features and further optimize computational efficiency.
DOI
10.1093/jamiaopen/ooaf012
PMID
40078164
PMCID
PMC11901043
PubMedCentral® Posted Date
3-12-2025
PubMedCentral® Full Text Version
Post-print
Published Open-Access
yes