Faculty, Staff and Student Publications
Publication Date
9-25-2023
Journal
Journal of the American Medical Informatics Association
Abstract
BACKGROUND: Alzheimer's disease (AD) is a progressive neurological disorder with no specific curative medications. Sophisticated clinical skills are crucial to optimize treatment regimens given the multiple coexisting comorbidities in the patient population.
OBJECTIVE: Here, we propose a study to leverage reinforcement learning (RL) to learn the clinicians' decisions for AD patients based on the longitude data from electronic health records.
METHODS: In this study, we selected 1736 patients from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. We focused on the two most frequent concomitant diseases-depression, and hypertension, thus creating 5 data cohorts (ie, Whole Data, AD, AD-Hypertension, AD-Depression, and AD-Depression-Hypertension). We modeled the treatment learning into an RL problem by defining states, actions, and rewards. We built a regression model and decision tree to generate multiple states, used six combinations of medications (ie, cholinesterase inhibitors, memantine, memantine-cholinesterase inhibitors, hypertension drugs, supplements, or no drugs) as actions, and Mini-Mental State Exam (MMSE) scores as rewards.
RESULTS: Given the proper dataset, the RL model can generate an optimal policy (regimen plan) that outperforms the clinician's treatment regimen. Optimal policies (ie, policy iteration and Q-learning) had lower rewards than the clinician's policy (mean -3.03 and -2.93 vs. -2.93, respectively) for smaller datasets but had higher rewards for larger datasets (mean -4.68 and -2.82 vs. -4.57, respectively).
CONCLUSIONS: Our results highlight the potential of using RL to generate the optimal treatment based on the patients' longitude records. Our work can lead the path towards developing RL-based decision support systems that could help manage AD with comorbidities.
Keywords
Humans, Alzheimer Disease, Memantine, Cholinesterase Inhibitors, Artificial Intelligence, Learning, Alzheimer’s disease, treatment learning, reinforcement learning, Q-learning, policy iteration, reward, action, policy
DOI
10.1093/jamia/ocad135
PMID
37463858
PMCID
PMC10531148
PubMedCentral® Posted Date
7-18-2023
PubMedCentral® Full Text Version
Post-print
Published Open-Access
yes
