2020/08/27 by Gabe Schamberg, Schamberg, Gabe, Marcus A. Badgeley +3
Medicine · #Anesthesia and Sedative Agents #Hemodynamic Monitoring and Therapy #Antibiotics Pharmacokinetics and Efficacy
paper · pdf · doi:10.48550/arxiv.2008.12333
Reinforcement Learning (RL) can be used to fit a mapping from patient state\nto a medication regimen. Prior studies have used deterministic and value-based\ntabular learning to learn a propofol dose from an observed anesthetic state.\nDeep RL replaces the table with a deep neural network and has been used to\nlearn medication regimens from registry databases. Here we perform the first\napplication of deep RL to closed-loop control of anesthetic dosing in a\nsimulated environment. We use the cross-entropy method to train a deep neural\nnetwork to map an observed anesthetic state to a probability of infusing a\nfixed propofol dosage. During testing, we implement a deterministic policy that\ntransforms the probability of infusion to a continuous infusion rate. The model\nis trained and tested on simulated pharmacokinetic/pharmacodynamic models with\nrandomized parameters to ensure robustness to patient variability. The deep RL\nagent significantly outperformed a proportional-integral-derivative controller\n(median absolute performance error 1.7% +/- 0.6 and 3.4% +/- 1.2). Modeling\ncontinuous input variables instead of a table affords more robust pattern\nrecognition and utilizes our prior domain knowledge. Deep RL learned a smooth\npolicy with a natural interpretation to data scientists and anesthesia care\nproviders alike.\n