Precision medicine as a control problem: Using simulation and deep\n reinforcement learning to discover adaptive, personalized multi-cytokine\n therapy for sepsis

2018/02/08 by Brenden K. Petersen, Petersen, Brenden K., Jiachen Yang +11
Biochemistry, Genetics and Molecular Biology · Medicine · #FOS: Biological sciences #FOS: Computer and information sciences #Influenza Virus Research Studies #Machine Learning (cs.LG) #Receptor Mechanisms and Signaling #Sepsis Diagnosis and Treatment #Tissues and Organs (q-bio.TO)

paper · pdf · doi:10.48550/arxiv.1802.10440

openalex publication_date 2018/02/08 · openalex created_date 2022/09/03 · openalex updated_date 2026/07/28

Abstract

Sepsis is a life-threatening condition affecting one million people per year\nin the US in which dysregulation of the body's own immune system causes damage\nto its tissues, resulting in a 28 - 50% mortality rate. Clinical trials for\nsepsis treatment over the last 20 years have failed to produce a single\ncurrently FDA approved drug treatment. In this study, we attempt to discover an\neffective cytokine mediation treatment strategy for sepsis using a previously\ndeveloped agent-based model that simulates the innate immune response to\ninfection: the Innate Immune Response agent-based model (IIRABM). Previous\nattempts at reducing mortality with multi-cytokine mediation using the IIRABM\nhave failed to reduce mortality across all patient parameterizations and\nmotivated us to investigate whether adaptive, personalized multi-cytokine\nmediation can control the trajectory of sepsis and lower patient mortality. We\nused the IIRABM to compute a treatment policy in which systemic patient\nmeasurements are used in a feedback loop to inform future treatment. Using deep\nreinforcement learning, we identified a policy that achieves 0% mortality on\nthe patient parameterization on which it was trained. More importantly, this\npolicy also achieves 0.8% mortality over 500 randomly selected patient\nparameterizations with baseline mortalities ranging from 1 - 99% (with an\naverage of 49%) spanning the entire clinically plausible parameter space of the\nIIRABM. These results suggest that adaptive, personalized multi-cytokine\nmediation therapy could be a promising approach for treating sepsis. We hope\nthat this work motivates researchers to consider such an approach as part of\nfuture clinical trials. To the best of our knowledge, this work is the first to\nconsider adaptive, personalized multi-cytokine mediation therapy for sepsis,\nand is the first to exploit deep reinforcement learning on a biological\nsimulation.\n

Citations

• A systematic review and meta-analysis of early goal-directed therapy for septic shock: the ARISE, ProCESS and ProMISe Investigators
• Playing Atari with Deep Reinforcement Learning
• Simulating cancer growth with multiscale agent-based modeling
• Continuous control with deep reinforcement learning
• Prioritized Experience Replay
• Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach

Related

• Model-Based Reinforcement Learning for Sepsis Treatment
• Learning Optimal Treatment Strategies for Sepsis Using Offline Reinforcement Learning in Continuous Space
• Offline reinforcement learning with uncertainty for treatment strategies in sepsis
• Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning
• Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation