Ctrl-DNA: Controllable Cell-Type-Specific Regulatory DNA Design via Constrained RL

2025/05/26 by Xingyu Chen, Shihao Ma, Chen, Xingyu +7 · 1 voice · 2 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Engineering · #Artificial Intelligence (cs.AI) #DNA and Biological Computing #Evolutionary Algorithms and Applications #FOS: Biological sciences #FOS: Computer and information sciences #Genomics (q-bio.GN) #Machine Learning (cs.LG) #Modular Robots and Swarm Intelligence #cs.AI #cs.LG #q-bio.GN

paper · pdf · doi:10.48550/arxiv.2505.20578

openalex publication_date 2025/05/26 · arxiv published 2025/05/26 · arxiv updated 2025/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Designing regulatory DNA sequences that achieve precise cell-type-specific gene expression is crucial for advancements in synthetic biology, gene therapy and precision medicine. Although transformer-based language models (LMs) can effectively capture patterns in regulatory DNA, their generative approaches often struggle to produce novel sequences with reliable cell-specific activity. Here, we introduce Ctrl-DNA, a novel constrained reinforcement learning (RL) framework tailored for designing regulatory DNA sequences with controllable cell-type specificity. By formulating regulatory sequence design as a biologically informed constrained optimization problem, we apply RL to autoregressive genomic LMs, enabling the models to iteratively refine sequences that maximize regulatory activity in targeted cell types while constraining off-target effects. Our evaluation on human promoters and enhancers demonstrates that Ctrl-DNA consistently outperforms existing generative and RL-based approaches, generating high-fitness regulatory sequences and achieving state-of-the-art cell-type specificity. Moreover, Ctrl-DNA-generated sequences capture key cell-type-specific transcription factor binding sites (TFBS), short DNA motifs recognized by regulatory proteins that control gene expression, demonstrating the biological plausibility of the generated sequences.

Citations

• Regulatory DNA sequence Design with Reinforcement Learning
• Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design
• Pareto Set Learning for Multi-Objective Reinforcement Learning
• Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• Latent Diffusion Model for DNA Sequence Generation
• Confronting Reward Model Overoptimization with Constrained RLHF
• HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
• Dirichlet Diffusion Score Model for Biological Sequence Generation
• Scaling Laws for Reward Model Overoptimization
• Biological Sequence Design with GFlowNets
• JASPAR 2022: the 9th release of the open-access database of transcription factor binding profiles
• Towards an Understanding of Default Policies in Multitask Policy Optimization
• Effective gene expression prediction from sequence by integrating long-range interactions
• AdaLead: A simple and robust adaptive greedy search algorithm for sequence design
• Responsive Safety in Reinforcement Learning by PID Lagrangian Methods
• IPO: Interior-point Policy Optimization under Constraints
• Conditioning by adaptive sampling for robust design
• Reward Constrained Policy Optimization
• Maximum a Posteriori Policy Optimisation
• Proximal Policy Optimization Algorithms
• Constrained Policy Optimization
• High-Dimensional Continuous Control Using Generalized Advantage\n Estimation
• Trust Region Policy Optimization
• Practical Bayesian Optimization of Machine Learning Algorithms

Cited by

• Biological Reasoning-Informed Regression for Interpretable Regulatory DNA Activity Prediction
• Conditional Monte Carlo Tree Diffusion for Designing Cell-Type-Specific and Biologically Faithful Regulatory DNA

Discussions

• From preprint: "Here, we introduce Ctrl-DNA, a novel constrained reinforcement learning (RL) framework tailored for designing regulatory DNA sequences with controllable cell-type specificity" Code: gi [bsky, 0 points, 0 comments]

Related

• Regulatory DNA sequence Design with Reinforcement Learning
• Conditional Monte Carlo Tree Diffusion for Designing Cell-Type-Specific and Biologically Faithful Regulatory DNA
• Language Models for Controllable DNA Sequence Design
• Regulatory grammar in human promoters uncovered by MPRA-based deep learning
• Biological Reasoning-Informed Regression for Interpretable Regulatory DNA Activity Prediction