Freeze, Diffuse, Decode: Geometry-Aware Adaptation of Pretrained Transformer Embeddings for Antimicrobial Peptide Design

2025/11/28 by Pankhil Gawade, Gawade, Pankhil, Adam Izdebski +10
Immunology and Microbiology · Biochemistry, Genetics and Molecular Biology · #Antimicrobial Peptides and Activities #Biochemical and Structural Characterization #Bacterial biofilms and quorum sensing

paper · pdf · doi:10.48550/arxiv.2511.23120

Abstract

Pretrained transformers provide rich, general-purpose embeddings, which are transferred to downstream tasks. However, current transfer strategies: fine-tuning and probing, either distort the pretrained geometric structure of the embeddings or lack sufficient expressivity to capture task-relevant signals. These issues become even more pronounced when supervised data are scarce. Here, we introduce Freeze, Diffuse, Decode (FDD), a novel diffusion-based framework that adapts pre-trained embeddings to downstream tasks while preserving their underlying geometric structure. FDD propagates supervised signal along the intrinsic manifold of frozen embeddings, enabling a geometry-aware adaptation of the embedding space. Applied to antimicrobial peptide design, FDD yields low-dimensional, predictive, and interpretable representations that support property prediction, retrieval, and latent-space interpolation.

Citations

• Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning
• Synergistic Benefits of Joint Molecule Generation and Property Prediction
• Random Forest Autoencoders for Guided Representation Learning
• Random Forest-Supervised Manifold Alignment
• A Strong Baseline for Molecular Few-Shot Learning
• Enhancing Molecular Property Prediction with Auxiliary Learning and Task-Specific Adaptation
• Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution
• Geometry- and Accuracy-Preserving Random Forest Proximities
• How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy
• The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning
• A Non-Linear Structural Probe
• Probing Classifiers: Promises, Shortcomings, and Advances
• Probing Classifiers: Promises, Shortcomings, and Advances
• ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
• Optimal Transport losses and Sinkhorn algorithm with general convex regularization
• Neural Ordinary Differential Equations
• Learning Generative Models with Sinkhorn Divergences

Related

• Graph Diffusion Transformers are In-Context Molecular Designers
• Protein generation with embedding learning for motif diversification
• Diffusion Sequence Models for Enhanced Protein Representation and Generation
• DiffSDS: A language diffusion model for protein backbone inpainting under geometric conditions and constraints
• De novo design of protein structure and function with RFdiffusion