Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencing

2025/06/16 by Xiang Zhang, Zhang, Xiang, Jiaqi Wei +11 · 2 voices · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · #Biomolecules (q-bio.BM) #FOS: Biological sciences #FOS: Computer and information sciences #Genetics, Bioinformatics, and Biomedical Research #Genomics and Phylogenetic Studies #Machine Learning (cs.LG) #cs.LG #q-bio.BM

paper · pdf · doi:10.48550/arxiv.2506.13485

openalex publication_date 2025/06/16 · arxiv published 2025/06/16 · arxiv updated 2025/06/16 · openalex created_date 2025/10/13 · openalex updated_date 2026/07/28

Abstract

Peptide sequencing-the process of identifying amino acid sequences from mass spectrometry data-is a fundamental task in proteomics. Non-Autoregressive Transformers (NATs) have proven highly effective for this task, outperforming traditional methods. Unlike autoregressive models, which generate tokens sequentially, NATs predict all positions simultaneously, leveraging bidirectional context through unmasked self-attention. However, existing NAT approaches often rely on Connectionist Temporal Classification (CTC) loss, which presents significant optimization challenges due to CTC's complexity and increases the risk of training failures. To address these issues, we propose an improved non-autoregressive peptide sequencing model that incorporates a structured protein sequence curriculum learning strategy. This approach adjusts protein's learning difficulty based on the model's estimated protein generational capabilities through a sampling process, progressively learning peptide generation from simple to complex sequences. Additionally, we introduce a self-refining inference-time module that iteratively enhances predictions using learned NAT token embeddings, improving sequence accuracy at a fine-grained level. Our curriculum learning strategy reduces NAT training failures frequency by more than 90% based on sampled training over various data distributions. Evaluations on nine benchmark species demonstrate that our approach outperforms all previous methods across multiple metrics and species.

Citations

• Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
• NovoBench: Benchmarking Deep Learning-based De Novo Peptide Sequencing Methods in Proteomics
• Simple and Effective Masked Diffusion Language Models
• Accurate structure prediction of biomolecular interactions with AlphaFold 3
• AdaNovo: Adaptive De Novo Peptide Sequencing with Conditional Mutual Information
• ContraNovo: A Contrastive Learning Approach to Enhance De Novo Peptide Sequencing
• TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models
• Structure-informed Language Models Are Protein Designers
• Evolutionary-scale prediction of atomic-level protein structure with a language model
• A Character-Level Length-Control Algorithm for Non-Autoregressive Sentence Summarization
• One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation
• Directed Acyclic Transformer for Non-Autoregressive Machine Translation
• A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond
• Step-unrolled Denoising Autoencoders for Text Generation
• Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervision
• AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate
• Highly accurate protein structure prediction with AlphaFold
• Progressive Multi-Granularity Training for Non-Autoregressive Translation
• Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation
• Non-Autoregressive Translation by Learning Target Categorical Codes
• Context-Aware Cross-Attention for Non-Autoregressive Translation
• Glancing Transformer for Non-Autoregressive Neural Machine Translation
• Improving Non-autoregressive Neural Machine Translation with Monolingual Data
• Non-Autoregressive Machine Translation with Latent Alignments
• Aligned Cross Entropy for Non-Autoregressive Machine Translation
• Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation
• Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
• Non-Autoregressive Machine Translation with Auxiliary Regularization
• Insertion Transformer: Flexible Sequence Generation via Insertion\n Operations
• Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input
• Non-Autoregressive Neural Machine Translation
• PEAKS: powerful software for peptide de novo sequencing by tandem mass spectrometry

Cited by

• Bidirectional Representations Augmented Autoregressive Biological Sequence Generation

Discussions

• Xiang Zhang, Jiaqi Wei, Zijie Qiu, Sheng Xu, Nanqing Dong, Zhiqiang Gao, Siqi Sun: Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencing https://arxiv.org/abs/ [bsky, 2 points, 0 comments]
• Curriculum Learning for Biological Sequence Prediction: The Case... arxiv.org/abs/2506.1... --- #proteomics #prot-preprint [bsky, 0 points, 0 comments]

Related

• Learning Fragmentation Physics or Exploiting Sequence Priors? Benchmarking Bias in Deep Learning Models for De Novo Peptide Sequencing
• Training self-supervised peptide sequence models on artificially chopped proteins
• Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models
• Peptide Sequencing Via Protein Language Models
• ContraNovo: A Contrastive Learning Approach to Enhance De Novo Peptide Sequencing