Protein Secondary Structure Prediction Using Transformers

2025/12/09 by Maxime, Manzi Kevin
Biochemistry, Genetics and Molecular Biology · #Machine Learning in Bioinformatics #Protein Structure and Dynamics #Fractal and DNA sequence analysis

paper · doi:10.48550/arxiv.2512.08613

Abstract

Predicting protein secondary structures such as alpha helices, beta sheets, and coils from amino acid sequences is essential for understanding protein function. This work presents a transformer-based model that applies attention mechanisms to protein sequence data to predict structural motifs. A sliding-window data augmentation technique is used on the CB513 dataset to expand the training samples. The transformer shows strong ability to generalize across variable-length sequences while effectively capturing both local and long-range residue interactions.

Citations

• ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing
• ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
• Attention Is All You Need
• Protein secondary structure prediction using deep convolutional neural fields

Related

• Protein Secondary Structure Prediction Using 3D Graphs and Relation-Aware Message Passing Transformers
• Learning inverse folding from millions of predicted structures
• Transformer Neural Networks Attending to Both Sequence and Structure for Protein Prediction Tasks
• Endowing Protein Language Models with Structural Knowledge
• Transformers in Protein: A Survey