Pre-trained Transformer-models using chronic invasive electrophysiology for symptom decoding without patient-individual training

2025/08/13 by Merk, Timon, Salehi, Saeed, Koehler, Richard M. +8
#FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2508.10160

Abstract

Neural decoding of pathological and physiological states can enable patient-individualized closed-loop neuromodulation therapy. Recent advances in pre-trained large-scale foundation models offer the potential for generalized state estimation without patient-individual training. Here we present a foundation model trained on chronic longitudinal deep brain stimulation recordings spanning over 24 days. Adhering to long time-scale symptom fluctuations, we highlight the extended context window of 30 minutes. We present an optimized pre-training loss function for neural electrophysiological data that corrects for the frequency bias of common masked auto-encoder loss functions due to the 1-over-f power law. We show in a downstream task the decoding of Parkinson's disease symptoms with leave-one-subject-out cross-validation without patient-individual training.

Citations

• BrainWave: A Brain Signal Foundation Model for Clinical Applications
• Multimodal Token Fusion for Vision Transformers
• Parameterizing neural power spectra into periodic and aperiodic components
• Language Models are Few-Shot Learners
• Learning to Encode Position for Transformer with Continuous Dynamical Model
• Scaling Laws for Neural Language Models
• Generating Long Sequences with Sparse Transformers
• Attention Is All You Need
• Separating Fractal and Oscillatory Components in the Power Spectrum of Neurophysiological Signal

Related

• Neurophysiologically Realistic Environment for Comparing Adaptive Deep Brain Stimulation Algorithms in Parkinson Disease
• Deep Learning Model Predictive Control for Deep Brain Stimulation in Parkinson's Disease
• Temporal Basis Function Models for Closed-Loop Neural Stimulation
• Offline Learning of Closed-Loop Deep Brain Stimulation Controllers for Parkinson Disease Treatment
• TransformEEG: Towards Improving Model Generalizability in Deep Learning-based EEG Parkinson's Disease Detection