Multi-Modal Contrastive Learning for Online Clinical Time-Series Applications

2024/03/27 by Baldenweg, Fabian, Burger, Manuel, Rätsch, Gunnar +1 · 1 citation
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2403.18316

Abstract

Electronic Health Record (EHR) datasets from Intensive Care Units (ICU) contain a diverse set of data modalities. While prior works have successfully leveraged multiple modalities in supervised settings, we apply advanced self-supervised multi-modal contrastive learning techniques to ICU data, specifically focusing on clinical notes and time-series for clinically relevant online prediction tasks. We introduce a loss function Multi-Modal Neighborhood Contrastive Loss (MM-NCL), a soft neighborhood function, and showcase the excellent linear probe and zero-shot performance of our approach.

Cited by

• Decoupled Complementary Spectral-Spatial Learning for Background Representation Enhancement in Hyperspectral Anomaly Detection

Related

• Global Contrastive Training for Multimodal Electronic Health Records with Language Supervision
• Contrastive Learning on Multimodal Analysis of Electronic Health Records
• Bridging Electronic Health Records and Clinical Texts: Contrastive Learning for Enhanced Clinical Tasks
• SCEHR: Supervised Contrastive Learning for Clinical Risk Prediction using Electronic Health Records
• Contrastive Learning Improves Critical Event Prediction in COVID-19 Patients