Enabling On-Device Self-Supervised Contrastive Learning With Selective Data Contrast

2021/06/07 by Yawen Wu, Wu, Yawen, Zhepeng Wang +7
Computer Science · #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification #Machine Learning and ELM #cs.LG

paper · pdf · doi:10.48550/arxiv.2106.03796

arxiv created 2021/06/07 · arxiv updated 2021/06/08

Abstract

After a model is deployed on edge devices, it is desirable for these devices to learn from unlabeled data to continuously improve accuracy. Contrastive learning has demonstrated its great potential in learning from unlabeled data. However, the online input data are usually none independent and identically distributed (non-iid) and storages of edge devices are usually too limited to store enough representative data from different data classes. We propose a framework to automatically select the most representative data from the unlabeled input stream, which only requires a small data buffer for dynamic learning. Experiments show that accuracy and learning speed are greatly improved.

Citations

• A Simple Framework for Contrastive Learning of Visual Representations

Related

• Enabling On-Device Learning via Experience Replay with Efficient Dataset Condensation
• Self-Supervised Learning at the Edge: The Cost of Labeling
• Contrastive Self-Supervised Learning at the Edge: An Energy Perspective
• Self-Damaging Contrastive Learning
• Unsupervised Contrastive Learning Using Out-Of-Distribution Data for Long-Tailed Dataset