Robust Training of Social Media Image Classification Models for Rapid Disaster Response

2021/04/09 by Firoj Alam, Tanvirul Alam, Alam, Firoj +6 · 5 citations
Computer Science · Engineering · Social Sciences · #68T50 #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computers and Society (cs.CY) #Convolutional neural network #Damages #Data science #Deep learning #Disaster response #Emergency management #Engineering #FOS: Computer and information sciences #I.2.7 #Image (mathematics) #Machine Learning (cs.LG) #Machine learning #Misinformation and Its Impacts #Public Relations and Crisis Communication #Seismology and Earthquake Studies #Situation awareness #Social and Information Networks (cs.SI) #Social media #World Wide Web #acm:68T50 #cs.CV #cs.CY #cs.LG #cs.SI #msc:68T50

paper · pdf · doi:10.48550/arxiv.2104.04184

published in arXiv (Cornell University) (Cornell University) · Social media images, Image Classification, Natural disasters, Crisis Informatics, Deep learning. Extended version of arXiv:2011.08916. arXiv admin note: substantial text overlap with arXiv:2011.08916

openalex publication_date 2021/04/09 · arxiv created 2021/07/19 · arxiv updated 2021/07/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Images shared on social media help crisis managers gain situational awareness and assess incurred damages, among other response tasks. As the volume and velocity of such content are typically high, real-time image classification has become an urgent need for a faster disaster response. Recent advances in computer vision and deep neural networks have enabled the development of models for real-time image classification for a number of tasks, including detecting crisis incidents, filtering irrelevant images, classifying images into specific humanitarian categories, and assessing the severity of the damage. To develop robust real-time models, it is necessary to understand the capability of the publicly available pre-trained models for these tasks, which remains to be under-explored in the crisis informatics literature. In this study, we address such limitations by investigating ten different network architectures for four different tasks using the largest publicly available datasets for these tasks. We also explore various data augmentation strategies, semi-supervised techniques, and a multitask learning setup. In our extensive experiments, we achieve promising results.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ImageNet classification with deep convolutional neural networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Localizing and Quantifying Damage in Social Media Images
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• Temporal Ensembling for Semi-Supervised Learning
• Unsupervised Data Augmentation for Consistency Training
• Self-Training With Noisy Student Improves ImageNet Classification
• Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response
• Deep Learning Benchmarks and Datasets for Social Media Image Classification for Disaster Response

Related

• Deep Learning Benchmarks and Datasets for Social Media Image Classification for Disaster Response
• Automatic Image Filtering on Social Networks Using Deep Learning and Perceptual Hashing During Crises
• Train and Deploy an Image Classifier for Disaster Response
• Rapid Classification of Crisis-Related Data on Social Networks using Convolutional Neural Networks
• CrisisBench: Benchmarking Crisis-related Social Media Datasets for Humanitarian Information Processing