Comparative Analysis of Vision Transformers and Traditional Deep Learning Approaches for Automated Pneumonia Detection in Chest X-Rays

2025/07/11 by Gaurav Singh, Singh, Gaurav
Medicine · #Artificial Intelligence (cs.AI) #COVID-19 diagnosis using AI #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Neural and Evolutionary Computing (cs.NE) #Radiomics and Machine Learning in Medical Imaging #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2507.10589

openalex publication_date 2025/07/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Pneumonia, particularly when induced by diseases like COVID-19, remains a critical global health challenge requiring rapid and accurate diagnosis. This study presents a comprehensive comparison of traditional machine learning and state-of-the-art deep learning approaches for automated pneumonia detection using chest X-rays (CXRs). We evaluate multiple methodologies, ranging from conventional machine learning techniques (PCA-based clustering, Logistic Regression, and Support Vector Classification) to advanced deep learning architectures including Convolutional Neural Networks (Modified LeNet, DenseNet-121) and various Vision Transformer (ViT) implementations (Deep-ViT, Compact Convolutional Transformer, and Cross-ViT). Using a dataset of 5,856 pediatric CXR images, we demonstrate that Vision Transformers, particularly the Cross-ViT architecture, achieve superior performance with 88.25% accuracy and 99.42% recall, surpassing traditional CNN approaches. Our analysis reveals that architectural choices impact performance more significantly than model size, with Cross-ViT's 75M parameters outperforming larger models. The study also addresses practical considerations including computational efficiency, training requirements, and the critical balance between precision and recall in medical diagnostics. Our findings suggest that Vision Transformers offer a promising direction for automated pneumonia detection, potentially enabling more rapid and accurate diagnosis during health crises.

Citations

• Escaping the Big Data Paradigm with Compact Transformers
• CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
• CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
• DeepViT: Towards Deeper Vision Transformer
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning

Related

• A novel method to enhance pneumonia detection via a model-level ensembling of CNN and vision transformer
• Deep Learning-Based Pneumonia Detection from Chest X-ray Images: A CNN Approach with Performance Analysis and Clinical Implications
• RepViT-CXR: A Channel Replication Strategy for Vision Transformers in Chest X-ray Tuberculosis and Pneumonia Classification
• Deep Convolutional Neural Networks to Diagnose COVID-19 and other Pneumonia Diseases from Posteroanterior Chest X-Rays
• Automated Methods for Detection and Classification Pneumonia based on X-Ray Images Using Deep Learning