Convolutional-LSTM for Multi-Image to Single Output Medical Prediction

2020/10/20 by L. Gary Leal, Leal, Luis, Marvin Castillo +9
Computer Science · Medicine · Neuroscience · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Brain Tumor Detection and Classification #COVID-19 diagnosis using AI #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2010.10004

openalex publication_date 2020/10/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Medical head CT-scan imaging has been successfully combined with deep learning for medical diagnostics of head diseases and lesions[1]. State of the art classification models and algorithms for this task usually are based on 3d convolution layers for volumetric data on a supervised learning setting (1 input volume, 1 prediction per patient) or 2d convolution layers in a supervised setting (1 input image, 1 prediction per image). However a very common scenario in developing countries is to have the volume metadata lost due multiple reasons for example formatting conversion in images (for example .dicom to jpg), in this scenario the doctor analyses the collection of images and then emits a single diagnostic for the patient (with possibly an unfixed and variable number of images per patient) , this prevents it from being possible to use state of the art 3d models, but also is not possible to convert it to a supervised problem in a (1 image,1 diagnostic) setting because different angles or positions of the images for a single patient may not contain the disease or lesion. In this study we propose a solution for this scenario by combining 2d convolutional[2] models with sequence models which generate a prediction only after all images have been processed by the model for a given patient \(i\), this creates a multi-image to single-diagnostic setting \(yⁱ=f(x₁,x₂,..,x_n)\) where \(n\) may be different between patients. The experimental results demonstrate that it is possible to get a multi-image to single diagnostic model which mimics human doctor diagnostic process: evaluate the collection of patient images and then use important information in memory to decide a single diagnostic for the patient.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Long Short-Term Memory
• Gradient-based learning applied to document recognition
• Densely Connected Convolutional Networks
• Development and Validation of Deep Learning Algorithms for Detection of Critical Findings in Head CT Scans

Related

• Disease Detection in Weakly Annotated Volumetric Medical Images using a Convolutional LSTM Network
• 3D Foundation Model for Generalizable Disease Detection in Head Computed Tomography
• An overview of deep learning in medical imaging
• A Novel Multi-branch ConvNeXt Architecture for Identifying Subtle Pathological Features in CT Scans
• MARL: Multimodal Attentional Representation Learning for Disease Prediction