A survey on incorporating domain knowledge into deep learning for medical image analysis

2020/04/30 by Xiaozheng Xie, Jianwei Niu, Xuefeng Liu +3 · 309 citations
Computer Science · Engineering · Medicine · #AI in cancer detection #Artificial intelligence #Bottleneck #COVID-19 diagnosis using AI #Categorization #Computer science #Data science #Deep learning #Domain (mathematical analysis) #Domain knowledge #Focus (optics) #Image segmentation #Leverage (statistics) #Machine learning #Radiomics and Machine Learning in Medical Imaging #Segmentation #Transfer of learning #cs.CV #eess.IV

paper · pdf · doi:10.1016/j.media.2021.101985

published in Medical Image Analysis 69, 101985 (Elsevier BV) · 27 pages, 18 figures

openalex publication_date 2021/01/30 · arxiv created 2021/02/08 · arxiv updated 2021/02/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Although deep learning models like CNNs have achieved great success in medical image analysis, the small size of medical datasets remains a major bottleneck in this area. To address this problem, researchers have started looking for external information beyond current available medical datasets. Traditional approaches generally leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors, to create networks that resemble how medical doctors are trained, mimic their diagnostic patterns, or focus on the features or areas they pay particular attention to. In this survey, we summarize the current progress on integrating medical domain knowledge into deep learning models for various tasks, such as disease diagnosis, lesion, organ and abnormality detection, lesion and organ segmentation. For each task, we systematically categorize different kinds of medical domain knowledge that have been utilized and their corresponding integrating methods. We also provide current challenges and directions for future research.

Citations

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans
• The Unreasonable Effectiveness of Data
• Conditional Generative Adversarial Nets
• Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response
• ImageNet classification with deep convolutional neural networks
• A Survey on Transfer Learning
• Unsupervised Domain Adaptation with Residual Transfer Networks
• A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI
• Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks
• ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function
• Detection and Attention: Diagnosing Pulmonary Lung Cancer from CT by Imitating Physicians
• Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification
• MURA: Large Dataset for Abnormality Detection in Musculoskeletal Radiographs
• Synergistic Reconstruction and Synthesis via Generative Adversarial Networks for Accelerated Multi-Contrast MRI
• Attention to Lesion: Lesion-Aware Convolutional Neural Network for Retinal Optical Coherence Tomography Image Classification
• Clinically Accurate Chest X-Ray Report Generation
• Med3D: Transfer Learning for 3D Medical Image Analysis
• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• A Survey on Neural Architecture Search
• PK-GCN: Prior Knowledge Assisted Image Classification using Graph Convolution Networks
• Learn like a Pathologist: Curriculum Learning by Annotator Agreement for Histopathology Image Classification
• Support-vector networks

Cited by

• A Lightweight Large Vision-language Model for Multimodal Medical Images