Recent Advances in Convolutional Neural Networks

2015/12/22 by Jiuxiang Gu, Zhenhua Wang, Gu, Jiuxiang +21 · 328 citations
Computer Science · #Activation function #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Algorithm #Artificial intelligence #Artificial neural network #Computation #Computer science #Convolutional neural network #Deep learning #Graphics #Graphics processing unit #Human Pose and Action Recognition #Machine learning #Parallel computing #cs.CV #cs.LG #cs.NE

paper · pdf · doi:10.48550/arxiv.1512.07108

published in arXiv (Cornell University) (Cornell University) · Pattern Recognition, Elsevier

openalex publication_date 2015/12/22 · arxiv created 2017/10/19 · arxiv updated 2017/10/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

In the last few years, deep learning has led to very good performance on a variety of problems, such as visual recognition, speech recognition and natural language processing. Among different types of deep neural networks, convolutional neural networks have been most extensively studied. Leveraging on the rapid growth in the amount of the annotated data and the great improvements in the strengths of graphics processor units, the research on convolutional neural networks has been emerged swiftly and achieved state-of-the-art results on various tasks. In this paper, we provide a broad survey of the recent advances in convolutional neural networks. We detailize the improvements of CNN on different aspects, including layer design, activation function, loss function, regularization, optimization and fast computation. Besides, we also introduce various applications of convolutional neural networks in computer vision, speech and natural language processing.

Citations

• ADADELTA: An Adaptive Learning Rate Method
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• Improving Deep Neural Networks with Probabilistic Maxout Units
• Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
• The Loss Surfaces of Multilayer Networks
• Auto-Encoding Variational Bayes
• Gradient-based learning applied to document recognition
• Online Batch Selection for Faster Training of Neural Networks
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• Bilinear CNNs for Fine-grained Visual Recognition
• HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
• Predicting Parameters in Deep Learning
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Cited by

• Bunch-by-Bunch Prediction of Beam Transverse Position, Phase, and Length in a Storage Ring Using Neural Networks
• An Interpretable Operator-Learning Model for Electric Field Profile Reconstruction in Discharges Based on the EFISH Method
• Deep Learning Framework for Enhanced Neutrino Reconstruction of Single-line Events in the ANTARES Telescope
• The curse of dimensionality: what lies beyond the capabilities of physics-informed neural networks
• Accelerating metamaterial topology optimization using deep super-resolution networks
• Object Detection in 20 Years: A Survey
• Adaptive Convolutional ELM For Concept Drift Handling in Online Stream Data
• FruitProm: Probabilistic Maturity Estimation and Detection of Fruits and Vegetables
• Deep-Learning-Empowered Programmable Topolectrical Circuits
• Recent progress of neuromorphic sensory and optoelectronic systems
• CAPSim: A Fast CPU Performance Simulator Using Attention-based Predictor
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• A New Dataset and Proposed Convolutional Neural Network Architecture for Classification of American Sign Language Digits
• Storing Sensor Events in the Interconnection Strength of Conducting Polymer Dendrites
• Neural Network Based Framework for Passive Intermodulation Cancellation in MIMO Systems
• Efficient Rectified Flow for Image Fusion
• Uncertainty-Aware Neural Networks for Fuzzy Dark Matter Model Selection from \texorpdfstringx_{\rm HI}x_HI Measurements
• Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
• Multi-Scale Deep Learning for Colon Histopathology: A Hybrid Graph-Transformer Approach
• D3PINNs: A Novel Physics-Informed Neural Network Framework for Staged Solving of Time-Dependent Partial Differential Equations
• Comparative analysis of data transformation methods for detecting non-technical losses in electricity grids
• Neural Network Training via Stochastic Alternating Minimization with Trainable Step Sizes
• von Mises-Fisher Mixture Model-based Deep learning: Application to Face Verification
• AVPDN: Learning Motion-Robust and Scale-Adaptive Representations for Video-Based Polyp Detection
• Proactive Disentangled Modeling of Trigger-Object Pairings for Backdoor Defense
• Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding
• Semantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform
• Tricks and Plug-ins for Gradient Boosting in Image Classification
• Rethinking Occlusion in FER: A Semantic-Aware Perspective and Go Beyond
• Visual Place Recognition for Large-Scale UAV Applications
• SuperCM: Improving Semi-Supervised Learning and Domain Adaptation through differentiable clustering
• Deep learning-aided inverse design of porous metamaterials
• Microaneurysm Detection in Fundus Images Using a Two-step Convolutional Neural Networks
• Distributed Averaging CNN-ELM for Big Data
• Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images
• Deformable Dynamic Convolution for Accurate yet Efficient Spatio-Temporal Traffic Prediction
• Prediction of Lane Change Intentions of Human Drivers using an LSTM, a CNN and a Transformer
• Empowering Bridge Digital Twins by Bridging the Data Gap with a Unified Synthesis Framework
• Indoor thermal comfort management: A Bayesian machine-learning approach to data denoising and dynamics prediction of HVAC systems
• Similarity Memory Prior is All You Need for Medical Image Segmentation
• Master's Thesis : Deep Learning for Visual Recognition
• AquaCluster: Using Satellite Images And Self-supervised Machine Learning Networks To Detect Water Hidden Under Vegetation
• The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches
• Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges
• Current AI technologies in cancer diagnostics and treatment
• A Tutorial-cum-Survey on Self-Supervised Learning for Wi-Fi Sensing: Trends, Challenges, and Outlook
• Inceptive Transformers: Enhancing Contextual Representations through Multi-Scale Feature Learning Across Domains and Languages
• Parameter Estimation Horizon of Core-Collapse Supernovae with a Network of Gravitational-Wave Detectors
• Field Matters: A lightweight LLM-enhanced Method for CTR Prediction
• Unsupervised anomaly detection in MeV ultrafast electron diffraction
• ChestyBot: Detecting and Disrupting Chinese Communist Party Influence Stratagems
• Improving Generalization of Medical Image Registration Foundation Model
• Deep Learning based Cephalometric Landmark Identification using Landmark-dependent Multi-scale Patches
• Optimising the Input Image to Improve Visual Relationship Detection
• A Wireless World Model for AI-Native 6G Networks
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• Experimenting with Convolutional Neural Network Architectures for the automatic characterization of Solitary Pulmonary Nodules' malignancy rating
• Utilizing Machine Learning and Digital Twin Technology for Rock Parameter Estimation from Drilling Data
• An Empirical Study of Language CNN for Image Captioning
• Identifying Lead Water Service Lines Using Ultrasonic Stress Wave Propagation and 1D-Convolutional Neural Network
• GADS: A Super Lightweight Model for Head Pose Estimation
• CSI2Dig: Recovering Digit Content from Smartphone Loudspeakers Using Channel State Information
• Recurrent Attentional Networks for Saliency Detection
• A Diffusion-Based Framework for Terrain-Aware Remote Sensing Image Reconstruction
• Beyond the Generative Learning Trilemma: Generative Model Assessment in Data Scarcity Domains
• A Solid-State Nanopore Signal Generator for Training Machine Learning Models

Related

• A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
• Fundamental tenis : resep meraih kemenangan / Tony Mottram
• Convolutional Neural Networks:
• A survey of the recent architectures of deep convolutional neural networks
• A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends