Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

2015/02/11 by Sergey Ioffe, Christian Szegedy, Ioffe, Sergey +1 · 3 voices · 846 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Machine Learning and ELM #Neural Networks and Applications #cs.LG

paper · pdf · doi:10.48550/arxiv.1502.03167

openalex publication_date 2015/02/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.

Citations

• Deep Image: Scaling up Image Recognition
• On the difficulty of training Recurrent Neural Networks
• Gradient-based learning applied to document recognition
• Independent component analysis: algorithms and applications

Cited by

• Efficiently Training Time-to-First-Spike Spiking Neural Networks from Scratch
• Do Transformers Actually Help Intrusion Detection? A Temporal Sequence Evaluation on CIC-IDS2017
• Toward High-Fidelity 3D Point-Cloud Learning for Brain Folding Morphology Prediction Using Trans-Unet
• PN-QNN: Harnessing Physical Noise as a Native Regularizer in Photonic Hybrid Quantum Neural Networks
• Q-PhotoNAS: Hybrid Quantum Neural Architecture Search Framework on Photonic Devices
• Chebyshev Manifold Adaptation
• LieBN: Batch Normalization over Lie Groups
• Computer vision-based neural networks for radioisotope identification in urban environments
• Long-Horizon Wireless Link Scheduling with State-Augmented Graph Neural Networks
• Certified Training for Convolutional Perturbations
• FedDP-PALD: A Privacy-Preserving Federated Latent Diffusion Framework with Prototype Aggregation for Medical Data Synthesis
• CMCC-ReID: Cross-Modality Clothing-Change Person Re-Identification
• UPLiFT: Efficient Pixel-Dense Feature Upsampling with Local Attenders
• Building a Neural Network from Scratch: Implementation, Evaluation, and Optimization
• DELUGE: Towards Continental-Scale Daily Pluvial Flood Damage Prediction via Interpretable Conditioning on Foundation Model Embeddings
• A Geometry-Limited Identification Floor and Its Consequences for Voice-Clone Attribution in Professional Voice Actors
• Approximating SPR Distance Between Phylogenetic Trees with Graph Neural Networks
• A stochastic smoothing framework for nonconvex-nonconcave minEmax problems with applications to Wasserstein distributionally robust optimization
• Sharp Stability Threshold and Certification for Designing Stable Residual Architectures
• Harnessing Machine Learning for Hybrid Constitutive Modelling of Viscoelastic Fluid Flows in Computational Rheology
• AlphaWiSE: Adaptive Weight Interpolation for Continual Multimodal Representation Learning
• Reducing Per-Sample Harm in Stochastic Optimization
• Tiny neural networks for multi-object tracking in a modular Kalman framework
• Chameleon: A Multiplier-Free Temporal Convolutional Network Accelerator for End-to-End Few-Shot and Continual Learning from Sequential Data
• Normalisation and Initialisation Strategies for Graph Neural Networks in Blockchain Anomaly Detection
• Transformers without Normalization
• Tiling artifacts and trade-offs of feature normalization in the segmentation of large biological images
• Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking)
• A deep learning strategy to identify cell types across species from high-density extracellular recordings
• On the accuracy of implicit neural representations for cardiovascular anatomies and hemodynamic fields
• On batch normalization for SPDnet
• Image Denoising Using Global and Local Circulant Representation
• Deep learning for pedestrians: backpropagation in Transformers
• Explainable Neural Inverse Kinematics for Obstacle-Aware Robotic Manipulation: A Comparative Analysis of IKNet Variants
• A Neural Network-Based Real-time Casing Collar Recognition System for Downhole Instruments
• Investigating Deep Learning Models for Ejection Fraction Estimation from Echocardiography Videos
• The Affine Divergence: Aligning Activation Updates Beyond Normalisation
• Joint Flow Matching for Generator-Consistent Classification
• Mutual Modality Trust with Lightweight Reconstruction Regularization for Fine-grained Tire Pattern Recognition
• PINNs for Electromagnetic Wave Propagation
• TopoFE: topology-aware LLM-guided Automated Feature Engineering
• Simplex Demixing: Disentangling Multiple Light-Flavor Jets at Colliders
• Mitigating the Impact of Retention Loss on Inference Accuracy in 65 nm Single-Poly Floating-Gate Analog In-Memory Computing
• Latent Space Probing for Adult Content Detection in Video Generative Models
• Patch-Discontinuity Mining for Generalized Deepfake Detection
• GraviBERT: Transformer-based inference for gravitational-wave time series
• Simplifying Multi-Task Architectures Through Task-Specific Normalization
• Machine-learning techniques for model-independent searches in dijet final states
• Binary Neural Network Implementation for Handwritten Digit Recognition on FPGA
• Inverse-Designed Phase Prediction in Digital Lasers Using Deep Learning and Transfer Learning
• Towards Ancient Plant Seed Classification: A Benchmark Dataset and Baseline Model
• Towards Reproducibility in Predictive Process Mining: SPICE -- A Deep Learning Library
• Batch Normalization-Free Fully Integer Quantized Neural Networks via Progressive Tandem Learning
• Latent-space variational data assimilation in two-dimensional turbulence
• Surrogate Neural Architecture Codesign Package (SNAC-Pack)
• Enhancing Alzheimer's Detection through Late Fusion of Multi-Modal EEG Features
• How Does Fourier Analysis Network Work? A Mechanism Analysis and a New Dual-Activation Layer Proposal
• VajraV1 -- The most accurate Real Time Object Detector of the YOLO family
• EEG-D3: A Solution to the Hidden Overfitting Problem of Deep Learning Models
• LitePT: Lighter Yet Stronger Point Transformer
• Element-wise Modulation of Random Matrices for Efficient Neural Layers
• Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approaches
• GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generation
• In-Context Learning for Seismic Data Processing
• All-in-One ASR: Unifying Encoder-Decoder Models of CTC, Attention, and Transducer in Dual-Mode ASR
• Stronger Normalization-Free Transformers
• VHOI: Controllable Video Generation of Human-Object Interactions from Sparse Trajectories via Motion Densification
• WTNN: Weibull-Tailored Neural Networks for survival analysis
• The Unseen Bias: How Norm Discrepancy in Pre-Norm MLLMs Leads to Visual Information Loss
• Instance-Aware Test-Time Segmentation for Continual Domain Shifts
• GlimmerNet: A Lightweight Grouped Dilated Depthwise Convolutions for UAV-Based Emergency Monitoring
• DeepAgent: A Dual Stream Multi Agent Fusion for Robust Multimodal Deepfake Detection
• ExPUFFIN: Thermodynamic Consistent Viscosity Prediction in an Extended Path-Unifying Feed-Forward Interfaced Network
• Proof of Concept for Mammography Classification with Enhanced Compactness and Separability Modules
• When Gender is Hard to See: Multi-Attribute Support for Long-Range Recognition
• Learning Invariant Graph Representations Through Redundant Information
• A Comparative Study on Synthetic Facial Data Generation Techniques for Face Recognition
• Neural Coherence : Find higher performance to out-of-distribution tasks from few samples
• Text-Only Training for Image Captioning with Retrieval Augmentation and Modality Gap Correction
• SweetDeep: A Wearable AI Solution for Real-Time Non-Invasive Diabetes Screening
• Action Anticipation at a Glimpse: To What Extent Can Multimodal Cues Replace Video?
• Breast Cell Segmentation Under Extreme Data Constraints: Quantum Enhancement Meets Adaptive Loss Stabilization
• Leveraging generative adversarial networks with spatially adaptive denormalization for multivariate stochastic seismic data inversion
• Delays in Spiking Neural Networks: A State Space Model Approach
• Multifractal Recalibration of Neural Networks for Medical Imaging Segmentation
• Upper Approximation Bounds for Neural Oscillators
• Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression
• Calibration-Free EEG-based Driver Drowsiness Detection with Online Test-Time Adaptation
• Interpretable Graph Neural Networks for Classifying Structure and Magnetism in Delafossite Compounds
• Hybrid Context-Fusion Attention (CFA) U-Net and Clustering for Robust Seismic Horizon Interpretation
• Shoe Style-Invariant and Ground-Aware Learning for Dense Foot Contact Estimation
• Merge and Bound: Direct Manipulations on Weights for Class Incremental Learning
• Anomaly Detection with Adaptive and Aggressive Rejection for Contaminated Training Data
• FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision
• You Can Trust Your Clustering Model: A Parameter-free Self-Boosting Plug-in for Deep Clustering
• Accumulated Local Effects and Graph Neural Networks for link prediction
• Identifying environmental factors associated with tetrodotoxin contamination in bivalve mollusks using eXplainable AI
• IrisNet: Infrared Image Status Awareness Meta Decoder for Infrared Small Targets Detection
• On the Limits of Momentum in Decentralized and Federated Optimization
• Leveraging Adversarial Learning for Pathological Fidelity in Virtual Staining
• Higgs Production Classifier using Weak Supervision
• TimePre: Bridging Accuracy, Efficiency, and Stability in Probabilistic Time-Series Forecasting
• Transforming Conditional Density Estimation Into a Single Nonparametric Regression Task
• Three-Class Emotion Classification for Audiovisual Scenes Based on Ensemble Learning Scheme
• A Lightweight Approach to Detection of AI-Generated Texts Using Stylometric Features
• A Priori Assessment of Rotational Invariance in Multiscale Convolutional Neural Network-Based Subgrid-Scale Model for Wall-Bounded Turbulent Flows
• MRI Super-Resolution with Deep Learning: A Comprehensive Survey
• From Qubits to Couplings: A Hybrid Quantum Machine Learning Framework for LHC Physics
• Change-of-Basis Pruning via Rotational Invariance
• Exploiting Inter-Sample Information for Long-tailed Out-of-Distribution Detection
• A Spatial Semantics and Continuity Perception Attention for Remote Sensing Water Body Change Detection
• Learning Where, What and How to Transfer: A Multi-Role Reinforcement Learning Approach for Evolutionary Multitasking
• CKDA: Cross-modality Knowledge Disentanglement and Alignment for Visible-Infrared Lifelong Person Re-identification
• A Neural Network Approach to Preferred Event Selection for Low-Latency Gravitational-Wave Alerts
• Meta-Black-Box Optimization with Bi-Space Landscape Analysis and Dual-Control Mechanism for SAEA
• CascadedViT: Cascaded Chunk-FeedForward and Cascaded Group Attention Vision Transformer
• Edge-aware baselines for ogbn-proteins in PyTorch Geometric: species-wise normalization, post-hoc calibration, and cost-accuracy trade-offs
• FLClear: Visually Verifiable Multi-Client Watermarking for Federated Learning
• MSLoRA: Multi-Scale Low-Rank Adaptation via Attention Reweighting
• FedSDA: Federated Stain Distribution Alignment for Non-IID Histopathological Image Classification
• From Classification to Cross-Modal Understanding: Leveraging Vision-Language Models for Fine-Grained Renal Pathology
• Evaluation of Attention Mechanisms in U-Net Architectures for Semantic Segmentation of Brazilian Rock Art Petroglyphs
• Batch Matrix-form Equations and Implementation of Multilayer Perceptrons
• Learning Conjugate Direction Fields for Planar Quadrilateral Mesh Generation
• RobIA: Robust Instance-aware Continual Test-time Adaptation for Deep Stereo
• MiniFool -- Physics-Constraint-Aware Minimizer-Based Adversarial Attacks in Deep Neural Networks
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
• Pruning as Regularization: Sensitivity-Aware One-Shot Pruning in ASR
• The Impact of Longitudinal Mammogram Alignment on Breast Cancer Risk Assessment
• Automatic Grid Updates for Kolmogorov-Arnold Networks using Layer Histograms
• Misaligned by Design: Incentive Failures in Machine Learning
• Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?
• Probably Approximately Global Robustness Certification
• Learning Topology-Driven Multi-Subspace Fusion for Grassmannian Deep Network
• Explainable Deep Learning-based Classification of Wolff-Parkinson-White Electrocardiographic Signals
• Identification of tau leptons using a convolutional neural network with domain adaptation
• SiamMM: A Mixture Model Perspective on Deep Unsupervised Learning
• Linear Mode Connectivity under Data Shifts for Deep Ensembles of Image Classifiers
• OMPQ: Orthogonal Mixed Precision Quantization
• Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?
• Addressing divergent representations from causal interventions on neural networks
• Signal Intensity-weighted coordinate channels improve learning stability and generalisation in 1D and 2D CNNs in localisation tasks on biomedical signals
• Diffusion-SDPO: Safeguarded Direct Preference Optimization for Diffusion Models
• Test Time Adaptation Using Adaptive Quantile Recalibration
• Learning with Category-Equivariant Architectures for Human Activity Recognition
• H-Infinity Filter Enhanced CNN-LSTM for Arrhythmia Detection from Heart Sound Recordings
• Using Deep Learning for Robust Classification of Fast Radio Bursts
• HyFormer-Net: A Synergistic CNN-Transformer with Interpretable Multi-Scale Fusion for Breast Lesion Segmentation and Classification in Ultrasound Images
• The Hidden Power of Normalization Layers in Neural Networks: Exponential Capacity Control
• Layer-Wise Modality Decomposition for Interpretable Multimodal Sensor Fusion
• Data-Augmented Deep Learning for Downhole Depth Sensing and Field Validation
• Temporal Cardiovascular Dynamics for Improved PPG-Based Heart Rate Estimation
• Soft Task-Aware Routing of Experts for Equivariant Representation Learning
• Spectral Neural Graph Sparsification
• Model-Free Robust Beamforming in Satellite Downlink using Reinforcement Learning
• Right for the Right Reasons: Avoiding Reasoning Shortcuts via Prototypical Neurosymbolic AI
• CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices
• Channel Pruning via Optimal Thresholding
• MetaFormer is Actually What You Need for Vision
• Hire-MLP: Vision MLP via Hierarchical Rearrangement
• Importance Weighted Adversarial Discriminative Transfer for Anomaly Detection
• CNN-based Steganalysis and Parametric Adversarial Embedding: a Game-Theoretic Framework
• Regularizing Recurrent Neural Networks via Sequence Mixup
• BN-NAS: Neural Architecture Search with Batch Normalization
• Learning from Temporal Gradient for Semi-supervised Action Recognition
• Cnns in land cover mapping with remote sensing imagery: a review and meta-analysis
• Graph Convolutional Networks for Hyperspectral Image Classification
• ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations
• Reinforcement Learning applied to Optimization of LHC beams in the CERN Proton Synchrotron
• Train-by-Reconnect: Decoupling Locations of Weights from their Values
• TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
• Objects are Different: Flexible Monocular 3D Object Detection
• The Tree Ensemble Layer: Differentiability meets Conditional Computation
• Quantal synaptic dilution enhances sparse encoding and dropout regularisation in deep networks
• Learning Channel Inter-dependencies at Multiple Scales on Dense Networks for Face Recognition
• Reducing Noise in GAN Training with Variance Reduced Extragradient
• Use the Online Network If You Can: Towards Fast and Stable Reinforcement Learning
• Overfitting the Data: Compact Neural Video Delivery via Content-aware Feature Modulation
• Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing
• Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search
• Characterizing signal propagation to close the performance gap in\n unnormalized ResNets
• ResT: An Efficient Transformer for Visual Recognition
• SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition
• Perceptual Attention-based Predictive Control
• IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction
• MisConv: Convolutional Neural Networks for Missing Data
• Generative Adversarial Text to Image Synthesis
• The Uncanny Similarity of Recurrence and Depth
• High-Performance Large-Scale Image Recognition Without Normalization
• Sparsity-Probe: Analysis tool for Deep Learning Models
• Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering
• A systematic study of DNN based speech enhancement in reverberant and reverberant-noisy environments
• Cross-Iteration Batch Normalization
• Separating the Effects of Batch Normalization on CNN Training Speed and Stability Using Classical Adaptive Filter Theory
• PAMS: Quantized Super-Resolution via Parameterized Max Scale
• How well do U-Net-based segmentation trained on adult cardiac magnetic\n resonance imaging data generalise to rare congenital heart diseases for\n surgical planning?
• Learning Various Length Dependence by Dual Recurrent Neural Networks
• Invertible Generative Modeling using Linear Rational Splines
• Neural Metric Learning for Fast End-to-End Relation Extraction
• DRD-Net: Detail-recovery Image Deraining via Context Aggregation Networks
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
• CSTNet: Contrastive Speech Translation Network for Self-Supervised Speech Representation Learning
• Encoding Robustness to Image Style via Adversarial Feature Perturbations
• Meta Feature Modulator for Long-tailed Recognition
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• End-to-End Blind Image Quality Assessment Using Deep Neural Networks
• DDLSTM: Dual-Domain LSTM for Cross-Dataset Action Recognition
• Convolutional neural networks in Vis–NIR chemometrics: From contradiction to conditional design
• LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search
• RepVGG: Making VGG-style ConvNets Great Again
• PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition
• Improving Shape Deformation in Unsupervised Image-to-Image Translation
• WaveGrad: Estimating Gradients for Waveform Generation
• Dynamic Resolution Network
• VNect
• Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
• Graph Pruning for Model Compression
• MixPath: A Unified Approach for One-shot Neural Architecture Search
• When Do Curricula Work?
• WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition
• From Kinematics To Dynamics: Estimating Center of Pressure and Base of Support from Video Frames of Human Motion
• Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network
• GMAIR: Unsupervised Object Detection Based on Spatial Attention and Gaussian Mixture
• Improving Network Slimming with Nonconvex Regularization
• BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation
• K for the Price of 1: Parameter-efficient Multi-task and Transfer Learning
• Domain-Specific Batch Normalization for Unsupervised Domain Adaptation
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Rapid protein stability prediction using deep learning representations
• Are Graph Neural Networks Miscalibrated?
• StarGAN v2: Diverse Image Synthesis for Multiple Domains
• GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
• A Self Validation Network for Object-Level Human Attention Estimation
• Revisiting Self-Supervised Visual Representation Learning
• Learning to Blindly Assess Image Quality in the Laboratory and Wild
• Exploring Autoencoder-based Error-bounded Compression for Scientific Data
• DINOSim: Zero-Shot Object Detection and Semantic Segmentation on Microscopy Images
• Averaging Weights Leads to Wider Optima and Better Generalization
• PD-GAN: Probabilistic Diverse GAN for Image Inpainting
• GhostNet: More Features From Cheap Operations
• DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing
• Towards Controllable and Personalized Review Generation
• Generative Modeling via Drifting
• Convergent Discovery of Critical Phenomena Mathematics Across Disciplines
• Comicolorization: Semi-Automatic Manga Colorization
• AdaptCL: Efficient Collaborative Learning with Dynamic and Adaptive Pruning
• AQD: Towards Accurate Fully-Quantized Object Detection
• Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks
• Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-identification
• IBNorm: Information-Bottleneck Inspired Normalization for Representation Learning
• Cost-Sensitive Unbiased Risk Estimation for Multi-Class Positive-Unlabeled Learning
• Test-Time Adaptive Object Detection with Foundation Model
• Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN
• Sim2Real 3D Object Classification using Spherical Kernel Point Convolution and a Deep Center Voting Scheme
• Cluster Dose Prediction in Carbon Ion Therapy: Using Transfer Learning from a Pretrained Dose Prediction U-Net
• HiMAE: Hierarchical Masked Autoencoders Discover Resolution-Specific Structure in Wearable Time Series
• PARN: Position-Aware Relation Networks for Few-Shot Learning
• Mitigating Negative Transfer via Reducing Environmental Disagreement
• ResNet: Enabling Deep Convolutional Neural Networks through Residual Learning
• Leveraging Sequence Embedding and Convolutional Neural Network for Protein Function Prediction
• Progressively Unfreezing Perceptual GAN
• <tt>Effort.jl</tt>: a fast and differentiable emulator for the Effective Field Theory of the Large Scale Structure of the Universe
• TAda! Temporally-Adaptive Convolutions for Video Understanding
• Fast and Flexible Image Blind Denoising via Competition of Experts
• OOS-DSD: Improving Out-of-stock Detection in Retail Images using Auxiliary Tasks
• Autonomous construction of parameterizable 3D leaf models from scanned sweet pepper leaves with deep generative networks
• Hierarchically Robust Representation Learning
• Chebyshev Moment Regularization (CMR): Condition-Number Control with Moment Shaping
• Detecting Interspecific Positive Selection Using Convolutional Neural Networks
• PANet: Perspective-Aware Network with Dynamic Receptive Fields and Self-Distilling Supervision for Crowd Counting
• High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times
• Initialization and Regularization of Factorized Neural Layers
• Floating-Point Neural Network Verification at the Software Level
• The Benchmarking Epistemology: Construct Validity for Evaluating Machine Learning Models
• QoSGMAA: A Robust Multi-Order Graph Attention and Adversarial Framework for Sparse QoS Prediction
• Toward Agents That Reason About Their Computation
• SeeDNorm: Self-Rescaled Dynamic Normalization
• Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections
• GALA: A GlobAl-LocAl Approach for Multi-Source Active Domain Adaptation
• Trajectory Normalized Gradients for Distributed Optimization
• Mint: A Simple Test-Time Adaptation of Vision-Language Models against Common Corruptions
• Energy-Efficient Domain-Specific Artificial Intelligence Models and Agents: Pathways and Paradigms
• The Elastic Lottery Ticket Hypothesis
• Domain Generalization with MixStyle
• Efficient Meningioma Tumor Segmentation Using Ensemble Learning
• Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications
• A Scalable, Causal, and Energy Efficient Framework for Neural Decoding with Spiking Neural Networks
• Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction
• Deep Learning for Face Anti-Spoofing: A Survey
• Convergence Analysis of SGD under Expected Smoothness
• Learned Threshold Pruning
• Predicting the 3D microstructure of SOFC anodes from 2D SEM images using stochastic microstructure modeling and CNNs
• Anderson-type acceleration method for Deep Neural Network optimization
• IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks
• Efficient learning of goal-oriented push-grasping synergy in clutter
• ARC: A Vision-based Automatic Retail Checkout System
• LAFEAT: Piercing Through Adversarial Defenses with Latent Features
• SS-CADA: A Semi-Supervised Cross-Anatomy Domain Adaptation for Coronary Artery Segmentation
• A Unified Perspective on Optimization in Machine Learning and Neuroscience: From Gradient Descent to Neural Adaptation
• NASirt: AutoML based learning with instance-level complexity information
• Single Image Super-Resolution Using Lightweight CNN with Maxout Units
• Ranked List Loss for Deep Metric Learning
• A high-bias, low-variance introduction to Machine Learning for physicists
• Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods
• CytoNet: A Foundation Model for the Human Cerebral Cortex at Cellular Resolution
• Learning from N-Tuple Data with M Positive Instances: Unbiased Risk Estimation and Theoretical Guarantees
• BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation
• Unifying and Enhancing Graph Transformers via a Hierarchical Mask Framework
• Deep Learning-Based Control Optimization for Glass Bottle Forming
• Deep Photovoltaic Nowcasting
• Swin Transformer V2: Scaling Up Capacity and Resolution
• Cortical-SSM: A Deep State Space Model for EEG and ECoG Motor Imagery Decoding
• BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer
• ProDAT: Progressive Density-Aware Tail-Drop for Point Cloud Coding
• Contrastive Adaptive Propagation Graph Neural Networks for Efficient Graph Learning
• Clinical Predictive Models for COVID-19: Systematic Study
• PassREfinder-FL: Privacy-Preserving Credential Stuffing Risk Prediction via Graph-Based Federated Learning for Representing Password Reuse between Websites
• Vision Mamba for Permeability Prediction of Porous Media
• Learning to Find Correlated Features by Maximizing Information Flow in Convolutional Neural Networks
• Adversarial Neuron Pruning Purifies Backdoored Deep Models
• Factor Graph Attention
• ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection
• Manifold Decoders: A Framework for Generative Modeling from Nonlinear Embeddings
• When Embedding Models Meet: Procrustes Bounds and Applications
• Towards robust vision by multi-task learning on monkey visual cortex
• Beyond Pixels: A Differentiable Pipeline for Probing Neuronal Selectivity in 3D
• Using Kolmogorov-Smirnov Distance for Measuring Distribution Shift in Machine Learning
• Behavioral Biometrics for Automatic Detection of User Familiarity in VR
• Dirichlet Energy Constrained Learning for Deep Graph Neural Networks
• Layer-Aware Influence for Online Data Valuation Estimation
• Learning Human Motion with Temporally Conditional Mamba
• MetaFormer Is Actually What You Need for Vision
• On Feature Decorrelation in Self-Supervised Learning
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
• Feature Stylization and Domain-aware Contrastive Learning for Domain Generalization
• R-Drop: Regularized Dropout for Neural Networks
• Joint Discriminative-Generative Modeling via Dual Adversarial Training
• MIEO: encoding clinical data to enhance cardiovascular event prediction
• Generalisation of automatic tumour segmentation in histopathological whole-slide images across multiple cancer types
• Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation
• Transferable, Controllable, and Inconspicuous Adversarial Attacks on Person Re-identification With Deep Mis-Ranking
• Deep semi-supervised approach based on consistency regularization and similarity learning for weeds classification
• Understanding Self-supervised Contrastive Learning through Supervised Objectives
• NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problem
• Ensemble Threshold Calibration for Stable Sensitivity Control
• Learning a Deep ConvNet for Multi-label Classification with Partial Labels
• Distributed Learning of Deep Neural Networks using Independent Subnet Training
• DeepLight: Learning Illumination for Unconstrained Mobile Mixed Reality
• Spin-Weighted Spherical CNNs
• DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentation
• Piecewise Linear Units Improve Deep Neural Networks
• A Review of Learning with Deep Generative Models from Perspective of Graphical Modeling
• Differential Diagnosis for Pancreatic Cysts in CT Scans Using Densely-Connected Convolutional Networks
• Wrapped Loss Function for Regularizing Nonconforming Residual\n Distributions
• Phase-Aware Deep Learning with Complex-Valued CNNs for Audio Signal Applications
• MTMD: A Multi-Task Multi-Domain Framework for Unified Ad Lightweight Ranking at Pinterest
• A Generic Machine Learning Framework for Radio Frequency Fingerprinting
• Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games
• Needles in Haystacks: On Classifying Tiny Objects in Large Images
• Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses
• Learning Regularizers: Learning Optimizers that can Regularize
• Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scale
• Conformal Risk Training: End-to-End Optimization of Conformal Risk Control
• Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks
• RAVEN: A Dataset for Relational and Analogical Visual rEasoNing
• LieTransformer: Equivariant self-attention for Lie Groups
• Deep GAN-Based Cross-Spectral Cross-Resolution Iris Recognition
• Demystifying Deep Learning-based Brain Tumor Segmentation with 3D UNets and Explainable AI (XAI): A Comparative Analysis
• Deep Neural Networks Inspired by Differential Equations
• Inductive biases and Self Supervised Learning in modelling a physical heating system
• Using Deep Neural Networks to Predict and Improve the Performance of\n Polar Codes
• ISTA-NAS: Efficient and Consistent Neural Architecture Search by Sparse Coding
• CLASS: Cross-Level Attention and Supervision for Salient Objects Detection
• Co-VeGAN: Complex-Valued Generative Adversarial Network for Compressive Sensing MR Image Reconstruction
• Random Window Augmentations for Deep Learning Robustness in CT and Liver Tumor Segmentation
• Controllable Orthogonalization in Training DNNs
• Towards democratizing music production with AI-Design of Variational Autoencoder-based Rhythm Generator as a DAW plugin
• Sequential Learning for Domain Generalization
• Entropy Regularizing Activation: Boosting Continuous Control, Large Language Models, and Image Classification with Activation as Entropy Constraints
• Validation of Various Normalization Methods for Brain Tumor Segmentation: Can Federated Learning Overcome This Heterogeneity?
• Pruning Self-attentions into Convolutional Layers in Single Path
• Random Erasing Data Augmentation
• "Tom" pet robot applied to urban autism
• Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation
• How does the optimizer implicitly bias the model merging loss landscape?
• Noise or Signal? Deconstructing Contradictions and An Adaptive Remedy for Reversible Normalization in Time Series Forecasting
• Tensor Regression Networks
• Data-Driven Filtering of the Spherical Harmonics Method
• Meta-Learning with Adaptive Hyperparameters
• Interactive Training: Feedback-Driven Neural Network Optimization
• Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learning
• Discretized Quadratic Integrate-and-Fire Neuron Model for Deep Spiking Neural Networks
• An Efficient Approach to Informative Feature Extraction from Multimodal Data
• A Comprehensive Review of Deep Learning-based Single Image Super-resolution
• FHEON: A Configurable Framework for Developing Privacy-Preserving Neural Networks Using Homomorphic Encryption
• Deep Learning for Image Super-Resolution: A Survey
• Image Enhancement Based on Pigment Representation
• On residual network depth
• On the Benefits of Weight Normalization for Overparameterized Matrix Sensing
• Momentum Contrast for Unsupervised Visual Representation Learning
• Digital Passport: A Novel Technological Strategy for Intellectual Property Protection of Convolutional Neural Networks
• [Extended version] Rethinking Deep Neural Network Ownership\n Verification: Embedding Passports to Defeat Ambiguity Attacks
• Randomized Matrix Sketching for Neural Network Training and Gradient Monitoring
• Generalized Parallel Scaling with Interdependent Generations
• High Fidelity Speech Synthesis with Adversarial Networks
• FedMuon: Federated Learning with Bias-corrected LMO-based Optimization
• Marginal Flow: a flexible and efficient framework for density estimation
• Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifier
• ProbMed: A Probabilistic Framework for Medical Multimodal Binding
• Gamma-Based Statistical Modeling for Extended Target Detection in mmWave Automotive Radar
• AttentionViG: Cross-Attention-Based Dynamic Neighbor Aggregation in Vision GNNs
• XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning
• U-SWIFT: A Unified Surface Wave Inversion Framework with Transformer via Normalization of Dispersion Curves
• Spatial-Functional awareness Transformer-based graph archetype contrastive learning for Decoding Visual Neural Representations from EEG
• Word-Level Emotional Expression Control in Zero-Shot Text-to-Speech Synthesis
• ScatterAD: Temporal-Topological Scattering Mechanism for Time Series Anomaly Detection
• Neural Visibility of Point Sets
• BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression
• Flexible Image Denoising with Multi-layer Conditional Feature Modulation
• ClimSat – A diffusion autoencoder model for climate-conditional satellite image editing
• Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videos
• 2nd Place Solution to ECCV 2020 VIPriors Object Detection Challenge
• Weather GAN: Multi-Domain Weather Translation Using Generative Adversarial Networks
• Tent: Fully Test-time Adaptation by Entropy Minimization
• Understanding the Effects of Pre-Training for Object Detectors via Eigenspectrum
• Differentiable Sparsity via D-Gating: Simple and Versatile Structured Penalization
• CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learning
• An Investigation of Batch Normalization in Off-Policy Actor-Critic Algorithms
• Deep Taxonomic Networks for Unsupervised Hierarchical Prototype Discovery
• Deep Optimization model for Screen Content Image Quality Assessment using Neural Networks
• Challenges in multi-task learning for fMRI-based diagnosis: Benefits for psychiatric conditions and CNVs would likely require thousands of patients
• Inferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images using VAEs
• U-Net Training with Instance-Layer Normalization
• Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation
• Deep Multi-Kernel Convolutional LSTM Networks and an Attention-Based\n Mechanism for Videos
• CProp: Adaptive Learning Rate Scaling from Past Gradient Conformity
• Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations
• Training Deep Normalization-Free Spiking Neural Networks with Lateral Inhibition
• Automatic Analysis System of Calcaneus Radiograph: Rotation-Invariant Landmark Detection for Calcaneal Angle Measurement, Fracture Identification and Fracture Region Segmentation
• Shape-Informed Clustering of Multi-Dimensional Functional Data via Deep Functional Autoencoders
• Learning Unified Representation of 3D Gaussian Splatting
• EqCo: Equivalent Rules for Self-supervised Contrastive Learning
• Restructuring Batch Normalization to Accelerate CNN Training
• Multiple Attentional Pyramid Networks for Chinese Herbal Recognition
• Cross-Dialect Bird Species Recognition with Dialect-Calibrated Augmentation
• Johnson-Lindenstrauss Lemma Guided Network for Efficient 3D Medical Segmentation
• FastEnhancer: Speed-Optimized Streaming Neural Speech Enhancement
• Prompt-guided Disentangled Representation for Action Recognition
• POEM: Explore Unexplored Reliable Samples to Enhance Test-Time Adaptation
• Revisiting Data Challenges of Computational Pathology: A Pack-based Multiple Instance Learning Training Framework
• Plant identification based on noisy web data: the amazing performance of deep learning (LifeCLEF 2017)
• Concept Learners for Few-Shot Learning
• Learning Data-adaptive Nonparametric Kernels
• Learning Student Networks via Feature Embedding
• ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks
• FerretNet: Efficient Synthetic Image Detection via Local Pixel Dependencies
• Learning to see across Domains and Modalities
• A State-of-the-art Survey of Artificial Neural Networks for Whole-slide Image Analysis:from Popular Convolutional Neural Networks to Potential Visual Transformers
• Transformation-based Adversarial Video Prediction on Large-Scale Data
• On the distance between two neural networks and the stability of learning
• Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data
• Meteosat Third Generation imagery improves CNN-based SSI retrieval
• Blind Inpainting of Large-scale Masks of Thin Structures with Adversarial and Reinforcement Learning
• More Data, Worse Decisions? Preference Reversals in Neural Networks under Gram Incompatibility
• Edge Prediction for Roof Wireframe Reconstruction with Transformers
• SNAC-Pack 2.0: Scaled-Out Surrogate Neural Architecture Codesign
• What is the Effect of Importance Weighting in Deep Learning?
• Data-Driven Sparse Structure Selection for Deep Neural Networks
• Mirror Descent View for Neural Network Quantization
• Semantic Understanding of Scenes Through the ADE20K Dataset
• A CNN approach to simultaneously count plants and detect plantation-rows from UAV imagery
• FabToys
• Evidence Networks: simple losses for fast, amortized, neural Bayesian model comparison
• Vegetation structural shift tells environmental changes on the Tibetan Plateau over 40 years
• Composition-Aware Image Aesthetics Assessment
• Thanks for Nothing: Predicting Zero-Valued Activations with Lightweight Convolutional Neural Networks
• Inductive Bias of Gradient Descent based Adversarial Training on Separable Data
• DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition
• A contrastive rule for meta-learning
• An Efficient and Automated Classification System for Rocks Based on Visually Explainable Deep Learning
• LeafDNet: Transforming Leaf Disease Diagnosis Through Deep Transfer Learning
• Accurate Retinal Vessel Segmentation via Octave Convolution Neural Network
• Efficient Heuristic Generation for Robot Path Planning with Recurrent Generative Model
• Learning Equivariant Representations
• Analysis of Hyper-Parameters for Small Games: Iterations or Epochs in Self-Play?
• Multi-concept adversarial attacks
• Understanding Self-supervised Learning with Dual Deep Networks
• Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation
• Scaling Up Exact Neural Network Compression by ReLU Stability
• Optical Wavelength Guided Self-Supervised Feature Learning For Galaxy\n Cluster Richness Estimate
• Self-Supervised Learning for Domain Adaptation on Point-Clouds
• Deep-STORM: super-resolution single-molecule microscopy by deep learning
• An Features Extraction and Recognition Method for Underwater Acoustic Target Based on ATCNN
• ThumbNet: One Thumbnail Image Contains All You Need for Recognition
• 3D Scattering Tomography by Deep Learning with Architecture Tailored to\n Cloud Fields
• Graphs for deep learning representations
• Generalizing Emergent Communication
• LLMulator: Generalizable Cost Modeling for Dataflow Accelerators with Input-Adaptive Control Flow
• Text Steganalysis with Attentional LSTM-CNN
• Focal Frequency Loss for Image Reconstruction and Synthesis
• Probabilistic Runtime Verification, Evaluation and Risk Assessment of Visual Deep Learning Systems
• A Kernel Space-based Multidimensional Sparse Model for Dynamic PET Image Denoising
• MK-UNet: Multi-kernel Lightweight CNN for Medical Image Segmentation
• Machine learning approach to single-shot multiparameter estimation for the non-linear Schrödinger equation
• Robustness Feature Adapter for Efficient Adversarial Training
• End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection
• Exploring Machine Learning Models for Physical Dose Calculation in Carbon Ion Therapy Using Heterogeneous Imaging Data -- A Proof of Concept Study
• On Mutual Information Neural Estimation for Localization
• Modeling Human Motion with Quaternion-Based Neural Networks
• CSDformer: A Conversion Method for Fully Spike-Driven Transformer
• Deep Clustering for Unsupervised Learning of Visual Features
• Attentional Pooling for Action Recognition
• Self-Supervised Discovery of Neural Circuits in Spatially Patterned Neural Responses with Graph Neural Networks
• SOLAR: Switchable Output Layer for Accuracy and Robustness in Once-for-All Training
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Learning Safety for Obstacle Avoidance via Control Barrier Functions
• PlaNet - Photo Geolocation with Convolutional Neural Networks
• PAN: Pillars-Attention-Based Network for 3D Object Detection
• Deep Learning Empowered Super-Resolution: A Comprehensive Survey and Future Prospects
• RangeSAM: On the Potential of Visual Foundation Models for Range-View represented LiDAR segmentation
• The Distribution Shift Problem in Transportation Networks using Reinforcement Learning and AI
• Rethinking "Batch" in BatchNorm
• Towards Multi-Scale Style Control for Expressive Speech Synthesis
• Physics-Informed GCN-LSTM Framework for Long-Term Forecasting of 2D and 3D Microstructure Evolution
• Crafting GBD-Net for Object Detection
• Being Bayesian about Categorical Probability
• Training Larger Networks for Deep Reinforcement Learning
• Unsupervised Object-Level Representation Learning from Scene Images
• Task-agnostic Continual Learning with Hybrid Probabilistic Models
• FedKLPR: Personalized Federated Learning for Person Re-Identification with Adaptive Pruning
• Incorporating Visual Cortical Lateral Connection Properties into CNN: Recurrent Activation and Excitatory-Inhibitory Separation
• Progressive Identification of True Labels for Partial-Label Learning
• Exploring the Relationship between Brain Hemisphere States and Frequency Bands through Deep Learning Optimization Techniques
• DSpAST: Disentangled Representations for Spatial Audio Reasoning with Large Language Models
• Noise Supervised Contrastive Learning and Feature-Perturbed for Anomalous Sound Detection
• Self Identity Mapping
• Plug-and-Play PDE Optimization for 3D Gaussian Splatting: Toward High-Quality Rendering and Reconstruction
• On the Out-of-Distribution Backdoor Attack for Federated Learning
• Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks
• Learning towards Minimum Hyperspherical Energy
• Bridging Performance Gaps for ECG Foundation Models: A Post-Training Strategy
• A Learnable Fully Interacted Two-Tower Model for Pre-Ranking System
• Reversible Deep Equilibrium Models
• CLIFF: Continual Learning for Incremental Flake Features in 2D Material Identification
• Temporal Attentive Alignment for Large-Scale Video Domain Adaptation
• BackPACK: Packing more into backprop
• A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model
• TSIT: A Simple and Versatile Framework for Image-to-Image Translation
• On the Bias-Variance Tradeoff: Textbooks Need an Update
• Denoising IMU Gyroscopes with Deep Learning for Open-Loop Attitude\n Estimation
• SubSpectral Normalization for Neural Audio Data Processing
• Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorization
• Learning Deep Image Priors for Blind Image Denoising
• 3D Quasi-Recurrent Neural Network for Hyperspectral Image Denoising
• On Empirical Comparisons of Optimizers for Deep Learning
• Batch-Shaping for Learning Conditional Channel Gated Networks
• BI-GCN: Boundary-Aware Input-Dependent Graph Convolution Network for Biomedical Image Segmentation
• Learning Deep Representations of Fine-grained Visual Descriptions
• Geolocation-Aware Robust Spoken Language Identification
• Bootstrap your own latent: A new approach to self-supervised Learning
• MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers
• DeepMET: Improving missing transverse momentum estimation with a deep neural network
• Optimizer Fusion: Efficient Training with Better Locality and Parallelism
• Data augmentation and image understanding
• Artificial Neural Networks for Neuroscientists: A Primer
• Is Attention Better Than Matrix Decomposition?
• Jet tagging in the Lund plane with graph networks
• Normalization Techniques in Training DNNs: Methodology, Analysis and Application
• Convolutional Neural Networks for Accurate Measurement of Train Speed
• Impact of a Sharpness Based Loss Function for Removing Out-of-Focus Blur
• Weisfeiler and Lehman Go Topological: Message Passing Simplicial\n Networks
• DeltaConv
• Machine Learning-Driven Predictive Resource Management in Complex Science Workflows
• An Attention-Based Deep Learning Approach for Sleep Stage Classification With Single-Channel EEG
• Promoting Shape Bias in CNNs: Frequency-Based and Contrastive Regularization for Corruption Robustness
• Video Super-resolution with Temporal Group Attention
• Neural networks in the search for fast radio bursts with RATAN-600
• Geometrically Constrained and Token-Based Probabilistic Spatial Transformers
• Optimal message passing for molecular prediction is simple, attentive and spatial
• Competitive Inner-Imaging Squeeze and Excitation for Residual Network
• An Efficient Dual-Line Decoder Network with Multi-Scale Convolutional Attention for Multi-organ Segmentation
• Deep Learning for Generic Object Detection: A Survey
• Rethinking Channel Dimensions for Efficient Model Design
• FedBiF: Communication-Efficient Federated Learning via Bits Freezing
• Towards Stabilizing Batch Statistics in Backward Propagation of Batch Normalization
• A Capsule-unified Framework of Deep Neural Networks for Graphical Programming
• LoFT: Parameter-Efficient Fine-Tuning for Long-tailed Semi-Supervised Learning in Open-World Scenarios
• Purge-Gate: Backpropagation-Free Test-Time Adaptation for Point Clouds Classification via Token Purging
• Multigrid Neural Architectures
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• MSDANet: A Multiscale Dual-Channel Spatial Attention Network with Depthwise Separable Convolution for Hyperspectral Image Classification
• TriagerX: Dual Transformers for Bug Triaging Tasks with Content and Interaction Based Rankings
• Network Pruning via Transformable Architecture Search
• Exploring Simple Siamese Representation Learning
• A holistic approach to polyphonic music transcription with neural\n networks
• Attention-based cross-modal fusion for audio-visual voice activity detection in musical video streams
• Learning Explainable Imaging-Genetics Associations Related to a Neurological Disorder
• Attentional Feature Fusion
• Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization
• Residual Dense Network for Image Restoration
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding
• Variational Depth Search in ResNets
• Fault Tolerant Control of a Quadcopter using Reinforcement Learning
• Root Mean Square Layer Normalization
• Using Optimal Transport Aligned Latent Embeddings for Separated Flow Analysis
• Neural Anisotropy Directions
• Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks
• Breaking the Conventional Forward-Backward Tie in Neural Networks: Activation Functions
• Evaluating the Impact of Adversarial Attacks on Traffic Sign Classification using the LISA Dataset
• Improving Consistency and Correctness of Sequence Inpainting using\n Semantically Guided Generative Adversarial Network
• Integrating Spatial and Semantic Embeddings for Stereo Sound Event Localization in Videos
• RotoGrad: Gradient Homogenization in Multitask Learning
• On the Reproducibility of "FairCLIP: Harnessing Fairness in Vision-Language Learning''
• Lookup multivariate Kolmogorov-Arnold Networks
• Self-Adaptive Training: beyond Empirical Risk Minimization
• Micro-Expression Recognition via Fine-Grained Dynamic Perception
• Regularization with Latent Space Virtual Adversarial Training
• RepVGG: Making VGG-style ConvNets Great Again
• Unity Style Transfer for Person Re-Identification
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
• Fine-grained Video Categorization with Redundancy Reduction Attention
• Deep learning-based phase prediction of high-entropy alloys: Optimization, generation, and explanation
• Unpaired Image Super-Resolution using Pseudo-Supervision
• 3DPillars: Pillar-based two-stage 3D object detection
• Segmentation and Tracking of Eruptive Solar Phenomena with Convolutional Neural Networks
• Intelligent Home 3D: Automatic 3D-House Design from Linguistic Descriptions Only
• Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
• COGITAO: A Visual Reasoning Framework To Study Compositionality &amp; Generalization
• Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization
• Empirical Studies on the Properties of Linear Regions in Deep Neural Networks
• VCMamba: Bridging Convolutions with Multi-Directional Mamba for Efficient Visual Representation
• Approximated Oracle Filter Pruning for Destructive CNN Width Optimization
• Heard More Than Heard: An Audio Steganography Method Based on GAN
• Generalizable Model-agnostic Semantic Segmentation via Target-specific Normalization
• Spatiotemporal Pyramid Network for Video Action Recognition
• Dari Barisan ke Pakatan: berubahnya dinamiks Pilihan Raya UmumKuala Lumpur 1955-2013
• Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization
• Differentiating through the Fréchet Mean
• Simpler Certified Radius Maximization by Propagating Covariances
• Center-based 3D Object Detection and Tracking
• Insights from Gradient Dynamics: Gradient Autoscaled Normalization
• Initialization Schemes for Kolmogorov-Arnold Networks: An Empirical Study
• Temporal social network modeling of mobile connectivity data with graph neural networks
• Deep Learning for High Speed Optical Coherence Elastography with a Fiber Scanning Endoscope
• Understanding deep learning requires rethinking generalization
• Network Implosion: Effective Model Compression for ResNets via Static Layer Pruning and Retraining
• Sharp Minima Can Generalize For Deep Nets
• StableSleep: Source-Free Test-Time Adaptation for Sleep Staging with Lightweight Safety Rails
• Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks
• The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines
• Making EfficientNet More Efficient: Exploring Batch-Independent Normalization, Group Convolutions and Reduced Resolution Training
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Deep neural network for pixel-level electromagnetic particle identification in the MicroBooNE liquid argon time projection chamber
• Deep Group-shuffling Random Walk for Person Re-identification
• Facial Expression Editing with Continuous Emotion Labels
• VISP: Volatility Informed Stochastic Projection for Adaptive Regularization
• ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias
• Training GANs with Stronger Augmentations via Contrastive Discriminator
• A Runtime-Based Computational Performance Predictor for Deep Neural Network Training
• Federated Generative Adversarial Learning
• Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detection
• Stochastic Function Norm Regularization of Deep Networks
• Modeling and benchmarking quantum optical neurons for efficient neural computation
• Physics-Guided Neural Networks for Constructing Nucleon-Nucleon Inverse Potentials
• Domain Adaptation via Feature Refinement
• Generalization by design: Shortcuts to Generalization in Deep Learning
• Towards Out-Of-Distribution Generalization: A Survey
• A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS
• Summarize and Search: Learning Consensus-aware Dynamic Convolution for Co-Saliency Detection
• Learning to match transient sound events using attentional similarity for few-shot sound recognition
• From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation
• CalibDNN: Multimodal Sensor Calibration for Perception Using Deep Neural Networks
• What Expressivity Theory Misses: Message Passing Complexity for GNNs
• 3D Pose Transfer with Correspondence Learning and Mesh Refinement
• Learning to Prune in Training via Dynamic Channel Propagation
• CO2: Consistent Contrast for Unsupervised Visual Representation Learning
• CNN with large memory layers
• Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
• An exact multiple-time-step variational formulation for the committor and the transition rate
• Deep Learning for CMB Foreground Removal and Beam Deconvolution: A U-Net GAN Approach
• Activation Subspaces for Out-of-Distribution Detection
• Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back
• Evaluation Function Approximation for Scrabble
• lifeXplore at the Lifelog Search Challenge 2020
• Scalable Equilibrium Propagation via Intermediate Error Signals for Deep Convolutional CRNNs
• Mini Autonomous Car Driving based on 3D Convolutional Neural Networks
• 3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image
• Rethinking Layer-wise Model Merging through Chain of Merges
• Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learning
• Local Virtual Nodes for Alleviating Over-Squashing in Graph Neural Networks
• Self-organized learning emerges from coherent coupling of critical neurons
• The Pitfall of More Powerful Autoencoders in Lidar-Based Navigation
• SimShear: Sim-to-Real Shear-based Tactile Servoing
• A Chain Graph Interpretation of Real-World Neural Networks
• Improved Adversarial Robustness by Reducing Open Space Risk via Tent Activations
• Learning Local Feature Descriptor with Motion Attribute for Vision-based Localization
• Exploring Randomly Wired Neural Networks for Image Recognition
• Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines
• Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems
• SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
• Filter then Attend: Improving attention-based Time Series Forecasting with Spectral Filtering
• Weisfeiler and Lehman Go Cellular: CW Networks
• The Impact of Reinitialization on Generalization in Convolutional Neural Networks
• CoPE: Conditional image generation using Polynomial Expansions
• How Does Batch Normalization Help Optimization?
• Video Representation Learning with Visual Tempo Consistency
• Whitening for Self-Supervised Representation Learning
• Hierarchical Contrastive Motion Learning for Video Action Recognition
• Backprop with Approximate Activations for Memory-efficient Network Training
• JEDI-linear: Fast and Efficient Graph Neural Networks for Jet Tagging on FPGAs
• Multiscale Deep Equilibrium Models
• DoSReMC: Domain Shift Resilient Mammography Classification using Batch Normalization Adaptation
• GRADSTOP: Early Stopping of Gradient Descent via Posterior Sampling
• A Time Attention based Fraud Transaction Detection Framework
• How Powerful are Graph Neural Networks?
• Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach
• Systematic evaluation of convolution neural network advances on the Imagenet
• OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
• JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video
• Conformalized Exceptional Model Mining: Telling Where Your Model Performs (Not) Well
• Glo-VLMs: Leveraging Vision-Language Models for Fine-Grained Diseased Glomerulus Classification
• Statistical Arbitrage in Options Markets by Graph Learning and Synthetic Long Positions
• URIE: Universal Image Enhancement for Visual Recognition in the Wild
• Convolutional-network models to predict wall-bounded turbulence from wall quantities
• Embedding Transfer with Label Relaxation for Improved Metric Learning
• torchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models
• A Neural Influence Diffusion Model for Social Recommendation
• GraphTSNE: A Visualization Technique for Graph-Structured Data
• Self-supervised learning for multiplexing super-resolution confocal microscopy
• Direct Prediction of Steady-State Flow Fields in Meshed Domain with Graph Networks
• Range-Angle Likelihood Maps for Indoor Positioning Using Deep Neural Networks
• RED-NET: A Recursive Encoder-Decoder Network for Edge Detection
• Deploying Models to Non-participating Clients in Federated Learning without Fine-tuning: A Hypernetwork-based Approach
• Online Ensemble Transformer for Accurate Cloud Workload Forecasting in Predictive Auto-Scaling
• Adversarial Mixing Policy for Relaxing Locally Linear Constraints in Mixup
• Switchable Whitening for Deep Representation Learning
• Discrete Word Embedding for Logical Natural Language Understanding
• Recognizing Handwritten Mathematical Expressions as LaTex Sequences Using a Multiscale Robust Neural Network
• Dissecting Hessian: Understanding Common Structure of Hessian in Neural Networks
• Optimal Transport Based Generative Autoencoders
• Per-Tensor Fixed-Point Quantization of the Back-Propagation Algorithm
• Dual-species atomic absorption image reconstruction using deep neural networks
• Deep Template Matching for Pedestrian Attribute Recognition with the Auxiliary Supervision of Attribute-wise Keypoints
• Noise Matters: Optimizing Matching Noise for Diffusion Classifiers
• Unveiling the Potential: Harnessing Deep Metric Learning to Circumvent Video Streaming Encryption
• Multi-Task Audio Source Separation
• DeepDive: An Integrative Algorithm/Architecture Co-Design for Deep Separable Convolutional Neural Networks
• Are Fourier Neural Operators Really Faster for Time-Domain Wave Propagation?
• Deeply Equal-Weighted Subset Portfolios
• Using Chinese Glyphs for Named Entity Recognition
• DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
• EventNet: Asynchronous Recursive Event Processing
• EDAPT: Towards Calibration-Free BCIs with Continual Online Adaptation
• Efficient Image Denoising Using Global and Local Circulant Representation
• ALTIS: Modernizing GPGPU Benchmarking
• Hey Human, If your Facial Emotions are Uncertain, You Should Use Bayesian Neural Networks!
• Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
• Anomaly Detection for IoT Global Connectivity
• MInDI-3D: Iterative Deep Learning in 3D for Sparse-view Cone Beam Computed Tomography
• Source Printer Identification from Document Images Acquired using Smartphone
• Go Small and Similar: A Simple Output Decay Brings Better Performance
• Arbitrary Marginal Neural Ratio Estimation for Simulation-based\n Inference
• Text-conditioned State Space Model For Domain-generalized Change Detection Visual Question Answering
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• Unsupervised Domain Alignment to Mitigate Low Level Dataset Biases
• Toward Lifelong Learning in Equilibrium Propagation: Sleep-like and Awake Rehearsal for Enhanced Stability
• Controlling Covariate Shift using Balanced Normalization of Weights
• Π-nets: Deep Polynomial Neural Networks
• Distribution Matching in Variational Inference
• Exploiting Layer Normalization Fine-tuning in Visual Transformer Foundation Models for Classification
• Learning Classifiers on Positive and Unlabeled Data with Policy Gradient
• Increasing Information Extraction in Low-Signal Regimes via Multiple Instance Learning
• Adaptive Weight Decay for Deep Neural Networks
• MuSLCAT: Multi-Scale Multi-Level Convolutional Attention Transformer for Discriminative Music Modeling on Raw Waveforms
• Low-Bit Data Processing Using Multiple-Output Spiking Neurons with Non-linear Reset Feedback
• Ensemble-Based Graph Representation of fMRI Data for Cognitive Brain State Classification
• Fast, Convex and Conditioned Network for Multi-Fidelity Vectors and Stiff Univariate Differential Equations
• EchoFree: Towards Ultra Lightweight and Efficient Neural Acoustic Echo Cancellation
• In-Context Reinforcement Learning via Communicative World Models
• Regularized Evolutionary Population-Based Training
• How and Why: Taming Flow Matching for Unsupervised Anomaly Detection and Localization
• Input Invex Neural Network
• Learning from Similarity-Confidence and Confidence-Difference
• Effective Abstract Reasoning with Dual-Contrast Network
• Robust Processing-In-Memory Neural Networks via Noise-Aware Normalization
• Transfer learning of neural surrogates on multifidelity groundwater simulations
• SiCmiR Atlas: Single-Cell miRNA Landscapes Reveals Hub-miRNA and Network Signatures in Human Cancers
• Self-paced Data Augmentation for Training Neural Networks
• Automated ultrasound doppler angle estimation using deep learning
• UNISELF: A Unified Network with Instance Normalization and Self-Ensembled Lesion Fusion for Multiple Sclerosis Lesion Segmentation
• Deep Cross Residual Learning for Multitask Visual Recognition
• PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting
• Spatiotemporal wall pressure forecast of a rectangular cylinder with physics-aware DeepU-Fourier neural network
• On Target Segmentation for Direct Speech Translation
• Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantization
• MoExDA: Domain Adaptation for Edge-based Action Recognition
• Fully-Convolutional Siamese Networks for Object Tracking
• Neural Networks with Orthogonal Jacobian
• Fast Neural Network Adaptation via Parameter Remapping and Architecture Search
• YOLOv1 to YOLOv11: A Comprehensive Survey of Real-Time Object Detection Innovations and Challenges
• Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Exploration
• Stochastic Encodings for Active Feature Acquisition
• Accumulative Poisoning Attacks on Real-time Data
• Spectrum Sensing with Deep Clustering: Label-Free Radio Access Technology Recognition
• The Vanishing Gradient Problem for Stiff Neural Differential Equations
• FASTER Recurrent Networks for Efficient Video Classification
• Perspective from a Broader Context: Can Room Style Knowledge Help Visual Floorplan Localization?
• Boosting Sensitivity to HH→ bb γγ with Graph Neural Networks and XGBoost
• PrivPy: Enabling Scalable and General Privacy-Preserving Machine Learning
• A Conditional GAN for Tabular Data Generation with Probabilistic Sampling of Latent Subspaces
• Image-Based Jet Analysis
• Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling
• Space-Time-Separable Graph Convolutional Network for Pose Forecasting
• Classification with Rejection Based on Cost-sensitive Classification
• Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks
• BS-NAS: Broadening-and-Shrinking One-Shot NAS with Searchable Numbers of Channels
• Do We Need Zero Training Loss After Achieving Zero Training Error?
• Orthonormal Product Quantization Network for Scalable Face Image Retrieval
• Multilingual Political Views of Large Language Models: Identification and Steering
• Pulling Back the Curtain on Deep Networks
• Real-time People Tracking and Identification from Sparse mm-Wave Radar\n Point-clouds
• Deep Likelihood Network for Image Restoration with Multiple Degradation Levels
• Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation
• From Sublinear to Linear: Fast Convergence in Deep Networks via Locally Polyak-Lojasiewicz Regions
• Exploring Probabilistic Modeling Beyond Domain Generalization for Semantic Segmentation
• Neural Bootstrapper
• Relational Data Synthesis using Generative Adversarial Networks: A Design Space Exploration
• Mask-Free Audio-driven Talking Face Generation for Enhanced Visual Quality and Identity Preservation
• Sound2Sight: Generating Visual Dynamics from Sound and Context
• False Negative Distillation and Contrastive Learning for Personalized Outfit Recommendation
• Combolutional Neural Networks
• Improving Protein Gamma-Turn Prediction Using Inception Capsule Networks
• Learning to See Inside Opaque Liquid Containers using Speckle Vibrometry
• Graph Generative Adversarial Networks for Sparse Data Generation in High Energy Physics
• Learning to Paint With Model-based Deep Reinforcement Learning
• Unsupervised Visual Representation Learning by Online Constrained K-Means
• A Multimodal Architecture for Endpoint Position Prediction in Team-based Multiplayer Games
• Normalization (machine learning) [wikipedia]
• Batch normalization [wikipedia]
• Vanishing gradient problem [wikipedia]
• Normalization (statistics) [wikipedia]
• Feature scaling [wikipedia]
• Glossary of artificial intelligence [wikipedia]

Discussions

• Batch Normalization: Accelerating Deep Network Training [pdf] [hn, 134 points, 38 comments]
• Batch Normalization: Accelerating Deep Network Training (2015) [hn, 1 points, 0 comments]
• Ioffe and Szegedy get 4.8% top-5 test on ImageNet(beating recent MSR result) [hn, 1 points, 0 comments]

Related

• Training Deep Neural Networks Without Batch Normalization
• Batch Normalization and the impact of batch structure on the behavior of deep convolution networks
• Unified Batch Normalization: Identifying and Alleviating the Feature Condensation in Batch Normalization and a Unified Framework
• Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks
• How Does Batch Normalization Help Optimization?