Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

2015/02/06 by Kaiming He, He, Kaiming, Xiangyu Zhang +5 · 3 voices · 316 citations
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Visual Attention and Saliency Detection #cs.AI #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.1502.01852

openalex publication_date 2015/02/06 · openalex created_date 2019/06/27 · openalex updated_date 2026/07/28

Abstract

Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on our PReLU networks (PReLU-nets), we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66%). To our knowledge, our result is the first to surpass human-level performance (5.1%, Russakovsky et al.) on this visual recognition challenge.

Cited by

• Correlating Cross-Iteration Noise for DP-SGD using Model Curvature
• Efficiently Training Time-to-First-Spike Spiking Neural Networks from Scratch
• Differentially Private Neural Network Training Under the Hidden State Assumption
• Axolotl3D: a Unified Framework for Faithful 3D Shape Completion
• Soft-TransFormers for Continual Learning
• The pretraining domain outweighs the training objective in setting the privacy-utility trade-off of differentially private medical image analysis
• Breaking the Block: Preserving Data Continuity to Train Superior SAEs for Instruct Models
• Lookahead Branching for Neural Network Verification
• Building a Neural Network from Scratch: Implementation, Evaluation, and Optimization
• CoG-Guided Weight Correction for Fault-Tolerant Deep Neural Networks
• Beyond Unfolding: 60x Faster One-Stage Unmixing for Closely-Spaced Infrared Small Targets
• NORACL: Neurogenesis for Oracle-free Resource-Adaptive Continual Learning
• Similarity-Based Bike Station Expansion via Hybrid Denoising Autoencoders
• Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights
• Chameleon: A Multiplier-Free Temporal Convolutional Network Accelerator for End-to-End Few-Shot and Continual Learning from Sequential Data
• Deciphering the 3D genome organization across species from Hi-C data
• Normalisation and Initialisation Strategies for Graph Neural Networks in Blockchain Anomaly Detection
• A Network of Biologically Inspired Rectified Spectral Units (ReSUs) Learns Hierarchical Features Without Error Backpropagation
• Dual-attention ResNet outperforms transformers in HER2 prediction on DCE-MRI
• Projections of Lightning‐Ignited Wildfire Risk in the Western United States
• Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models
• NNN: Next-Generation Neural Networks for Marketing Measurement
• Bilevel gradient methods and the Morse parametric qualification condition
• Flow2GAN: Hybrid Flow Matching and GAN with Multi-Resolution Network for Few-step High-Fidelity Audio Generation
• Investigating Deep Learning Models for Ejection Fraction Estimation from Echocardiography Videos
• AFA-LoRA: Enabling Non-Linear Adaptations in LoRA with Activation Function Annealing
• Müntz-Szász Networks: Neural Architectures with Learnable Power-Law Bases
• Generative Adversarial Networks for High-Dimensional Item Factor Analysis: A Deep Adversarial Learning Algorithm
• Algorithmic Separation between Constant-Depth and Logarithmic-Depth Neural Networks
• Simplex Demixing: Disentangling Multiple Light-Flavor Jets at Colliders
• Calibrated Partial Resets: Preventing Policy Collapse in Continual Reinforcement Learning
• Explainable Multimodal Regression via Information Decomposition
• More Consistent Accuracy PINN via Alternating Easy-Hard Training
• GraviBERT: Transformer-based inference for gravitational-wave time series
• Machine-learning techniques for model-independent searches in dijet final states
• Orthogonal Activation with Implicit Group-Aware Bias Learning for Class Imbalance
• Spectral or spatial? Leveraging both for speaker extraction in challenging data conditions
• ReasonCD: A Multimodal Reasoning Large Model for Implicit Change-of-Interest Semantic Mining
• DeepGESI: A Non-Intrusive Objective Evaluation Model for Predicting Speech Intelligibility in Hearing-Impaired Listeners
• Adaptive Probability Flow Residual Minimization for High-Dimensional Fokker-Planck Equations
• Approximation and learning with compositional tensor trains
• SG-RIFE: Semantic-Guided Real-Time Intermediate Flow Estimation with Diffusion-Competitive Perceptual Quality
• When Does Learning Renormalize? Sufficient Conditions for Power Law Spectral Dynamics
• Dispatch-Aware Deep Neural Network for Optimal Transmission Switching
• MULTIAQUA: A multimodal maritime dataset and robust training strategies for multimodal semantic segmentation
• Semi-Supervised Online Learning on the Edge by Transforming Knowledge from Teacher Models
• Learning holographic QCD with unflavoured meson spectra
• StarCraft+: Benchmarking Multi-agent Algorithms in Adversary Paradigm
• Panther: A Cost-Effective Privacy-Preserving Framework for GNN Training and Inference Services in Cloud Environments
• Deep Reinforcement Learning for EH-Enabled Cognitive-IoT Under Jamming Attacks
• Deep Learning-Driven Quantitative Spectroscopic Photoacoustic Imaging for Segmentation and Oxygen Saturation Estimation
• Deep learning water-unsuppressed MRSI at ultra-high field for simultaneous quantitative metabolic, susceptibility and myelin water imaging
• How Does Fourier Analysis Network Work? A Mechanism Analysis and a New Dual-Activation Layer Proposal
• Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domains
• MeltwaterBench: Deep learning for spatiotemporal downscaling of surface meltwater
• Neural Network-based Partial-Linear Single-Index Models for Environmental Mixtures Analysis
• A Model-Guided Neural Network Method for the Inverse Scattering Problem
• Analysis of Dirichlet Energies as Over-smoothing Measures
• Complexity of One-Dimensional ReLU DNNs
• A Large-Scale Multimodal Dataset and Benchmarks for Human Activity Scene Understanding and Reasoning
• A Patient-Doctor-NLP-System to contest inequality for less privileged
• Learning to Hedge Swaptions
• Theoretical Compression Bounds for Wide Multilayer Perceptrons
• On fine-tuning Boltz-2 for protein-protein affinity prediction
• A new initialisation to Control Gradients in Sinusoidal Neural network
• SPOOF: Simple Pixel Operations for Out-of-Distribution Fooling
• LDLT L-Lipschitz Network: Generalized Deep End-To-End Lipschitz Network Construction
• Gradient Descent with Provably Tuned Learning-rate Schedules
• STELLA: Guiding Large Language Models for Time Series Forecasting with Semantic Abstractions
• Matrix-Free Photoacoustic Image Reconstruction via Sensor-Token Self-Attention
• Parabolic Position Encoding: Vision-Centric, Principled, Extrapolatable, General
• SignRoundV2: Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs
• Mitigating the Curse of Detail: Scaling Arguments for Feature Learning and Sample Complexity
• Semantic-aware Random Convolution and Source Matching for Domain Generalization in Medical Image Segmentation
• Upper Approximation Bounds for Neural Oscillators
• One Swallow Does Not Make a Summer: Understanding Semantic Structures in Embedding Spaces
• Learning Neural Network Safe Tracking Controllers from Backward Reachable Sets
• A Unified and Stable Risk Minimization Framework for Weakly Supervised Learning with Theoretical Guarantees
• Scale-Agnostic Kolmogorov-Arnold Geometry in Neural Networks
• SUPN: Shallow Universal Polynomial Networks
• Adaptive Gradient Descent MPPT Algorithm With Complexity-Aware Benchmarking for Low-Power PV Systems
• Effects of Initialization Biases on Deep Neural Network Training Dynamics
• Physics Enhanced Deep Surrogates for the Phonon Boltzmann Transport Equation
• RIS-Assisted Downlink Pinching-Antenna Systems: GNN-Enabled Optimization Approaches
• Advancing Image Classification with Discrete Diffusion Classification Modeling
• Deterministic Continuous Replacement: Fast and Stable Module Replacement in Pretrained Transformers
• Learning Scalable Temporal Representations in Spiking Neural Networks Without Labels
• Towards Open-Ended Visual Scientific Discovery with Sparse Autoencoders
• Robust Detection of Retinal Neovascularization in Widefield Optical Coherence Tomography
• Deep Learning Framework for Enhanced Neutrino Reconstruction of Single-line Events in the ANTARES Telescope
• Change-of-Basis Pruning via Rotational Invariance
• Addressing the gravitational collapse of a massless scalar field with Physics-Informed Neural Networks
• Batch Matrix-form Equations and Implementation of Multilayer Perceptrons
• Compiling to linear neurons
• Improving Continual Learning of Knowledge Graph Embeddings via Informed Initialization
• N-ReLU: Zero-Mean Stochastic Extension of ReLU
• TuckA: Hierarchical Compact Tensor Experts for Efficient Fine-Tuning
• Superhuman AI for Stratego Using Self-Play Reinforcement Learning and Test-Time Search
• Point Cloud Segmentation of Integrated Circuits Package Substrates Surface Defects Using Causal Inference: Dataset Construction and Methodology
• Identification of tau leptons using a convolutional neural network with domain adaptation
• DL101 Neural Network Outputs and Loss Functions
• Self-adaptive weighting and sampling for physics-informed neural networks
• Accelerating metamaterial topology optimization using deep super-resolution networks
• TwIST: Rigging the Lottery in Transformers with Independent Subnetwork Training
• Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?
• Depth-induced NTK: Bridging Over-parameterized Neural Networks and Deep Neural Kernels
• SCALE: Upscaled Continual Learning of Large Language Models
• Cannistraci-Hebb Training on Ultra-Sparse Spiking Neural Networks
• Why Not Put a Microphone Near the Loudspeaker? A New Paradigm for Acoustic Echo Cancellation
• Condition Numbers and Eigenvalue Spectra of Shallow Networks on Spheres
• Object Detection as an Optional Basis: A Graph Matching Network for Cross-View UAV Localization
• It's Much Easier for Neural Networks to learn Game of Life Dynamics with the Right Activation Function: Polynomial Kolmogorov-Arnold Networks
• TweetyBERT: Automated parsing of birdsong through self-supervised machine learning
• Scene-Centric Unsupervised Video Panoptic Segmentation
• OpenMAP-BrainAge: generalizable and interpretable brain age predictor from MRI
• Symmetry and Generalisation in Neural Approximations of Renormalisation Transformations
• Generative Bayesian Optimization: Generative Models as Acquisition Functions
• Cost-Sensitive Unbiased Risk Estimation for Multi-Class Positive-Unlabeled Learning
• A Quadratic Actor Network for Model-Free Reinforcement Learning
• Topic Analysis with Side Information: A Neural-Augmented LDA Approach
• SHA-256 Infused Embedding-Driven Generative Modeling of High-Energy Molecules in Low-Data Regimes
• LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis
• Deep Learning-Enhanced Calibration of the Heston Model: A Unified Framework
• OOS-DSD: Improving Out-of-stock Detection in Retail Images using Auxiliary Tasks
• Alias-Free ViT: Fractional Shift Invariance via Linear Attention
• If You Want to Be Robust, Be Wary of Initialization
• EBOP MAVEN: A machine learning model to estimate the input parameters for analytic fitting of detached eclipsing binary light curves
• Model Merging with Functional Dual Anchors
• ParaRNN: Unlocking Parallel Training of Nonlinear RNNs for Large Language Models
• Convergence Analysis of SGD under Expected Smoothness
• General transformation neural networks: A class of parametrized functions for high-dimensional function approximation
• Physics-Informed Neural Networks for Solving Multiscale Mode-Resolved Phonon Boltzmann Transport Equation
• FPT-Noise: Dynamic Scene-Aware Counterattack for Test-Time Adversarial Defense in Vision-Language Models
• A Unified Perspective on Optimization in Machine Learning and Neuroscience: From Gradient Descent to Neural Adaptation
• The Emergence of Complex Behavior in Large-Scale Ecological Environments
• Contextual Attention Modulation: Towards Efficient Multi-Task Adaptation in Large Language Models
• Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability
• On the Generalization Properties of Learning the Random Feature Models with Learnable Activation Functions
• Robust Layerwise Scaling Rules by Proper Weight Decay Tuning
• From Universal Approximation Theorem to Tropical Geometry of Multi-Layer Perceptrons
• A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space
• Prompt-based Adaptation in Large-scale Vision Models: A Survey
• Pretraining in Actor-Critic Reinforcement Learning for Robot Locomotion
• Ivan-ISTD: Rethinking Cross-domain Heteroscedastic Noise Perturbations in Infrared Small Target Detection
• Learning at the Speed of Physics: Equilibrium Propagation on Oscillator Ising Machines
• OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting during Parameter-Efficient Fine-Tuning
• Generalisation of automatic tumour segmentation in histopathological whole-slide images across multiple cancer types
• Decoupled Multimodal Fusion for User Interest Modeling in Click-Through Rate Prediction
• Enhancing Phase Transition Calculations with Fitting and Neural Network
• A Copula-based variational autoencoder for uncertainty quantification in inverse problems: application to damage identification in an offshore wind turbine
• Phase-Aware Deep Learning with Complex-Valued CNNs for Audio Signal Applications
• Holistic Order Prediction in Natural Scenes
• Cluster-Aware Prompt Ensemble Learning for Few-Shot Vision-Language Model Adaptation
• Design Principles for Sequence Models via Coefficient Dynamics
• Rewiring Development in Brain Segmentation: Leveraging Adult Brain Priors for Enhancing Infant MRI Segmentation
• Weight Initialization and Variance Dynamics in Deep Neural Networks and Large Language Models
• NRRS: Neural Russian Roulette and Splitting
• Symmetry-Aware Fully-Amortized Optimization with Scale Equivariant Graph Metanetworks
• Integral Signatures of Activation Functions: A 9-Dimensional Taxonomy and Stability Theory for Deep Learning
• To Ask or Not to Ask: Learning to Require Human Feedback
• Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder
• Is the Hard-Label Cryptanalytic Model Extraction Really Polynomial?
• A Comparative Analysis of Contextual Representation Flow in State-Space and Transformer Architectures
• Wide Neural Networks as a Baseline for the Computational No-Coincidence Conjecture
• MASA: Rethinking the Representational Bottleneck in LoRA with Multi-A Shared Adaptation
• Riddled basin geometry sets fundamental limits to predictability and reproducibility in deep learning
• Control of dynamical systems with neural networks
• Learning to Interpret Weight Differences in Language Models
• ERDE: Entropy-Regularized Distillation for Early-exit
• Scalable Causal Discovery from Recursive Nonlinear Data via Truncated Basis Function Scores and Tests
• Multi-Source Fusion and Automatic Predictor Selection for Zero-Shot Video Object Segmentation
• Arithmetic-Mean μP for Modern Architectures: A Unified Learning-Rate Scale for CNNs and ResNets
• Super-resolution image projection over an extended depth of field using a diffractive decoder
• Optimizing Fine-Tuning through Advanced Initialization Strategies for Low-Rank Adaptation
• Learning Explicit Single-Cell Dynamics Using ODE Representations
• Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification
• Curl Descent: Non-Gradient Learning Dynamics with Sign-Diverse Plasticity
• Kolmogorov-Arnold Networks in Thermoelectric Materials Design
• Indirect Attention: Turning Context Misalignment into a Feature
• Noise-Guided Transport for Imitation Learning
• Echoes of Humanity: Exploring the Perceived Humanness of AI Music
• Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
• Spontaneous High-Order Generalization in Neural Theory-of-Mind Networks
• Classifier-Centric Adaptive Framework for Open-Vocabulary Camouflaged Object Segmentation
• Wavelet-Assisted Mamba for Satellite-Derived Sea Surface Temperature Super-Resolution
• Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition
• Interpretable Self-Supervised Learning via Representer Landmarks and Nyström Approximation
• Gradient Flow Convergence Guarantee for General Neural Network Architectures
• AISHELL6-whisper: A Chinese Mandarin Audio-visual Whisper Speech Dataset with Speech Recognition Baselines
• LORT: Locally Refined Convolution and Taylor Transformer for Monaural Speech Enhancement
• Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations
• Training Deep Normalization-Free Spiking Neural Networks with Lateral Inhibition
• Mining Domain Knowledge: Improved Framework towards Automatically Standardizing Anatomical Structure Nomenclature in Radiotherapy
• Activation Function Design Sustains Plasticity in Continual Learning
• Spectral Collapse Drives Loss of Plasticity in Deep Continual Learning
• Enhancing Low-Rank Adaptation with Structured Nonlinear Transformations
• Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers
• Shapley Features for Robust Signal Prediction in Tactile Internet
• Explaining Grokking and Information Bottleneck through Neural Collapse Emergence
• Segmentation of optic disc, fovea and retinal vasculature using a single\n convolutional neural network
• Predicting Path Failure In Time-Evolving Graphs
• Modular Machine Learning with Applications to Genetic Circuit Composition
• Deep Control - a simple automatic gain control for memory efficient and\n high performance training of deep convolutional neural networks
• Convolutional neural shading for high-quality 3D reconstruction from multi-view images
• Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution
• EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images
• Shared-Weights Extender and Gradient Voting for Neural Network Expansion
• Online and Offline Handwritten Chinese Character Recognition: A Comprehensive Study and New Benchmark
• A Scalable Lift-and-Project Differentiable Approach For the Maximum Cut Problem
• M4SER: Multimodal, Multirepresentation, Multitask, and Multistrategy Learning for Speech Emotion Recognition
• Can LLMs Reason Over Non-Text Modalities in a Training-Free Manner? A Case Study with In-Context Representation Learning
• Revisiting Vision Language Foundations for No-Reference Image Quality Assessment
• An Unlearning Framework for Continual Learning
• The evolution of neural network-based chart patterns
• Fast, Better Training Trick -- Random Gradient
• PMRT: A Training Recipe for Fast, 3D High-Resolution Aerodynamic Prediction
• Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
• Delving into Cryptanalytic Extraction of PReLU Neural Networks
• Towards Optimal Convolutional Transfer Learning Architectures for Breast Lesion Classification and ACL Tear Detection
• Learning Safety for Obstacle Avoidance via Control Barrier Functions
• Blind Room Impulse Response Identification via Reverberant Speech Spectrum Reconstruction
• CoDoL: Conditional Domain Prompt Learning for Out-of-Distribution Generalization
• Parallel Simulation of Contact and Actuation for Soft Growing Robots
• Template-Based Cortical Surface Reconstruction with Minimal Energy Deformation
• Towards Privacy-Preserving and Heterogeneity-aware Split Federated Learning via Probabilistic Masking
• Jim137/qkan: v0.1.0
• Learning Minimal Representations of Many-Body Physics from Snapshots of a Quantum Simulator
• Developing Training Procedures for Piecewise-linear Spline Activation Functions in Neural Networks
• A Learnable Fully Interacted Two-Tower Model for Pre-Ranking System
• Don't Forget the Nonlinearity: Unlocking Activation Functions in Efficient Fine-Tuning
• Entanglement and optimization within autoregressive neural quantum states
• A Deep Learning Approach to Renewable Capacity Installation under Jump Uncertainty
• Real-Time Super-Resolution Imaging System Based on Zero-Shot Learning for Infrared Non-Destructive Testing
• Adapting Medical Vision Foundation Models for Volumetric Medical Image Segmentation via Active Learning and Selective Semi-supervised Fine-tuning
• FedBiF: Communication-Efficient Federated Learning via Bits Freezing
• Soft Autoencoder and Its Wavelet Adaptation Interpretation
• FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution
• MSPCaps: A Multi-Scale Patchify Capsule Network with Cross-Agreement Routing for Visual Recognition
• ViewSparsifier: Killing Redundancy in Multi-View Plant Phenotyping
• A machine learning assistant for detecting fraudulent activities in synchronous online programming exams
• Unsupervised Regenerative Learning of Hierarchical Features in Spiking\n Deep Networks for Object Recognition
• Scale Normalization
• Depth-Aware Initialization for Stable and Efficient Neural Network Training
• Differential Morphological Profile Neural Networks for Semantic Segmentation
• SAC-MIL: Spatial-Aware Correlated Multiple Instance Learning for Histopathology Whole Slide Image Classification
• Initialization Schemes for Kolmogorov-Arnold Networks: An Empirical Study
• TeRA: Vector-based Random Tensor Network for High-Rank Adaptation of Large Language Models
• Silent Until Sparse: Backdoor Attacks on Semi-Structured Sparsity
• Sharp Minima Can Generalize For Deep Nets
• MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices
• Thermodynamic uncertainty relation for generalized time-reversal observables
• Social gaze fingerprints: identifying social virtual reality users by their eye gaze patterns
• SC-GIR: Goal-oriented Semantic Communication via Invariant Representation Learning
• MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper
• ER-LoRA: Effective-Rank Guided Adaptation for Weather-Generalized Depth Estimation
• Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
• Scalable Equilibrium Propagation via Intermediate Error Signals for Deep Convolutional CRNNs
• Adaptive Heavy-Tailed Stochastic Gradient Descent
• Self-organized learning emerges from coherent coupling of critical neurons
• Dual-Model Weight Selection and Self-Knowledge Distillation for Medical Image Classification
• Distance-informed Neural Processes
• Can Classical Initialization Help Variational Quantum Circuits Escape the Barren Plateau?
• Systematic evaluation of convolution neural network advances on the Imagenet
• Hybrid Least Squares/Gradient Descent Methods for DeepONets
• LyLA-Therm: Lyapunov-based Langevin Adaptive Thermodynamic Neural Network Controller
• EffiFusion-GAN: Efficient Fusion Generative Adversarial Network for Speech Enhancement
• NeRC: Neural Ranging Correction through Differentiable Moving Horizon Location Estimation
• One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression
• Refining Contrastive Learning and Homography Relations for Multi-Modal Recommendation
• Text2Weight: Bridging Natural Language and Neural Network Weight Spaces
• SIS-Challenge: Event-based Spatio-temporal Instance Segmentation Challenge at the CVPR 2025 Event-based Vision Workshop
• A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks
• Optimal Condition for Initialization Variance in Deep Neural Networks: An SGD Dynamics Perspective
• Mesh Processing Non-Meshes via Neural Displacement Fields
• Dense Regression Activation Maps For Lesion Segmentation in CT scans of COVID-19 patients
• Dual-species atomic absorption image reconstruction using deep neural networks
• Finite-Width Neural Tangent Kernels from Feynman Diagrams
• SimAQ: Mitigating Experimental Artifacts in Soft X-Ray Tomography using Simulated Acquisitions
• GNN-based Unified Deep Learning
• Analysis of the Compaction Behavior of Textile Reinforcements in Low-Resolution In-Situ CT Scans via Machine-Learning and Descriptor-Based Methods
• What-Meets-Where: Unified Learning of Action and Contact Localization in a New Dataset
• Towards Scalable Lottery Ticket Networks using Genetic Algorithms
• Solving Approximation Tasks with Greedy Deep Kernel Methods
• Real-time forecasting of chaotic dynamics from sparse data and autoencoders
• Towards Multimodal Sentiment Analysis via Contrastive Cross-modal Retrieval Augmentation and Hierachical Prompts
• FIVA: Federated Inverse Variance Averaging for Universal CT Segmentation with Uncertainty Estimation
• Differentiable Grouped Feedback Delay Networks for Learning Coupled Volume Acoustics
• Rotation Equivariant Arbitrary-scale Image Super-Resolution
• FedGIN: Federated Learning with Dynamic Global Intensity Non-linear Augmentation for Organ Segmentation using Multi-modal Images
• ULU: A Unified Activation Function
• Learning from Oblivion: Predicting Knowledge Overflowed Weights via Retrodiction of Forgetting
• Gaussian mixture layers for neural networks
• LA-CaRe-CNN: Cascading Refinement CNN for Left Atrial Scar Segmentation
• Augmentation-based Domain Generalization and Joint Training from Multiple Source Domains for Whole Heart Segmentation
• Scalable neural network-based blackbox optimization
• Automatic deep learning for trend prediction in time series data
• Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
• Neural Networks with Orthogonal Jacobian
• LOST: Low-rank and Sparse Pre-training for Large Language Models
• IAUNet: Instance-Aware U-Net
• Towards Generalizable AI-Generated Image Detection via Image-Adaptive Prompt Learning
• Shape Distribution Matters: Shape-specific Mixture-of-Experts for Amodal Segmentation under Diverse Occlusions
• The Vanishing Gradient Problem for Stiff Neural Differential Equations
• Explaining GNN Explanations with Edge Gradients
• Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product
• Large-Scale Evolution of Image Classifiers
• Reinitializing weights vs units for maintaining plasticity in neural networks
• A holomorphic Kolmogorov-Arnold network framework for solving elliptic problems on arbitrary 2D domains
• Prediction of acoustic field in 1-D uniform duct with varying mean flow and temperature using neural networks
• Emergent interactions lead to collective frustration in robotic matter
• Mitigating Imperfections in Mixed-Signal Neuromorphic Circuits
• End-to-End DOA-Guided Speech Extraction in Noisy Multi-Talker Scenarios
• Learning Neural Vocoder from Range-Null Space Decomposition
• Maximize margins for robust splicing detection
• What Can Grokking Teach Us About Learning Under Nonstationarity?
• Graph Representation Learning via Graphical Mutual Information Maximization

Discussions

• Surpassing Human-Level Performance on ImageNet Classification [pdf] [hn, 53 points, 13 comments]
• Surpassing Human-Level Performance on ImageNet Classification [hn, 9 points, 0 comments]
• Deep learning surpasses humans on the most recent #AI vision benchmark | http://arxiv.org/abs/1502.01852 #ArtificialIntelligence [bsky, 0 points, 0 comments]

Related

• RecNets: Channel-wise Recurrent Convolutional Neural Networks
• Dynamic ReLU
• Empirical Evaluation of Rectified Activations in Convolutional Network
• ImageNet classification with deep convolutional neural networks
• Deep Global-Connected Net With The Generalized Multi-Piecewise ReLU Activation in Deep Learning