FastONN -- Python based open-source GPU implementation for Operational Neural Networks

2020/06/03 by Junaid Malik, Serkan Kıranyaz, Malik, Junaid +3 · 4 citations
Computer Science · Physics and Astronomy · #Computational Physics and Python Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.2006.02267

openalex publication_date 2020/06/03 · openalex created_date 2020/06/12 · openalex updated_date 2026/07/28

Abstract

Operational Neural Networks (ONNs) have recently been proposed as a special class of artificial neural networks for grid structured data. They enable heterogenous non-linear operations to generalize the widely adopted convolution-based neuron model. This work introduces a fast GPU-enabled library for training operational neural networks, FastONN, which is based on a novel vectorized formulation of the operational neurons. Leveraging on automatic reverse-mode differentiation for backpropagation, FastONN enables increased flexibility with the incorporation of new operator sets and customized gradient flows. Additionally, bundled auxiliary modules offer interfaces for performance tracking and checkpointing across different data partitions and customized metrics.

Cited by

• CASR-Net: An Image Processing-focused Deep Learning-based Coronary Artery Segmentation and Refinement Network for X-ray Coronary Angiogram
• Early Bearing Fault Diagnosis of Rotating Machinery by 1D Self-Organized\n Operational Neural Networks
• Convolutional versus Self-Organized Operational Neural Networks for Real-World Blind Image Denoising
• 1D convolutional neural networks and applications: A survey
• Expert Operational GANS: Towards Real-Color Underwater Image Restoration

Related

• MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training
• Benchmarking GPU and TPU Performance with Graph Neural Networks
• GPU Asynchronous Stochastic Gradient Descent to Speed Up Neural Network Training
• AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learning
• Large Scale Artificial Neural Network Training Using Multi-GPUs