Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks

2017/01/20 by Dey, Rahul, Salem, Fathi M. · 39 citations
#FOS: Computer and information sciences #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)

paper · doi:10.48550/arxiv.1701.05923

Abstract

The paper evaluates three variants of the Gated Recurrent Unit (GRU) in recurrent neural networks (RNN) by reducing parameters in the update and reset gates. We evaluate the three variant GRU models on MNIST and IMDB datasets and show that these GRU-RNN variant models perform as well as the original GRU RNN model while reducing the computational expense.

Cited by

• Efficient Automatic Modulation Classification for Next-Generation Wireless Networks
• DK-STN: A Domain Knowledge Embedded Spatio-Temporal Network Model for MJO Forecast
• Low-Rank Compression of Language Models via Differentiable Rank Selection
• GCMCG: A Clustering-Aware Graph Attention and Expert Fusion Network for Multi-Paradigm, Multi-task, and Cross-Subject EEG Decoding
• ParaGate: Parasitic-Driven Domain Adaptation Transfer Learning for Netlist Performance Prediction
• Alzheimers Disease Progression Prediction Based on Manifold Mapping of Irregularly Sampled Longitudinal Data
• Learning to Compress Graphs via Dual Agents for Consistent Topological Robustness Evaluation
• CAMS: Towards Compositional Zero-Shot Learning via Gated Cross-Attention and Multi-Space Disentanglement
• Learning Time-Varying Graph Signals via Koopman
• DoFlow: Causal Generative Flows for Interventional and Counterfactual Time-Series Prediction
• Spiking Neural Networks: The Future of Brain-Inspired Computing
• STAR: A Privacy-Preserving, Energy-Efficient Edge AI Framework for Human Activity Recognition via Wi-Fi CSI in Mobile and Pervasive Computing Environments
• Deep recurrent-convolutional neural network learning and physics Kalman filtering comparison in dynamic load identification
• Convolutional Spiking-based GRU Cell for Spatio-temporal Data
• Beyond Leakage and Complexity: Towards Realistic and Efficient Information Cascade Prediction
• Efficient Large-Deformation Medical Image Registration via Recurrent Dynamic Correlation
• Towards Low-Latency and Adaptive Ransomware Detection Using Contrastive Learning
• VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-end Autonomous Driving
• A Data-Driven Framework for Online Mitigation of False Data Injection Signals in Networked Control Systems
• Attention-Enhanced LSTM Modeling for Improved Temperature and Rainfall Forecasting in Bangladesh
• Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers
• DyMoDreamer: World Modeling with Dynamic Modulation
• Graph Neural Networks with Diversity-aware Neighbor Selection and Dynamic Multi-scale Fusion for Multivariate Time Series Forecasting
• EfficientMIL: Efficient Linear-Complexity MIL Method for WSI Classification
• Unlocking the Power of Mixture-of-Experts for Task-Aware Time Series Analytics
• MARG: MAstering Risky Gap Terrains for Legged Robots with Elevation Mapping
• SeqBattNet: A Discrete-State Physics-Informed Neural Network with Aging Adaptation for Battery Modeling
• Sound Separation and Classification with Object and Semantic Guidance
• Anomaly Detection in Industrial Control Systems Based on Cross-Domain Representation Learning
• Making Judicial Reasoning Visible: Structured Annotation of Holding, Evidentiary Considerations, and Subsumption in Criminal Judgments
• C3DE: Causal-Aware Collaborative Neural Controlled Differential Equation for Long-Term Urban Crowd Flow Prediction
• Knowledge Distillation for Sensing-Assisted Long-Term Beam Tracking in mmWave Communications
• Defining, Understanding, and Detecting Online Toxicity: Challenges and Machine Learning Approaches
• AI-Generated Content in Cross-Domain Applications: Research Trends, Challenges and Propositions
• TransMPC: Transformer-based Explicit MPC with Variable Prediction Horizon
• FireRedChat: A Pluggable, Full-Duplex Voice Interaction System with Cascaded and Semi-Cascaded Implementations
• Robust Anomaly Detection through Multi-Modal Autoencoder Fusion for Small Vehicle Damage Detection
• Dissecting Generalized Category Discovery: Multiplex Consensus under Self-Deconstruction
• Systolic Array-based Accelerator for Structured State-Space Models

Related

• Simplified Minimal Gated Unit Variations for Recurrent Neural Networks
• Minimal Gated Unit for Recurrent Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Deep Gate Recurrent Neural Network
• Gated Feedback Recurrent Neural Networks