FastFusionNet: New State-of-the-Art for DAWNBench SQuAD

2019/02/28 by Felix Wu, Wu, Felix, Boyi Li +9 · 1 voice
Computer Science · #Anomaly Detection Techniques and Applications #Computation and Language (cs.CL) #FOS: Computer and information sciences #Speech Recognition and Synthesis #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1902.11291

openalex publication_date 2019/02/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this technical report, we introduce FastFusionNet, an efficient variant of FusionNet [12]. FusionNet is a high performing reading comprehension architecture, which was designed primarily for maximum retrieval accuracy with less regard towards computational requirements. For FastFusionNets we remove the expensive CoVe layers [21] and substitute the BiLSTMs with far more efficient SRU layers [19]. The resulting architecture obtains state-of-the-art results on DAWNBench [5] while achieving the lowest training and inference time on SQuAD [25] to-date. The code is available at https://github.com/felixgwu/FastFusionNet.

Citations

• Variational Dropout and the Local Reparameterization Trick
• Long Short-Term Memory
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
• Bidirectional Attention Flow for Machine Comprehension
• Bi-Directional Block Self-Attention for Fast and Memory-Efficient\n Sequence Modeling
• Pay Less Attention with Lightweight and Dynamic Convolutions

Discussions

• FastFusionNet: New State-of-the-Art for DAWNBench SQuAD [hn, 2 points, 0 comments]

Related

• FFN Fusion: Rethinking Sequential Computation in Large Language Models
• FlashFuser: Expanding the Scale of Kernel Fusion for Compute-Intensive Operators via Inter-Core Connection
• FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension
• Multi-Level Quickening: Ten Years Later
• ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive