2021/07/12 by Pavlo Molchanov, Jimmy Hall, Molchanov, Pavlo +9 · 1 citation
Computer Science · Neuroscience · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Brain Tumor Detection and Classification #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.2107.10624
openalex publication_date 2021/07/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We introduce latency-aware network acceleration (LANA) - an approach that builds on neural architecture search techniques and teacher-student distillation to accelerate neural networks. LANA consists of two phases: in the first phase, it trains many alternative operations for every layer of the teacher network using layer-wise feature map distillation. In the second phase, it solves the combinatorial selection of efficient operations using a novel constrained integer linear optimization (ILP) approach. ILP brings unique properties as it (i) performs NAS within a few seconds to minutes, (ii) easily satisfies budget constraints, (iii) works on the layer-granularity, (iv) supports a huge search space O(10100), surpassing prior search approaches in efficacy and efficiency. In extensive experiments, we show that LANA yields efficient and accurate models constrained by a target latency budget, while being significantly faster than other techniques. We analyze three popular network architectures: EfficientNetV1, EfficientNetV2 and ResNeST, and achieve accuracy improvement for all models (up to 3.0%) when compressing larger models to the latency level of smaller models. LANA achieves significant speed-ups (up to 5×) with minor to no accuracy drop on GPU and CPU. The code will be shared soon.