2025/05/10 by Tianhao Cai, Liang Wang, Cai, Tianhao +11
Computer Science · Neuroscience · #Artificial Intelligence (cs.AI) #Brain Tumor Detection and Classification #Caching and Content Delivery #FOS: Computer and information sciences #Hardware Architecture (cs.AR) #IoT and Edge/Fog Computing #Operating Systems (cs.OS)
paper · pdf · doi:10.48550/arxiv.2505.06625
openalex publication_date 2025/05/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
With the rapid development of DNN applications, multi-tenant execution, where multiple DNNs are co-located on a single SoC, is becoming a prevailing trend. Although many methods are proposed in prior works to improve multi-tenant performance, the impact of shared cache is not well studied. This paper proposes CaMDN, an architecture-scheduling co-design to enhance cache efficiency for multi-tenant DNNs on integrated NPUs. Specifically, a lightweight architecture is proposed to support model-exclusive, NPU-controlled regions inside shared cache to eliminate unexpected cache contention. Moreover, a cache scheduling method is proposed to improve shared cache utilization. In particular, it includes a cache-aware mapping method for adaptability to the varying available cache capacity and a dynamic allocation algorithm to adjust the usage among co-located DNNs at runtime. Compared to prior works, CaMDN reduces the memory access by 33.4% on average and achieves a model speedup of up to 2.56× (1.88× on average).