vix.ing · top · new · best · stats · spec

EdgeCoInfer: Hierarchical Collaborative Inference for On-Device Multimodal Large Models

2026/07/19 by Lin Tan, David K. Y. Yau, Songtao Guo
#cs.DC

paper · pdf

Abstract

Modern mobile applications predominantly execute concurrent Multimodal Large Language Models (MLLMs) to provide ubiquitous intelligence. However, satisfying this demand within edge environments faces significant challenges due to multi-task concurrency and strictly coupled hard constraints. To address these issues, we propose EdgeCoInfer, a framework enabling granularity-adaptive deployment by co-optimizing inter-model functional module sharing and intra-model fine-grained partitioning. We solve the underlying Mixed-Integer Non-Linear Programming (MINLP) problem via a Hybrid Evolutionary Hierarchical Reinforcement Learning (HE-HRL) paradigm, which synchronizes a Genetic Algorithm (GA) for discrete model placement with a Soft Actor-Critic (SAC) agent for continuous resource allocation. To navigate the sparse feasible region, we introduce a feasibility-guided constructive execution mechanism, integrating a constructive cut-step decoder with pre-act pruning and a two-phase curriculum strategy for stable adaptation. Experimental results demonstrate that EdgeCoInfer ensures a 100% task completion rate in high-concurrency scenarios, achieving a 76% reduction in system cost and 71.88% memory savings compared to state-of-the-art baselines.

Citations

Related