Adaptive Edge-Cloud Inference for Speech-to-Action Systems Using ASR and Large Language Models

2025/12/14 by Mohammad Jalili Torkamani, Torkamani, Mohammad Jalili, Israt Zarin +1
Computer Science · Neuroscience · #Artificial Intelligence (cs.AI) #C.2.4 #C.3 #EEG and Brain-Computer Interfaces #FOS: Computer and information sciences #H.5.2 #I.2.7 #IoT and Edge/Fog Computing #Sound (cs.SD) #Speech Recognition and Synthesis

paper · pdf · doi:10.48550/arxiv.2512.12769

openalex publication_date 2025/12/14 · openalex created_date 2025/12/17 · openalex updated_date 2026/07/28

Abstract

Voice-based interaction has emerged as a natural and intuitive modality for controlling IoT devices. However, speech-driven edge devices face a fundamental trade-off between cloud-based solutions, which offer stronger language understanding capabilities at the cost of latency, connectivity dependence, and privacy concerns, and edge-based solutions, which provide low latency and improved privacy but are limited by computational constraints. This paper presents ASTA, an adaptive speech-to-action solution that dynamically routes voice commands between edge and cloud inference to balance performance and system resource utilization. ASTA integrates on-device automatic speech recognition and lightweight offline language-model inference with cloud-based LLM processing, guided by real-time system metrics such as CPU workload, device temperature, and network latency. A metric-aware routing mechanism selects the inference path at runtime, while a rule-based command validation and repair component ensures successful end-to-end command execution. We implemented our solution on an NVIDIA Jetson-based edge platform and evaluated it using a diverse dataset of 80 spoken commands. Experimental results show that ASTA successfully routes all input commands for execution, achieving a balanced distribution between online and offline inference. The system attains an ASR accuracy of 62.5% and generates executable commands without repair for only 47.5% of inputs, highlighting the importance of the repair mechanism in improving robustness. These results suggest that adaptive edge-cloud orchestration is a viable approach for resilient and resource-aware voice-controlled IoT systems.

Citations

• Bayes-Split-Edge: Bayesian Optimization for Constrained Collaborative Inference in Wireless Edge Systems
• HySim-LLM: Embedding-Weighted Fine-Tuning Bounds and Manifold Denoising for Domain-Adapted LLMs
• AutoPK: Leveraging LLMs and a Hybrid Similarity Metric for Advanced Retrieval of Pharmacokinetic Data from Complex Tables and Documents
• Efficient Routing of Inference Requests across LLM Instances in Cloud-Edge Computing
• Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Models
• Sanitizing Manufacturing Dataset Labels Using Vision-Language Models
• LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM
• Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens
• Tiny-Align: Bridging Automatic Speech Recognition and Large Language Model on the Edge
• VoiceBench: Benchmarking LLM-Based Voice Assistants
• Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition
• XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model
• Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization
• Transforming LLMs into Cross-modal and Cross-lingual Retrieval Systems
• On decoder-only architecture for speech-to-text and large language model integration
• Taxonomic Classification of IoT Smart Home Voice Control
• SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model
• Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
• The Emergence of Edge Computing

Related

• Mobile Edge Intelligence for Large Language Models: A Contemporary Survey
• Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Models
• Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models
• Emotion Filtering at the Edge
• Optimizing Speech Recognition For The Edge