Deep Reinforcement Learning for Dynamic Sensing and Communications

2025/09/23 by Zakeri, Abolfazl, Nguyen, Nhan Thanh, Alkhateeb, Ahmed +1 · 1 citation
#FOS: Electrical engineering #Signal Processing (eess.SP) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2509.19130

Abstract

Environmental sensing can significantly enhance mmWave communications by assisting beam training, yet its benefits must be balanced against the associated sensing costs. To this end, we propose a unified machine learning framework that dynamically determines when to sense and leverages sensory data for beam prediction. Specifically, we formulate a joint sensing and beamforming problem that maximizes the average signal-to-noise ratio under an average sensing budget. Lyapunov optimization is employed to enforce the sensing constraint, while a deep Q-Network determines the sensing slots. A pretrained deep neural network then maps the sensing data to optimal beams in the codebook. Simulations based on the real-world DeepSense dataset demonstrate that the proposed approach substantially reduces sensing overhead while maintaining satisfactory communications performance.

Citations

• M2BeamLLM: Multimodal Sensing-empowered mmWave Beam Prediction with Large Language Models
• Constrained Multimodal Sensing-Aided Communications: A Dynamic Beamforming Design
• Multimodal Deep Learning-Empowered Beam Prediction in Future THz ISAC Systems
• Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Framework
• BeamLLM: Vision-Empowered mmWave Beam Prediction with Large Language Models
• Harnessing Multimodal Sensing for Multi-user Beamforming in mmWave Systems
• Environment Semantic Communication: Enabling Distributed Sensing Aided Networks
• DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset
• Minimizing the AoI in Resource-Constrained Multi-Source Relaying Systems: Dynamic and Learning-based Scheduling
• LiDAR Aided Future Beam Prediction in Real-World Millimeter Wave V2I Communications
• Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets
• Leveraging Sensing at the Infrastructure for mmWave Communication
• Leveraging Sensing at the Infrastructure for mmWave Communication

Cited by

• AoI-Aware Machine Learning for Constrained Multimodal Sensing-Aided Communications

Related

• AoI-Aware Machine Learning for Constrained Multimodal Sensing-Aided Communications
• Constrained Multimodal Sensing-Aided Communications: A Dynamic Beamforming Design
• Multi-User Beamforming with Deep Reinforcement Learning in Sensing-Aided Communication
• Active Sensing for Communications by Learning
• Harnessing Multimodal Sensing for Multi-user Beamforming in mmWave Systems