Bridging Subjective and Objective QoE: Operator-Level Aggregation Using LLM-Based Comment Analysis and Network MOS Comparison

2025/06/01 by Parsa Hassani Shariat Panahi, Panahi, Parsa Hassani Shariat, Amir Hossein Jalilvand +3
Computer Science · #Advanced Text Analysis Techniques #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Networking and Internet Architecture (cs.NI) #Sentiment Analysis and Opinion Mining

paper · pdf · doi:10.48550/arxiv.2506.00924

openalex publication_date 2025/06/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper introduces a dual-layer framework for network operator-side quality of experience (QoE) assessment that integrates both objective network modeling and subjective user perception extracted from live-streaming platforms. On the objective side, we develop a machine learning model trained on mean opinion scores (MOS) computed via the ITU-T P.1203 reference implementation, allowing accurate prediction of user-perceived video quality using only network parameters such as packet loss, delay, jitter, and throughput without reliance on video content or client-side instrumentation. On the subjective side, we present a semantic filtering and scoring pipeline that processes user comments from live streams to extract performance-related feedback. A large language model is used to assign scalar MOS scores to filtered comments in a deterministic and reproducible manner. To support scalable and interpretable analysis, we construct a labeled dataset of 47,894 live-stream comments, of which about 34,000 are identified as QoE-relevant through multi-layer semantic filtering. Each comment is enriched with simulated Internet Service Provider attribution and temporally aligned using synthetic timestamps in 5-min intervals. The resulting dataset enables operator-level aggregation and time-series analysis of user-perceived quality. A delta MOS metric is proposed to measure each Internet service provider's deviation from platform-wide sentiment, allowing detection of localized degradations even in the absence of direct network telemetry. A controlled outage simulation confirms the framework's effectiveness in identifying service disruptions through comment-based trends alone. The system provides each operator with its own subjective MOS and the global platform average per interval, enabling real-time interpretation of performance deviations and comparison with objective network-based QoE estimates.

Citations

• A Survey of Uncertainty Estimation Methods on Large Language Models
• Enhancing Multimodal Affective Analysis with Learned Live Comment Features
• Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation
• Large Language Model Aided QoS Prediction for Service Recommendation
• Enhancing Quality of Experience in Telecommunication Networks: A Review of Frameworks and Machine Learning Algorithms
• The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values
• Language Models are Few-Shot Learners

Related

• From QoS Distributions to QoE Distributions: a System's Perspective
• Enhancing Quality of Experience in Telecommunication Networks: A Review of Frameworks and Machine Learning Algorithms
• Satellite Streaming Video QoE Prediction: A Real-World Subjective Database and Network-Level Prediction Models
• Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming
• Towards Perceptually Optimized End-to-end Adaptive Video Streaming