T-VEC: A Telecom-Specific Vectorization Model with Enhanced Semantic Understanding via Deep Triplet Loss Fine-Tuning

2025/04/23 by Vignesh Ethiraj, Ethiraj, Vignesh, A Clifton David +5 · 2 citations
Social Sciences · #68T50 #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2504.16460

openalex publication_date 2025/04/23 · openalex created_date 2025/10/18 · openalex updated_date 2026/07/28

Abstract

The specialized vocabulary and nuanced concepts of the telecommunications industry pose persistent challenges for standard Natural Language Processing (NLP) models. Generic embedding models often struggle to represent telecom-specific semantics, limiting their utility in retrieval and downstream tasks. We present T-VEC (Telecom Vectorization Model), a domain-adapted embedding model fine-tuned from the gte-Qwen2-1.5B-instruct backbone using a triplet loss objective. Fine-tuning was performed on T-Embed, a high-quality, large-scale dataset covering diverse telecom concepts, standards, and operational scenarios. Although T-Embed contains some proprietary material and cannot be fully released, we open source 75% of the dataset to support continued research in domain-specific representation learning. On a custom benchmark comprising 1500 query-passage pairs from IETF RFCs and vendor manuals, T-VEC surpasses MPNet, BGE, Jina and E5, demonstrating superior domain grounding and semantic precision in telecom-specific retrieval. Embedding visualizations further showcase tight clustering of telecom-relevant concepts. We release T-VEC and its tokenizer to support semantically faithful NLP applications within the telecom domain.

Citations

• Greenback Bears and Fiscal Hawks: Finance is a Jungle and Text Embeddings Must Adapt
• PhysBERT: A Text Embedding Model for Physics Scientific Literature
• Investigating Distributions of Telecom Adapted Sentence Embeddings for Document Retrieval
• Qwen Technical Report
• Towards General Text Embeddings with Multi-stage Contrastive Learning
• One Embedder, Any Task: Instruction-Finetuned Text Embeddings
• Text Embeddings by Weakly-Supervised Contrastive Pre-training
• MTEB: Massive Text Embedding Benchmark
• Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
• MPNet: Masked and Permuted Pre-training for Language Understanding
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Publicly Available Clinical BERT Embeddings
• SciBERT: A Pretrained Language Model for Scientific Text
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Learning Domain-Specific Word Embeddings from Sparse Cybersecurity Texts
• SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• Graph-Symbolic Policy Enforcement and Control (G-SPEC): A Neuro-Symbolic Framework for Safe Agentic AI in 5G Autonomous Networks
• Toward Low-Latency End-to-End Voice Agents for Telecommunications Using Streaming ASR, Quantized LLMs, and Real-Time TTS

Related

• MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications
• Telecom Language Models: Must They Be Large?
• TelcoLM: collecting data, adapting, and benchmarking language models for the telecommunication domain
• Tele-LLMs: A Series of Specialized Large Language Models for Telecommunications
• Enhancing Large Language Models (LLMs) for Telecommunications using Knowledge Graphs and Retrieval-Augmented Generation