A Word is Worth 4-bit: Efficient Log Parsing with Binary Coded Decimal Recognition

2025/06/01 by Prerak Srivastava, Srivastava, Prerak, Giulio Corallo +3
Computer Science · #Advanced Image and Video Retrieval Techniques #Algorithms and Data Compression #Computation and Language (cs.CL) #FOS: Computer and information sciences #Image Retrieval and Classification Techniques #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2506.01147

openalex publication_date 2025/06/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

System-generated logs are typically converted into categorical log templates through parsing. These templates are crucial for generating actionable insights in various downstream tasks. However, existing parsers often fail to capture fine-grained template details, leading to suboptimal accuracy and reduced utility in downstream tasks requiring precise pattern identification. We propose a character-level log parser utilizing a novel neural architecture that aggregates character embeddings. Our approach estimates a sequence of binary-coded decimals to achieve highly granular log templates extraction. Our low-resource character-level parser, tested on revised Loghub-2k and a manually annotated industrial dataset, matches LLM-based parsers in accuracy while outperforming semantic parsers in efficiency.

Citations

• Understanding the Performance and Estimating the Cost of LLM Fine-Tuning
• LogPTR: Variable-Aware Log Parsing with Pointer Network
• LILAC: Log Parsing using LLMs with Adaptive Parsing Cache
• A Large-Scale Evaluation for Log Parsing Techniques: How Far Are We?
• Did We Miss Something Important? Studying and Exploring Variable-Aware Log Abstraction
• Log Parsing with Prompt-based Few-shot Learning
• SemParser: A Semantic Parser for Log Analysis
• Calibrate Before Use: Improving Few-Shot Performance of Language Models
• Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics
• Self-Supervised Log Parsing
• TENER: Adapting Transformer Encoder for Named Entity Recognition
• Three scenarios for continual learning
• Length Matters: Clustering System Log Messages using Length of Words
• End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
• Adam: A Method for Stochastic Optimization
• An Introduction to Conditional Random Fields

Related

• LogParser-LLM: Advancing Efficient Log Parsing with Large Language Models
• LILAC: Log Parsing using LLMs with Adaptive Parsing Cache
• Stronger, Cheaper and Demonstration-Free Log Parsing with LLMs
• Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction
• Prompting for Automatic Log Template Extraction