2024/01/18 by Ioannis Mavroudopoulos, Mavroudopoulos, Ioannis, Anastasios Gounaris +1
Computer Science · Decision Sciences · #Advanced Database Systems and Queries #Data Mining Algorithms and Applications #Data Quality and Management #Databases (cs.DB) #FOS: Computer and information sciences
paper · pdf · doi:10.48550/arxiv.2401.09960
openalex publication_date 2024/01/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Detecting complex patterns in large volumes of event logs has diverse applications in various domains, such as business processes and fraud detection. Existing systems like ELK are commonly used to tackle this challenge, but their performance deteriorates for large patterns, while they suffer from limitations in terms of expressiveness and explanatory capabilities for their responses. In this work, we propose a solution that integrates a Complex Event Processing (CEP) engine into a broader query processsor on top of a decoupled storage infrastructure containing inverted indices of log events. The results demonstrate that our system excels in scalability and robustness, particularly in handling complex queries. Notably, our proposed system delivers responses for large complex patterns within seconds, while ELK experiences timeouts after 10 minutes. It also significantly outperforms solutions relying on FlinkCEP and executing MATCHRECOGNIZE SQL queries.