2016/08/26 by Ting Chen, Chen, Ting, Lu‐An Tang +7 · 4 citations
Computer Science · Medicine · #Anomaly Detection Techniques and Applications #Cryptography and Security (cs.CR) #Data-Driven Disease Surveillance #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Network Security and Intrusion Detection #Time Series Analysis and Forecasting
paper · pdf · doi:10.48550/arxiv.1608.07502
openalex publication_date 2016/08/26 · openalex created_date 2019/07/30 · openalex updated_date 2026/07/28
Anomaly detection plays an important role in modern data-driven security\napplications, such as detecting suspicious access to a socket from a process.\nIn many cases, such events can be described as a collection of categorical\nvalues that are considered as entities of different types, which we call\nheterogeneous categorical events. Due to the lack of intrinsic distance\nmeasures among entities, and the exponentially large event space, most existing\nwork relies heavily on heuristics to calculate abnormal scores for events.\nDifferent from previous work, we propose a principled and unified probabilistic\nmodel APE (Anomaly detection via Probabilistic pairwise interaction and Entity\nembedding) that directly models the likelihood of events. In this model, we\nembed entities into a common latent space using their observed co-occurrence in\ndifferent events. More specifically, we first model the compatibility of each\npair of entities according to their embeddings. Then we utilize the weighted\npairwise interactions of different entity types to define the event\nprobability. Using Noise-Contrastive Estimation with "context-dependent" noise\ndistribution, our model can be learned efficiently regardless of the large\nevent space. Experimental results on real enterprise surveillance data show\nthat our methods can accurately detect abnormal events compared to other\nstate-of-the-art abnormal detection techniques.\n