2023/10/25 by Beltiukov, Sylee, Satyandra Guthula, Guthula, Satyandra +10 · 8 citations
Computer Science · #Advanced Malware Detection Techniques #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Internet Traffic Analysis and Secure E-voting #Network Security and Intrusion Detection #Networking and Internet Architecture (cs.NI)
paper · pdf · doi:10.48550/arxiv.2310.17025
openalex publication_date 2023/10/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31
Network foundation models promise reusable representations for diverse traffic analysis tasks, but recent diagnostic works have revealed fundamental problems: models exploit dataset shortcuts rather than learning genuine traffic patterns, produce collapsed embedding spaces, and fail to capture the exogenous network conditions that shape real-world behavior. We translate these diagnostic insights into four concrete design principles: protocol-aware tokenization, operational context embedding, burst-flow hierarchical attention, and privacy-by-construction input design, and build netFound, a network foundation model whose architecture is motivated by this failure analysis. We pretrain netFound on a billion-token-scale corpus over 5000 GPU hours, and demonstrate that it produces high-quality representations with lower anisotropy, significantly higher alignment with domain-expert features, and an F1 of 0.95 on exogenous context discrimination where existing state-of-the-art models score below 0.62, while preserving privacy by excluding payload and IP addresses. netFound demonstrates significant improvements in frozen-encoder evaluation, showing that pretrained embeddings themselves carry useful structure, and remains the top performer across all benchmarks in end-to-end fine-tuned settings. We release full open-source code, weights for three model sizes on HuggingFace, a containerized pipeline from raw PCAPs to downstream inference, and the full 4.2 billion flows pretraining dataset to facilitate reproducibility and further research.