2026/04/27 by Dominik Winecki, Arnab Nandi · 1 voice
Computer Science · Social Sciences · #Abstraction #Compiler #Data Visualization and Analytics #Data visualization #Dataflow #Grammar #Multimedia Communication and Technology #Semantics (computer science) #Syntax #Video Analysis and Summarization #Visualization #cs.HC
paper · pdf · doi:10.48550/arxiv.2604.24958
openalex publication_date 2026/04/27 · arxiv published 2026/04/27 · arxiv updated 2026/04/27 · openalex created_date 2026/04/30 · openalex updated_date 2026/07/28
Video data is increasingly used alongside conventional data for interactive data exploration, necessitating interfaces for exploring and presenting mixed-modality data. However, integrating video into visualizations remains difficult due to its distinct paradigms and inherent performance challenges. We identify three classes of video data visualization - synchronization, annotation, and transformation - and integrate them into the Vega declarative grammar. We show that these abstractions enable high-performance implementation. To reconcile Vega's instantaneous dataflow with video player state, we introduce a split-signal architecture that preserves declarative semantics while masking video update delays. We detect continuous scrubbing interactions at compile time to apply encoding-aware optimizations that improve responsiveness by up to 4x. We also repurpose VOD protocols to transform videos in real time, delivering sub-200ms updates even on multi-hour-long compilations. These contributions enable seamless integration of conventional and video data visualization.