SaltiNet: Scan-path Prediction on 360 Degree Images using Saliency\n Volumes

2017/07/11 by Marc Assens, Assens, Marc, Kevin McGuinness +5 · 1 citation
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimedia (cs.MM) #Visual Attention and Saliency Detection

paper · pdf · doi:10.48550/arxiv.1707.03123

openalex publication_date 2017/07/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce SaltiNet, a deep neural network for scanpath prediction trained\non 360-degree images. The model is based on a temporal-aware novel\nrepresentation of saliency information named the saliency volume. The first\npart of the network consists of a model trained to generate saliency volumes,\nwhose parameters are fit by back-propagation computed from a binary cross\nentropy (BCE) loss over downsampled versions of the saliency volumes. Sampling\nstrategies over these volumes are used to generate scanpaths over the\n360-degree images. Our experiments show the advantages of using saliency\nvolumes, and how they can be used for related tasks. Our source code and\ntrained models available at\nhttps://github.com/massens/saliency-360salient-2017.\n

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking
• ImageNet classification with deep convolutional neural networks
• DeepGaze II: Reading fixations from deep features trained on object recognition
• A Deep Spatial Contextual Long-term Recurrent Convolutional Network for Saliency Detection

Cited by

• DiffEye: Diffusion-Based Continuous Eye-Tracking Data Generation Conditioned on Natural Images

Related

• SalyPath360: Saliency and Scanpath Prediction Framework for Omnidirectional Images
• ScanDTM: A Novel Dual-Temporal Modulation Scanpath Prediction Model for Omnidirectional Images
• A Simple and efficient deep Scanpath Prediction
• PathGAN: Visual Scanpath Prediction with Generative Adversarial Networks
• SALYPATH: A Deep-Based Architecture for visual attention prediction