ResearchDoom and CocoDoom: Learning Computer Vision with Games

2016/10/07 by Aravindh Mahendran, A. Mahendran, Hakan Bilen +9 · 2 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #cs.CV

paper · pdf · doi:10.48550/arxiv.1610.02431

arxiv created 2016/10/07 · openalex publication_date 2016/10/07 · arxiv updated 2016/10/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this short note we introduce ResearchDoom, an implementation of the Doom first-person shooter that can extract detailed metadata from the game. We also introduce the CocoDoom dataset, a collection of pre-recorded data extracted from Doom gaming sessions along with annotations in the MS Coco format. ResearchDoom and CocoDoom can be used to train and evaluate a variety of computer vision methods such as object recognition, detection and segmentation at the level of instances and categories, tracking, ego-motion estimation, monocular depth estimation and scene segmentation. The code and data are available at http://www.robots.ox.ac.uk/~vgg/research/researchdoom.

Citations

• Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
• Playing Atari with Deep Reinforcement Learning
• Synthetic Data for Text Localisation in Natural Images

Cited by

• Experience enrichment based task independent reward model
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks

Related

• ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning
• Play and Learn: Using Video Games to Train Computer Vision Models
• Playing for Data: Ground Truth from Computer Games
• Task-Relevant Object Discovery and Categorization for Playing First-person Shooter Games
• VideoGameBench: Can Vision-Language Models complete popular video games?