2019/06/27 by Evan Racah, Christopher Pal, Racah, Evan +1 · 1 citation
Computer Science · #Artificial Intelligence in Games #Reinforcement Learning in Robotics #Multimodal Machine Learning Applications
paper · pdf · doi:10.48550/arxiv.1906.11951
Self-supervised methods, wherein an agent learns representations solely by\nobserving the results of its actions, become crucial in environments which do\nnot provide a dense reward signal or have labels. In most cases, such methods\nare used for pretraining or auxiliary tasks for "downstream" tasks, such as\ncontrol, exploration, or imitation learning. However, it is not clear which\nmethod's representations best capture meaningful features of the environment,\nand which are best suited for which types of environments. We present a\nsmall-scale study of self-supervised methods on two visual environments: Flappy\nBird and Sonic The Hedgehog. In particular, we quantitatively evaluate the\nrepresentations learned from these tasks in two contexts: a) the extent to\nwhich the representations capture true state information of the agent and b)\nhow generalizable these representations are to novel situations, like new\nlevels and textures. Lastly, we evaluate these self-supervised features by\nvisualizing which parts of the environment they focus on. Our results show that\nthe utility of the representations is highly dependent on the visuals and\ndynamics of the environment.\n