2020/05/01 by Ulrik Günther, Günther, Ulrik, Kyle Harrington +5
Biochemistry, Genetics and Molecular Biology · Computer Science · Medicine · #Cell Image Analysis Techniques #FOS: Computer and information sciences #Gaze Tracking and Assistive Technology #Graphics (cs.GR) #Human-Computer Interaction (cs.HC) #Retinal Imaging and Analysis
paper · pdf · doi:10.48550/arxiv.2005.00387
openalex publication_date 2020/05/01 · openalex created_date 2022/07/26 · openalex updated_date 2026/07/28
We present Bionic Tracking, a novel method for solving biological cell\ntracking problems with eye tracking in virtual reality using commodity\nhardware. Using gaze data, and especially smooth pursuit eye movements, we are\nable to track cells in time series of 3D volumetric datasets. The problem of\ntracking cells is ubiquitous in developmental biology, where large volumetric\nmicroscopy datasets are acquired on a daily basis, often comprising hundreds or\nthousands of time points that span hours or days. The image data, however, is\nonly a means to an end, and scientists are often interested in the\nreconstruction of cell trajectories and cell lineage trees. Reliably tracking\ncells in crowded three-dimensional space over many timepoints remains an open\nproblem, and many current approaches rely on tedious manual annotation and\ncuration. In our Bionic Tracking approach, we substitute the usual 2D\npoint-and-click annotation to track cells with eye tracking in a virtual\nreality headset, where users simply have to follow a cell with their eyes in 3D\nspace in order to track it. We detail the interaction design of our approach\nand explain the graph-based algorithm used to connect different time points,\nalso taking occlusion and user distraction into account. We demonstrate our\ncell tracking method using the example of two different biological datasets.\nFinally, we report on a user study with seven cell tracking experts,\ndemonstrating the benefits of our approach over manual point-and-click\ntracking.\n