2022/01/17 by Lange, Elke B., Fink, Lauren, Saxena, Shreshth
#blink detection #computer vision #deep learning #eye tracking #eye-tracking #gaze estimation #in the wild #online #online experiment #online study #remote #task battery #webcam
paper · doi:10.17605/osf.io/qh8kx
Eye-tracking studies are mostly conducted with high-end equipment in restricted lab settings. However researchers in the field of computer vision have proposed several deep learning methods to perform eye tracking with regular webcams. Although the proposed models have remarkably improved over recent years, they are yet to be tested for their application in online studies that require accurate and robust performance. Moreover, these models are often trained and evaluated on different datasets and performance measures, which makes it hard to benchmark state-of-the-art performance. We selected three pre-trained models (Zhang et al, 2015 [MPIIGaze], Park et al, 2019 [FAZE], Zhang et al, 2020 [ETH-XGaze]) that were trained to perform in unrestricted environments (similar to an online experiment setting) and will have online participants perform a battery of standard eye tracking tasks (Ehinger et al, 2019, Judd et al, 2009) to evaluate the performance of these models for different eye movements. In addition we also evaluate the classification accuracy of two other methods (Soukupová & Cech, 2016 and Cortacero et al, 2019) for detecting blinks in the recordings. We are interested in 1) how accurate eye-tracking during online studies can be, 2) whether accuracies can be improved via different calibration methods and frequency of recalibration throughout the experiment, 3) how accuracy will differ between different tasks with different affordances (e.g., fixation accuracy on small targets versus zones, blink detection), 4) which, if any, of the models outperform the others, and why. By answering such questions, we hope to illuminate what types of tasks and eye movement measures are most robustly detected in online experiments and to provide the research community with advice and best practices for conducting eye-tracking studies online, via webcam.