2023/03/21 by Girish Narayanswamy, Yujia Liu, Narayanswamy, Girish +11 · 6 citations
Engineering · Medicine · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Non-Invasive Vital Sign Monitoring #Optical Imaging and Spectroscopy Techniques
paper · pdf · doi:10.48550/arxiv.2303.11573
openalex publication_date 2023/03/21 · openalex created_date 2023/03/23 · openalex updated_date 2026/07/28
Understanding of human visual perception has historically inspired the design of computer vision architectures. As an example, perception occurs at different scales both spatially and temporally, suggesting that the extraction of salient visual information may be made more effective by paying attention to specific features at varying scales. Visual changes in the body due to physiological processes also occur at different scales and with modality-specific characteristic properties. Inspired by this, we present BigSmall, an efficient architecture for physiological and behavioral measurement. We present the first joint camera-based facial action, cardiac, and pulmonary measurement model. We propose a multi-branch network with wrapping temporal shift modules that yields both accuracy and efficiency gains. We observe that fusing low-level features leads to suboptimal performance, but that fusing high level features enables efficiency gains with negligible loss in accuracy. Experimental results demonstrate that BigSmall significantly reduces the computational costs. Furthermore, compared to existing task-specific models, BigSmall achieves comparable or better results on multiple physiological measurement tasks simultaneously with a unified model.