2021/08/20 by Andrey Sheka, Sheka, Andrey, Victor Samun +1
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Face recognition and analysis #Human Pose and Action Recognition #cs.CV
paper · pdf · doi:10.48550/arxiv.2108.09183
openalex publication_date 2021/08/20 · arxiv created 2022/01/28 · arxiv updated 2022/01/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We propose a response-based method of knowledge distillation (KD) for the head pose estimation problem. A student model trained by the proposed KD achieves results better than a teacher model, which is atypical for the response-based method. Our method consists of two stages. In the first stage, we trained the base neural network (NN), which has one regression head and four regression via classification (RvC) heads. We build the convolutional ensemble over the base NN using offsets of face bounding boxes over a regular grid. In the second stage, we perform KD from the convolutional ensemble into the final NN with one RvC head. The KD improves the results by an average of 7.7% compared to base NN. This feature makes it possible to use KD as a booster and effectively train deeper NNs. NNs trained by our KD method partially improved the state-of-the-art results. KD-ResNet152 has the best results, and KD-ResNet18 has a better result on the AFLW2000 dataset than any previous method.We have made publicly available trained NNs and face bounding boxes for the 300W-LP, AFLW, AFLW2000, and BIWI datasets.Our method potentially can be effective for other regression problems.