2020/02/26 by Shihao Jiang, Dylan Campbell, Jiang, Shihao +8 · 1 citation
Computer Science · Mathematics · #Advanced Image Processing Techniques #Advanced Vision and Imaging #Algorithm #Artificial intelligence #Brightness #Camera auto-calibration #Camera resectioning #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Epipolar geometry #FOS: Computer and information sciences #Flow (mathematics) #Geometry #Image (mathematics) #Mathematical analysis #Mathematics #Motion (physics) #Motion estimation #Motion field #Optical flow #Optical measurement and interference techniques #Optics #Optimization problem #Physics #Smoothness #cs.CV
paper · pdf · doi:10.48550/arxiv.2002.11826
published in arXiv (Cornell University) (Cornell University)
arxiv created 2020/02/26 · openalex publication_date 2020/02/26 · arxiv updated 2020/02/28 · openalex created_date 2020/03/06 · openalex updated_date 2026/08/06
We address the problem of joint optical flow and camera motion estimation in rigid scenes by incorporating geometric constraints into an unsupervised deep learning framework. Unlike existing approaches which rely on brightness constancy and local smoothness for optical flow estimation, we exploit the global relationship between optical flow and camera motion using epipolar geometry. In particular, we formulate the prediction of optical flow and camera motion as a bi-level optimization problem, consisting of an upper-level problem to estimate the flow that conforms to the predicted camera motion, and a lower-level problem to estimate the camera motion given the predicted optical flow. We use implicit differentiation to enable back-propagation through the lower-level geometric optimization layer independent of its implementation, allowing end-to-end training of the network. With globally-enforced geometric constraints, we are able to improve the quality of the estimated optical flow in challenging scenarios and obtain better camera motion estimates compared to other unsupervised learning methods.