Research Group on Visual Computation

Description

Pose estimation is a fundamental building block of various vision applications, e.g. visual odometry, image based localization and navigation, fusion, and augmented reality. Herein, we introduce our novel proposed solutions for pose estimation based on matching 2D-3D lines, which are common in urban environments. The proposed methods compute either the absolute pose of a camera system (relative poses between cameras are considered to be known) or the absolute and the relative poses as well, relying on a known vertical direction. Since modern cameras are frequently equipped with various locations and orientation sensors, the vertical direction (e.g. a gravity vector) can be available in many applications.

Video 1. Sample of the extracted lines used for testing our pose estimation algorithms. The corresponding 3D lines are shown with the same color as the camera it was detected on.

Therefore we formulate the problem in four different ways:

1. Absolute and Relative Pose Solver

   
   we propose a novel solution that estimates the absolute and relative pose of a multi-view camera system from line correspondences with known vertical direction. First, a direct solver is proposed for the minimal case (two camera and 6 line pairs) suitable for hypothesis testing like RANSAC. Then, using the same equations, a least squares solver is proposed which can be used for n > 6 lines as well as for N > 2 camera systems [1].
   



Video 4. shows the 3D point cloud including the used lines (red lines), the ground truth camera poses (green) and the estimated poses with the absolute and relative pose algorithm (red). The middle camera (green with blue edges) was set as the reference camera, the relative camera position was set from right to left




2. gPnLup

   
   we propose a novel solution to the gPnL problem with known vertical direction. The only assumption about our generalized camera is that projection rays of a 3D line fall into coplanar subsets yielding a pencil of projection planes. Important special cases of such a camera include stereo and multiview perspective camera systems, perspective camera moving along a trajectory , as well as other non-perspective cameras with central omnidirectional or orthographic projection. The algorithm can be used as a minimal gP3L solver with 3 line correspondences suitable for hypothesis testing like RANSAC. Furthermore, the same algorithm can be used without reformulation for n > 3 lines as well as for classical single-view PnL problems [2].
   
   

   Video 2. Lidar laser scan for testing our pose estimation algorithms with a 3-perspective-1-omnidirectional multi-view camera system. On the Lidar scan red dots are the estimated positions and green dots are the real location of the markers in metric 3D space.

   Video 3. Lidar laser scan for testing our pose estimation algorithms solver with a 3-perspective camera system. On the Lidar scan, red dots are the estimated positions of our minimal solver, blue dots are the estimated positions of NP3L [G.H.Lee, ECCV 2016], and green dots are the real location of the markers in the 3D metric space.





3. NPnLupL and NPnLupC

   
   we propose two new solutions to the NPnL problem with known vertical direction linear and a cubic one. Both algorithms can be used as a minimal NP3L solver with 3 line correspondences suitable for hypothesis testing like RANSAC. The methods work for single- and multi-view camera systems without reformulation. The minimal number of line correspondences has been discussed for various common camera configurations [3].
   





Figure 2. Lidar laser scan for testing our pose estimation algorithms with 4-camera system. 2D detected lines are shown next to the 3D point cloud which colors are the same to their corresponding camera.






Figure 3. Lidar laser scan for testing our pose estimation algorithms with 3-camera system. 2D detected lines are shown next to the 3D point cloud which colors are the same to their corresponding camera.